End-to-end testing
An end-to-end test walks the assembled system the way a person would: through the real interface, across every service, against real configuration, to a real outcome. It is the only level that can prove the parts hold together — and it is also the slowest, the most expensive to maintain and the most likely to fail for reasons that have nothing to do with your change. Both halves of that sentence are true at once, which is why the right number of E2E tests is small rather than zero.
- It proves
- The parts hold together
- Cost per case
- Highest of any level
- Right number
- Small, not zero
The one thing no other level can prove
Every level below end-to-end tests something in a controlled setting. A unit test replaces its neighbours with fakes. An integration test uses a real database but a stubbed payment provider. A contract test proves two services agree about a message shape without either of them running the other. All of that is valuable and none of it can tell you that the deployed system, with its real routing, its real environment variables, its real session handling and its real front-end build, lets a person complete the thing they came to do.
That gap is not theoretical. The classic escaped defect in a well-tested system is not a wrong calculation — it is a misconfigured redirect, a cookie flag that breaks the session in a real browser, a feature flag that is on in staging and off in production, a CORS header that only matters when the front end is served from its real domain. Every one of those has full green coverage at every level below, because every level below replaced the exact thing that broke.
What the position at the top actually costs
The runtime is the obvious cost and the least important one. The real expense is the distance between a failure and its cause. When a unit test fails you are looking at one function. When an end-to-end test fails you are looking at a browser, a front end, three services, a queue, a database and an environment, and any of them could be responsible — including none of them, because the test itself may simply have run faster than the application.
| Why an E2E test went red | Share of failures, typically | What it actually means |
|---|---|---|
| The application is genuinely broken | The minority | The reason the test exists. Everything below is the tax you pay to keep this signal. |
| The test asserted before the app finished | The largest single cause | A timing bug in the test. Waiting for a condition instead of a duration removes almost all of it. |
| The data was not what the test assumed | Very common | A shared environment problem. Cases that create their own data and clean up stop it. |
| A selector changed in a redesign | Common | Maintenance, not a defect. Stable test ids move this cost from every release to once. |
| A third party was slow or down | Occasional and infuriating | A decision you postponed: which dependencies are part of the test, and which are stubbed at the network boundary. |
Read that table as a budget rather than as a complaint. A suite of sixty end-to-end cases with a two per cent per-case failure rate goes red on roughly two runs in three even when nothing is wrong — which is why "we will just add retries" is the wrong first move, and why the same suite at ten times the size is not ten times more valuable but ten times more likely to be ignored.
Choose journeys, not features
The selection mistake almost every team makes is to write one end-to-end test per feature. Features are how the backlog is organised; they are not how a user moves through the product, and covering them one by one produces hundreds of slow cases that mostly re-prove logic already proved below. Choose journeys instead: a small number of paths that a user actually completes, each one crossing several services because that crossing is the thing being tested.
The money path
Browse, add to basket, pay, receive confirmation. If exactly one E2E test exists, it is this one, because it crosses every service the business depends on.
Getting in and staying in
Sign up, verify, sign in, stay signed in across a page reload. Session and cookie behaviour is invisible to every lower level.
The path across a boundary
An action in the web app that a background worker completes and an email confirms. Nothing below can prove the handoff happens in the real deployment.
The path with a hard rule on it
A permission boundary, an age check, a regulated disclosure. Where being wrong is a legal event rather than a support ticket.
A useful sizing heuristic: one case per journey a business could not survive losing, plus one per integration seam that exists only in the deployed system. For most products that lands somewhere between ten and forty cases. If your number is in the hundreds, the suite is not more thorough than everyone else’s — it is carrying work that belongs two levels down, and it is paying the highest per-case price in the codebase to do it.
Making a small suite trustworthy
A trustworthy end-to-end suite is not a matter of writing better assertions. It comes from removing every source of variation that is not the behaviour under test, and there are five of them worth handling deliberately.
- Wait for a condition, never for a duration. A fixed sleep is either too short on a loaded machine or wasted time on a fast one; waiting for the element, the response or the state to exist is both faster and stable.
- Give every case its own data. It creates the account, the order, the document it needs, and removes them afterwards. A case that depends on a row someone else made is a case that fails on a Tuesday for no reason.
- Select elements by something that exists for testing — a stable test id or an accessible role and name — never by a CSS class or a position in the DOM, both of which change every time a designer touches the page.
- Decide explicitly which third parties are inside the test. A payment sandbox usually is; an email provider, a maps API and an analytics beacon usually are not, and stubbing them at the network boundary keeps the test about your system.
- Capture enough on failure to diagnose without reproducing: a trace, a video, the network log and the browser console. The cost of an E2E failure is the investigation, and artefacts are what shorten it.
Retries deserve a policy rather than a default. Retrying a whole case hides real intermittent bugs, because a race condition in your application looks exactly like a flaky test from the outside — and the retry converts a genuine defect into a green tick. A defensible arrangement: retry at most once, record every retry as a first-class signal, and treat a case that needed a retry twice in a week as quarantined. A suite whose green result depends on retries is not green; it is unmeasured.
Anti-patterns worth naming
- One long case that walks fifteen screens, because when it fails you learn only that something in fifteen screens is broken.
- Cases that depend on running in order, so a failure in the third one cascades into six more and the report describes nothing.
- Running the suite only nightly, so a regression is attributed to a day of merges rather than to a commit.
- A suite that runs against a hand-maintained environment nobody can rebuild, which fails for environment reasons more often than for code ones.
- Sleeps scattered through the code as the standard fix for flakiness, which trades minutes of runtime for a problem that comes back anyway.
When to use it
Use it when
- For the handful of journeys the business could not survive losing, each crossing several services in the deployed system.
- For behaviour that only exists once the real pieces are wired together — sessions, redirects, configuration, front-end build output.
- Against an environment rebuilt from the same artefact that ships, so a pass says something about production.
- Sharded across parallel workers, so the suite is fast enough to run on every merge rather than once a night.
Avoid it when
- For validation rules, error copy, sorting and arithmetic — anything a component or API test proves in milliseconds.
- As the main body of the test suite, where the cost per case is highest and the diagnosis is furthest from the cause.
- Against a shared long-lived environment with hand-made data, which makes every result depend on what a colleague did an hour ago.
- With blanket retries used to keep the pipeline green, because that converts genuine intermittent defects into passing runs.
Found this useful?
Share it with someone who is working on the same problem.