Smoke testing

It is a gate, not a level of testing

Unit, integration and end-to-end are levels: they describe how much of the system a test touches. Smoke is not one of those. It is a purpose, and it can be built out of tests at any level — an API call, a browser flow, a health probe, whatever answers the question fastest. The question is always the same one, and it is about the build rather than about the feature: has this deployment come up in a state where further work makes sense?

The name comes from hardware. You assemble a board, apply power, and look for smoke — you are not measuring anything, you are checking that the thing has not obviously destroyed itself before you invest in measuring it. Software borrowed the metaphor exactly. In continuous delivery the same suite is often called a build verification test, and the two names describe the same object: a small fixed set of checks that runs immediately after a build is deployed and produces one binary verdict.

What belongs in the suite, and the rule that keeps it small

There is one selection rule and it is worth stating as a sentence you can apply to any candidate check: include it only if a failure means nobody can do useful work on this build. Not "this is important". Not "this broke once". If the check fails and a tester could still spend a productive afternoon, the check belongs in the regression suite instead. Applied honestly, that rule usually produces between five and fifteen checks for a normal product.

The time budget is not a nice-to-have; it is the mechanism. A gate only works if the answer arrives before the decision is made, and the decision here is "should the next person start". Under five minutes is a good target for most products and under ten is a defensible ceiling. Beyond that people stop waiting for the result, start work in parallel, and the gate becomes a report that arrives after the damage.

Where it runs — and it runs in more than one place

Most teams have one smoke suite and run it in one place, usually after deploying to a test environment. The suite is more useful than that, because the same set of checks answers a slightly different question in each environment it runs in, and the cost of running it again is minutes.

• After a deploy to test

  The classic position. Answers "is this build worth handing to QA", and its result should gate the handover rather than accompany it as a note.

• After a deploy to staging

  Answers a different question: is the ENVIRONMENT correctly configured. Most failures here are secrets, connection strings and DNS rather than code.

• After a deploy to production

  A read-only subset, plus one safe write if you have a test account. This is the check that decides whether an automated rollback fires, so it must be wired to the rollback.

• On a schedule in production

  The same checks every few minutes are synthetic monitoring: they find the breakage that no deploy caused — an expired certificate, a dependency that went down on its own.

Both are worth having and they are not substitutes, because they fail for different reasons. The pre-merge run protects the main branch from obviously broken code; the post-deploy run protects the environment from a deployment that went wrong in a way the code could not predict. If you can only afford one, take the post-deploy run: a broken branch costs one developer an hour, a broken environment costs a team a day.

Smoke, sanity, confirmation — three names people use interchangeably

These terms get swapped freely in job interviews and in team chat, and the confusion is worth clearing up once because the three describe genuinely different moments. All three are small and fast; what differs is what triggers them and what their result licenses you to do.

• Smoke

  Triggered by a build or a deploy. A fixed set, the same every time. Its verdict decides whether anyone else starts work.

• Sanity

  Triggered by a specific change. Narrow and improvised — you look at the area that changed to decide whether deeper testing is worth booking.

• Confirmation

  Triggered by a fix. You re-run the exact steps from the bug report to prove the specific defect is gone. It then becomes a regression case.

What has to be true for the gate to actually work

A smoke suite is easy to write and easy to render meaningless. Four conditions decide which of the two you end up with, and each of them is an organisational choice rather than a technical one.

• It blocks. A red smoke run stops the handover, stops the promotion, or triggers the rollback automatically. A suite whose failure is discussed rather than obeyed is a dashboard.
• It is trusted, which means flakiness is an incident. One check that fails randomly twice a week teaches everyone to re-run the suite instead of reading it, and that habit costs you the real failure.
• It has an owner. Not "QA" as a department — a named person or rotation who is expected to fix a red smoke run today rather than file it.
• It is fast enough that people wait for it. Every minute added is a minute of someone standing still, which is exactly the pressure that makes teams start work before the result arrives.

Flakiness deserves the strongest of those four, because it is where smoke suites usually rot. A flaky check in a regression suite is annoying; a flaky check in a smoke suite is corrosive, since the whole mechanism depends on a red result being believed without discussion. The policy that works is blunt: a check that fails without a real cause is quarantined the same day — removed from the gate, kept running and reported separately — and either fixed within a fixed window or deleted outright. Leaving it in the gate "because it is usually right" is the decision that destroys the gate.

Anti-patterns worth naming

• A suite of HTTP 200 checks. A page that returns 200 with an error rendered inside it passes, which is the exact failure users would report first.
• Smoke run manually, when someone remembers. An unrun gate protects nothing, and the day it is skipped is the day it was needed.
• Results posted to a channel with no consequence attached, so a red run is information rather than an instruction.
• The same suite kept identical in production while it writes real data — one forgotten test order per deploy becomes a finance question by the end of the quarter.
• Treating a green smoke run as a quality statement. It says the build is testable, and nothing whatsoever about whether the feature is correct.

When to use it

Related pages

• QARegression testingRegression testing is the practice of quickly catching a regression — something that used to work stopping working, when nobody meant to touch it. The suite it produces is best understood as institutional memory: a record of everything the system has already been taught, kept in a form that can be replayed. The hard parts are not writing the cases. They are deciding what earns a permanent place, what runs when the whole suite is too slow to run, and what to do about a case that fails for no reason.
• QAEnd-to-end testingAn end-to-end test walks the assembled system the way a person would: through the real interface, across every service, against real configuration, to a real outcome. It is the only level that can prove the parts hold together — and it is also the slowest, the most expensive to maintain and the most likely to fail for reasons that have nothing to do with your change. Both halves of that sentence are true at once, which is why the right number of E2E tests is small rather than zero.
• InfrastructureCI/CDContinuous integration is not a server that runs your tests — it is the habit of merging into one shared branch often enough that conflicts stay small, and the automation exists to make that habit safe. Continuous delivery is the second half: one artifact, built once, promoted through environments without being rebuilt. This page covers how to order the stages, why a flaky test destroys the pipeline's authority, what the deployment strategies actually trade, and the four numbers that tell you whether any of it is working.