Smoke testing

One shop’s smoke suite on one screen — eleven checks, what each proves, what a red result stops

Nobody opens a page about smoke testing for the history of the word. They open it because a pipeline needs a suite, or because the suite they have takes forty minutes and nobody waits for it. Below is a finished one, for the online shop whose order assistant the AI-testing pages test. It runs after every deploy to every environment. Rows 1–2 ask whether this is the build that was deployed, 3–6 whether it is connected, 7–8 whether a person can get in, and 9–11 whether the flow that earns the money completes. Later sections cite the rows by number.

A gate, not a level of testing

Unit, integration and end-to-end are the layers of the pyramid: they describe how much of the system a test touches. Smoke is not one of them. The ISTQBISTQBInternational Software Testing Qualifications Board — testing certification bodyThe international body behind the most widely taken software-testing certifications, which also publishes a glossary of testing terms. Its definitions are a common reference in interviews and contracts; they fix vocabulary, not how a team has to work.All terms glossary calls it a test type and defines it by its purpose — “to gain sufficient confidence that a test object is ready for planned testing”. The rows above show what that means in practice. Row 1 is a plain HTTPHTTPHyperText Transfer Protocol — the protocol of the webThe protocol browsers and servers talk in: the client sends a method and an address, the server answers with a status code and a body. It keeps no memory between requests — every request stands alone, which is why anything that must be remembered is carried explicitly.All terms request, row 5 an APIAPIApplication Programming Interface — contract between programsThe rules one program follows to call another: which requests it may send, in what shape, and what comes back. To the caller it is a contract — everything behind it stays a black box, which is exactly what lets the other side be rewritten without breaking you.Full article →All terms call, row 9 a browser journey and row 11 a conversation with a model. What they share is the question, and the question is about the build rather than about any one feature.

The term reached software through Microsoft’s daily builds. In 1996 Steve McConnell described the practice in IEEE Software: every file compiled and linked into the product every day, and the result put through a smoke test — “a relatively simple check to see whether the product ‘smokes’ when it runs”. The image is an electrician’s: switch a new board on and watch for smoke before measuring anything. McConnell called the smoke test the sentry against creeping integration problems, and wrote that the daily build has little value without it.

What belongs in it — one rule, and the budget that enforces it

There is one selection rule, and it is worth stating as a sentence you can hold any candidate against: include a check only if its failure means nobody can do useful work on this build. Not “this is important”. Not “this broke once”. If the check fails and a tester could still spend a productive afternoon, it belongs in the regression suite instead. Applied honestly, the rule gives between five and fifteen checks for an ordinary product — the shop has eleven.

How a check passes matters as much as which checks there are. A status code proves that something answered, not that it answered correctly: a page that shows “Something went wrong” inside a perfectly good 200 passes every status check ever written. So each row asserts content — a commit hash, a product name, an order number, the status a tool returned. The column “Passes only if” is the one teams most often leave out.

The time budget is not a preference; it is the mechanism. A gate works only if its answer arrives before the decision it informs, and the decision here is whether the next person starts. Under five minutes is a good target, and under ten is a ceiling you can still defend. Beyond that people stop waiting, start in parallel, and the gate becomes a report that arrives after the damage.

Here the page disagrees with the article that named the practice, and the disagreement is worth keeping in view. McConnell expected the smoke test to grow with the system, from seconds at first to “30 minutes, an hour, or more”. In 1996 that was sound: the build ran once a night, and a smoke test that finished before the testers arrived in the morning was on time. A team that deploys ten times a day has a different clock. The next person starts minutes after a deploy, so the answer has to come in minutes, and the suite has to stop growing. The purpose is the same; the clock decides the size.

Health checks, smoke tests and monitoring answer different questions

Most platforms already run checks against a service, and teams sometimes count those as their smoke suite. They are not the same thing, and the difference is what each is allowed to conclude. Kubernetes defines three probes. A startup probe waits for a slow container to finish starting. A liveness probeliveness proberestart-if-stuck checkA periodic check the platform runs against a container to decide whether to restart it, meant to catch a process that is stuck, such as a deadlock. It should test only the process itself: point it at a dependency, and a slow database restarts every container at once.Full article →All terms decides when to restart a container — a deadlock is the documentation’s example. A readiness probereadiness probecan-it-take-traffic checkA periodic check the platform runs to decide whether a container should receive traffic; while it fails, requests go to other instances and nothing is restarted. It is the right place to check dependencies, but it still asks about one instance, not about whether the product works.Full article →All terms decides whether the container receives traffic at all. All three ask about one process, and none asks whether a customer can buy a jumper.

The same checks as the smoke suite, run on a schedule against production, are synthetic monitoring. They find what no deploy caused — a certificate that expired at three on a Sunday morning, a payment provider that changed something on its side. In the diagram, that is the only column that catches the certificate. The two are worth keeping as one set of scripts with two schedules: what the gate checks after a deploy is exactly what you want to know every five minutes afterwards.

Where it runs — one suite in five places, with five consequences

Most teams run their smoke suite in one place, usually after a deploy to a test environment. The same scripts are worth more than that, because they answer a slightly different question in each environment, and running them again costs minutes. What changes from place to place is which rows can run and what a red result is allowed to do.

Both are worth having, and they are not substitutes, because they fail for different reasons. If you can afford only one, take the run after the deploy. A broken branch costs one developer an hour; a broken environment costs a team a day, and in production it costs customers.

Smoke, build verification, sanity, confirmation — four names people swap

These words get swapped in interviews and in team chat, and the confusion is worth clearing up once. Three of them have a definition in the ISTQB glossary. Smoke testing is the purpose quoted above. A build verification test is “an automated test that validates the integrity of each new build and verifies its key/core functionality, stability, and testability” — the same gate, named for the build and required to be automated. Confirmation testing, also called retesting, is performed after a defect is fixed, to confirm that the failure it caused does not happen again. The fourth word, sanity testing, has no entry in the current glossary at all — which is why arguments about it never end.

• Smoke

  Triggered by a build or a deploy. A fixed set, the same every time. Its verdict decides whether anyone else starts work.

• Build verification test

  The same gate under the name ISTQB gives it in the context of builds: automated, run on every new build, checking that it is testable at all.

• Sanity

  No standard definition. In most teams: a short, unscripted look at the area a specific change touched, to decide whether deeper testing is worth booking.

• Confirmation

  Triggered by a fix. The exact steps from the bug report, re-run to show the defect is gone. Then it becomes a regression case.

Four deployments, and the check that was missing or ignored

None of these companies was short of tests. Each had a check that could have stopped the deployment, and each of the four lost it in a different place: the check did not exist where it was needed, it existed and was skipped, it worked and nobody listened, or it ran after everyone already had the change.

Where it degrades

The honest limit is that a smoke suite checks one build, on the happy path, at one moment. It cannot see a code path that nothing has exercised yet. In June 2025 a new check in Google Cloud’s Service Control had been in production for two weeks, with every check of the build passing, when a policy change with blank fields finally reached it and took down services around the world; the Release Planning page tells that case. Configuration and data change after the deploy, and they need gates of their own. A green smoke run says the build is worth testing, never that it is correct.

Flakiness is where smoke suites usually rot. A flaky check in a regression suite is annoying; in a smoke suite it is corrosive, because the whole mechanism depends on a red result being believed without discussion. One check that fails at random twice a week teaches everyone to re-run the suite instead of reading it, and that habit costs you the real failure. The policy that works is blunt: a check that fails without a real cause is quarantined the same day — taken out of the gate, still run and reported separately — and then fixed within a fixed window or deleted.

• A suite of HTTP 200 checks, which a page with an error inside it passes.
• Asking the load balancerload balancerspreads requests across serversA server or managed service that receives incoming requests and spreads them across several copies of an application, skipping any copy that fails its health check. It is also one more network hop with its own timeouts, so its settings and the application’s have to agree.All terms for the version, so seven right servers hide the eighth.
• Smoke run by hand when someone remembers; the day it is skipped is the day it was needed.
• Results posted to a channel with no consequence attached, so a red run is information rather than an instruction.
• A gate whose red result has never been seen to stop anything, because nobody has tested the gate.
• The suite left unchanged in production while it writes real data — one forgotten test order per deploy becomes a finance question by the end of the quarter.
• Twenty reasonable additions, and a suite that now takes forty minutes and is still called smoke.
• An owner called “QA”, which is a department, not a person who fixes a red run today.
• Treating a green run as a statement about quality.

When to use it

Related pages

• QARegression testingRegression testing is re-running tests after a change to catch anything that used to work and has stopped, in parts of the system nobody meant to touch. The page opens with sixteen cases from one shop’s suite — why each is there, the level that proves it and when it runs — then covers what earns a case and what removes one, which subset runs when, flaky tests by the numbers, the cost nobody budgets for, and four regressions where the case was missing.
• QAEnd-to-end testingEnd-to-end testing walks a business process through the assembled system from start to finish, the way a customer would, in conditions close to production. The page opens with one shop’s whole suite — twelve journeys, the seam only each one proves, and what is real and what is replaced — then covers what no lower level can prove, the shapes a portfolio takes, what a red journey costs, how to choose journeys and keep them trustworthy, and three failures where a test stopped one step short.
• InfrastructureCI/CDContinuous integration is not a server that runs your tests — it is the habit of merging into one shared branch often enough that conflicts stay small, and the automation exists to make that habit safe. Continuous delivery is the second half: one artifact, built once, promoted through environments without being rebuilt. This page covers how to order the stages, why a flaky test destroys the pipeline's authority, what the deployment strategies actually trade, and the four numbers that tell you whether any of it is working.