Release Planning

A release plan on one screen — finished

Most people who look up release planning want to see a plan that works. Here is one, for the online shop used across the Decomposition pages: the release that puts Reorder from order history — feature 1 of the reorder epicepica large backlog itemA large piece of work that is split into features or stories before anyone builds it, best written as a bet with a hypothesis and a condition that closes it. Its size depends on whose vocabulary is in use — anything above a story in Jira, a portfolio investment with a business case in SAFe — so check before comparing two epics.Full article →All terms — in front of customers. The code for it has been reaching production for weeks, switched off. So the plan is not about a deployment date. It is about who sees the feature, in what order, what would stop it, and how it is switched off again. Later sections cite the lines by number.

1. 1.What is released, and what is not

   1. 1.1Reorder from order history — stories 1.1 to 1.4 of the reorder epic, behind the release flag reorder-from-history
   2. 1.2The flag’s owner is the team’s product manager, and it expires on 18 December
   3. 1.3Not in this release: saved baskets, feature 2 — moved to January rather than squeezed in before the freeze
   4. 1.4One changelogchangelogthe record of what changed, per versionA file, usually CHANGELOG.md, that lists the notable changes in each version of a piece of software, newest first, grouped as added, changed, deprecated, removed, fixed and security. It is written for the person deciding whether to upgrade, which is why a dump of commit messages does not count as one.Full article →All terms line per story, written before stage 3.3, for customers and for support

2. 2.Schema and data

   1. 2.1Expand: a nullable reorder_source column on orders, deployed on 20 October, two weeks before anyone sees the feature
   2. 2.2No backfill: past orders are read from the order history service, enablerenablerwork that makes later work possibleA backlog item no user asked for that makes later items possible: research, architecture, infrastructure or compliance work, in SAFe’s four types. Naming it lets that work compete for capacity openly instead of hiding inside the estimates of user stories.Full article →All terms 4 on the epics page
   3. 2.3Nothing is removed or renamed in this release; any contract step waits until after the freeze

3. 3.Stages — each opens only if the one before met every gate in group 5

   1. 3.1Dark launch from Monday 26 October: every order-history view calls Reorder in the background and discards the result; errors under 0.1%, p95p99latency for the slowest sliceThe response time that 99% of requests come in under — p95 and p50 are the same measure at 95% and at the median. The average hides exactly the requests that make people leave, which is why these are the numbers anyone serious watches.All terms under 300 ms
   2. 3.2Staff accounts, about 200, on Monday 2 November
   3. 3.31% of returning customers on Wednesday 4 November, after the go/no-go — until Monday, so a weekend is in the sample
   4. 3.410% on Monday 9 November
   5. 3.550% on Wednesday 11 November, again over a weekend
   6. 3.6100% on Monday 16 November; if it is not reached by Wednesday 18 November, the flag stays where it is until 2 December

4. 4.The way back

   1. 4.1Flag off: the on-call engineer or the product manager, from the flag console, effective in under a minute; rehearsed on stagingstagingthe rehearsal environmentA copy of production that changes are tried on before real users meet them. How much it proves depends entirely on how closely it matches production — a staging environment with a tenth of the data, no real traffic and a different config demonstrates that the code starts, and not much more.All terms on 30 October
   2. 4.2A defect in the code: fix forward through the normal pipeline, 25 minutes; the previous build stays deployable because 2.1 only added a column
   3. 4.3What cannot be undone: orders placed and paid through Reorder stay placed; support has a refund macro for an order a customer did not mean to place

5. 5.Signals, and who watches them

   1. 5.1Payment success at or above 97.5% in the flagged group — the committed key result on the OKROKRObjectives and Key Results — a goal with measured resultsA goal-setting method that pairs one qualitative objective with two to five measurable key results proving it was reached, set and reviewed every quarter. It stops working when the key results become a task list or the scores feed into pay.Full article →All terms page; an alert, not a dashboard
   2. 5.2Reorder taps that end in an error: under 0.5%
   3. 5.3Checkout conversion of the flagged group against the unflagged group, read every morning by the product manager
   4. 5.4Support contacts tagged “reorder”, counted every morning by the support lead

6. 6.People and dates

   1. 6.1Release owner: the product manager, who opens each stage and may close any of them
   2. 6.2Stages open Monday to Thursday, before 14:00, with an engineer on call until 20:00
   3. 6.3Support is briefed on 30 October: a one-page FAQ, the refund macro, the changelog lines
   4. 6.4Go/no-go on Tuesday 3 November, fifteen minutes, before stage 3.3 — its five questions are in the section on the way back
   5. 6.5Peak-season change freeze from 19 November to 1 December (Black Friday is 27 November): no stage moves forward, but switching the flag off is always allowed

7. 7.Done when

   1. 7.1The flag has been at 100% for fourteen days without being switched off
   2. 7.2The flag and the old code path are deleted by 18 December, after the freeze
   3. 7.3The leading indicators go to the epic’s review, which decides on feature 3

Deploying is not releasing

Deployment is a technical event: new code is running on production infrastructure. Release is a product event: users can now do something they could not do before. Most organisations treat these as one moment, which is why “release day” carries so much fear — every technical risk and every product risk arrives together, at midnight, on a Friday somebody chose months ago.

Separating them changes what a release plan is. Code ships to production continuously, dark and inert behind a flag. The release becomes a separate, reversible decision: turn it on for 1% of users, watch the numbers, turn it on for everyone — or turn it off in seconds without a rollback, a hotfix or a war room.

The split has a short history. Jez Humble and David Farley made it the centre of “Continuous Delivery” (2010): every change that passes the pipeline is releasable, and releasing it is a business decision. Facebook had already shown the extreme case in May 2008. Chat was built to be switched on for all of its roughly 70 million users at once, so for weeks beforehand live pages quietly generated chat traffic with no interface visible. The servers met the real load before any user did. That technique is a dark launch, and line 3.1 of the plan is one.

A fixed date or a fixed scope — the cadence a release sits in

Before any rollout, a release plan answers a question that is often skipped: which is fixed, the date or the contents? Holding both fixed is the classic way to miss both. The only variable left is quality, and nobody writes that down as a decision. There are three workable answers.

A release train works because it removes the most expensive conversation in software: “can we wait two more days for my feature?” The answer is always no, and the next train is never far away, so nobody needs to argue. A feature that is not ready when the branch is cut stays switched off, and the train leaves without it. The cost is that a large feature has to be built so that it can ride several trains switched off — which is the deploy-versus-release split again. SAFeSAFeScaled Agile Framework — a framework for running many agile teams as oneThe Scaled Agile Framework: roles, events and practices for coordinating dozens of agile teams, built around the Agile Release Train — five to twelve teams that plan together every eight to twelve weeks. It is the most prescriptive and the most widely adopted scaling framework, which is also why it can be installed without changing who makes the decisions.Full article →All terms scales the same idea to a whole department: an Agile Release TrainAgile Release Train5–12 teams that plan and deliver togetherIn SAFe, a long-lived group of five to twelve agile teams, usually 50 to 125 people, that plan together at PI Planning, work to the same iteration boundaries and integrate their work every iteration. It should be drawn around the path a product takes to its customers; a train drawn around the existing departments coordinates them without changing anything.Full article →All terms plans together for each Planning Interval, usually eight to twelve weeks — and its own phrase, “develop on cadence, release on demand”, keeps the plan and the release apart.

Forecasting a release with a fixed scope

When the contents are fixed, the date has to be forecast, and an honest forecast is a range. The arithmetic Mike Cohn set out in “Agile Estimating and Planning” (2005) still works. Divide the remaining scope by the slowest and by the fastest velocity the team has actually had. 120 points left, at 18 to 26 points a sprint, is five to seven sprints — not 5.6. Say the range, and say what would narrow it.

Release planning used to be part of Scrum itself. The 2010 Scrum Guide described a release planning meeting and a release burndown chart. The July 2011 edition removed both, and no edition since has put them back: every sprint is meant to produce a releasable increment, so how releases are grouped is left to the team. Extreme ProgrammingExtreme Programmingthe engineering-first agile methodAn agile methodology from the late 1990s built around engineering practices: test-first development, pair programming, continuous integration, refactoring and small, frequent releases. Much of what teams now call technical agility comes from it, and a team that adopts Scrum’s events without these practices usually watches its sprints slow down.All terms kept it. Its planning game, in Kent Beck’s “Extreme Programming Explained” (1999), has a release plan of stories above the plan for each iteration.

Rollout strategies — from everyone at once to one percent

A rollout strategy decides who meets the new version first. Strategies differ in what moves — servers, traffic or users — and in how the way back works. They combine rather than compete: the shop deploys with a rolling update and exposes the feature with a flag.

Blue-green was named around 2005 by Daniel Terhorst-North and Jez Humble, and Martin Fowler wrote it up in March 2010. A canary release borrows the miners’ canary: a small group meets the danger first. Rings apply the same idea to people rather than traffic, and Microsoft ships Azure DevOps that way — its own engineers’ ring first, the rest of the world last. Two related techniques solve problems the others do not. A dark launch runs the new path on real traffic without showing the result, as line 3.1 does. A rolling update is what most orchestrators do by default: cheaper than blue-green, but both versions serve traffic at the same time.

Apps in stores cannot be canaried in the usual sense, because a version on a phone cannot be pulled back. Apple’s phased releasestaged rolloutan update released to a growing share of usersReleasing an update to a small share of users first — 1%, then 5%, 20%, everyone — and checking crash and error rates before each step, so a bad release reaches a few people instead of all of them. It limits how many a bad release reaches, not whether it reaches them: on the app stores a halted rollout does not take the update back from those who already have it.All terms spreads an update to users with automatic updates over seven days — 1, 2, 5, 10, 20, 50 and 100 percent — and can be paused for up to 30 days in total. Anyone can still download it by hand. Google Play’s staged rollout can be halted, but users who already have the version keep it. In a mobile release plan, the flag inside the app is the thing that can still be switched off.

A canary needs a comparison, not a dashboard

A canary only protects you if something compares it with what it replaced, automatically, before the next step. In April 2018 Netflix and Google released Kayenta, the canary judge Netflix had used internally, as open source. Its design carries a detail most teams miss: the canary is not compared with production. A fresh baseline running the old version is started at the same moment and at the same size. Long-running servers differ from new ones in warm caches, memory and connections, and that difference would swamp the one you are looking for.

The judge scores the canary from 0 to 100 against the baseline across the chosen metrics. A success promotes it, a marginal score asks a person, and a failure rolls it back. The shop’s plan applies the same logic with a flag. The flagged group and the unflagged group are the canary and the baseline, which is why line 5.3 compares them instead of reading one number on its own.

Four kinds of flag, four lifetimes

Teams get into trouble with feature flags because they treat them as one thing. The reason a flag exists decides how long it should live and who should own it, and mixing the kinds is how a codebase ends up with two hundred toggles nobody dares delete. The taxonomy most teams now use is Pete Hodgson’s, from October 2017. It sorts flags by two questions: how long they live, and how often the decision changes while they live.

The first two are temporary and the second two are not. Confusing them in either direction hurts. A release toggle left in place becomes an untested code path that quietly rots. A permission toggle logged as “flag debt” gets deleted by someone doing cleanup, taking the paid tier with it. Line 1.1 of the plan is a release toggle, so lines 1.2 and 7.2 give it an owner and a date to die.

Hodgson lists the ways teams keep temporary flags temporary. A removal task goes into the backlog on the day the flag is created. The flag gets an expiry date. A “time bomb” test fails the build once a flag outlives that date. Or the system has a cap on how many flags it may carry at once. Line 7.2 is the second of these. The third is the one that still works when everyone is busy.

The way back — and when there is none

Release plans routinely list “roll back” as the mitigation for every risk, and for stateless code that is honest. The moment a release touches persistent data, it usually stops being true — and the plan is now describing a safety net that does not exist. The four ways back undo different things.

The reason rollback fails is simple. If version 2 dropped a column, wrote data in a new shape or consumed a message from a queue, going back to version 1 does not restore the world. Version 1 now meets data it was never written to handle. This is why experienced teams plan to roll forward — fix and deploy again, in minutes — and keep rollback for the releases that can genuinely take it.

The technique that keeps the way back open is expand and contract, which Danilo Sato wrote up as parallel change in May 2014. Instead of one migration that changes the shape of the data, there are three steps, each its own release. Expand: add the new structure while the old one keeps working, and write to both. Migrate: backfill, and move the readers across one at a time. Contract: remove the old structure once nothing reads it. Every step but the last can be undone, and the last happens weeks later, when the new path has proven itself.

The shop’s plan uses only the first step. Line 2.1 adds a column that no old code reads, so the previous build can still be deployed (4.2). Line 2.3 pushes anything destructive past the freeze. A rename is the classic trap here. It looks like one harmless migration, and it is the expand step and the contract step done at once, with no release in between.

What a go/no-go should actually check

Where a release decision genuinely needs a moment of judgement, keep the questions about what cannot be automated. Whether the tests passed is not one of them — the pipeline already knows, and asking a room of people to confirm it adds delay without information. These are the five for line 6.4.

• Is the way back defined for each kind of failure — flag off, fix forward, a rehearsed data path — and has anyone actually done it? Line 4.1 was rehearsed on 30 October.
• What signal would tell us within ten minutes that this made things worse — and is it an alert, or a dashboard somebody must remember to open? Line 5.1.
• Who outside engineering needs to know before customers notice — support, sales, the people answering the phone? Line 6.3.
• Does anything irreversible happen in this release, and can it be moved to a later one? Lines 2.3 and 4.3.
• If it goes wrong at the worst plausible moment, who is available and how are they reached? Line 6.2.

Four releases that went wrong, and the line each one lacked

Each case below is documented by a regulator or by the company itself, and each maps to lines of the plan at the top. None was caused by a lack of testing alone. Each was a release that reached everyone at once, or had no way back, or both.

Knight Capital, 2012 — a flag with a past

Knight Capital was one of the largest traders in US equities. In July 2012 it prepared its order router, SMARS, for a new New York Stock Exchange programme for retail orders. The new code reused a flag that had once switched on “Power Peg”, a function unused since 2003 but never deleted — and broken since 2005, when code it relied on was moved and Power Peg was not retested. From 27 July a technician copied the new code to the eight servers by hand. One was missed, and no second person checked.

On 1 August the market opened, orders arrived with the flag set, and the eighth server ran Power Peg. Ninety-seven automated emails reading “Power Peg disabled” had gone out before the open; they were not designed as alerts, and nobody acted on them. In about 45 minutes the system made 4 million executions in 154 stocks, more than 397 million shares, and Knight lost more than $460 million. One attempt to fix it made things worse. The team uninstalled the new code from the seven correct servers, which put Power Peg behind the flag on all eight. The SEC’s order of 16 October 2013 fined the firm $12 million.

TSB, 2018 — a weekend with no way back

TSB, a British bank with about 5.2 million customers, moved off the platform it rented from Lloyds Banking Group onto Proteo4UK, built by the IT arm of its new parent, Sabadell. Sabadell’s method was a single migration weekend, and TSB used it for most of the data: it was cheaper, and migrating by product would have broken customers’ single view of their accounts. The migration ran over 20–22 April 2018, and the data moved correctly. The platform then failed in online and mobile banking, telephone banking and branches, and TSB did not return to normal service until 10 December.

The regulators’ final notice of 20 December 2022 names the two things a release plan is for. There was no way back: once the migration ran, it was “essentially impossible to roll back”. And the evidence behind the go decision included a supplier letter whose confirmations were “forward looking statements of good intention or expectation rather than statements of fact”, most of them caveated with tests not yet done. 225,492 complaints followed within a year, £32.7 million in redress, and fines of £48.65 million from the FCA and the PRA.

CrowdStrike, 2024 — configuration is a release too

Early on 19 July 2024 CrowdStrike pushed a content update — Channel File 291 — to its Falcon sensor on Windows. It was configuration rather than code, and configuration went to every online sensor at once. The new template defined 21 input fields while the sensor supplied 20, and reading the 21st crashed Windows. The file was replaced 78 minutes later. Microsoft estimated that 8.5 million Windows devices were affected, and many had to be repaired by hand, one machine at a time.

The root-cause analysis of 6 August 2024 answers with a release plan. Each new template instance now goes through staged deployment: canary first, then successive rings, each followed by bake-in time while telemetry is read. Customers can also choose where and when content updates land. That is group 3 of the plan, applied to data. A file that changes behaviour is a release, whatever the deployment tool calls it.

Google Cloud, 2025 — the flag that was not there

On 29 May 2025 Google added a quota policy check to Service Control, the service that checks authorisation, policy and quota for requests to Google Cloud’s services. The new code path had no error handling for blank fields and no feature flag, and for two weeks nothing exercised it. On 12 June a policy change with unintended blank fields was written to the global store and, in Google’s words, “replicated globally within seconds”. Every region hit the null pointer at once, and requests failed from 10:49 to 13:49 Pacific time.

Google’s report says it plainly: “If this had been flag protected, the issue would have been caught in staging.” Its commitments are a release plan in two lines. Every change to a critical binary is to be flag-protected and disabled by default. And every system that consumes globally replicated data is to be audited. The second commitment is the CrowdStrike lesson again: a staged rollout of the code does nothing if the data that triggers it arrives everywhere at once.

Measuring the process — and where it degrades

DORA, the research programme now run by Google Cloud, has published a report on software delivery every year since 2014. Its four key metrics became five in 2024. Change lead time, deployment frequency and failed deployment recovery time describe throughput. Change fail rate and the new rework rate describe instability. Rework rate is the share of deployments that were not planned and were made to fix a bug users could see.

Two caveats come from the reports themselves. The 2024 clusters are not a ladder: the medium cluster reported a lower change fail rate (10%) than the high one (20%), so some teams trade speed for stability, and the levels are snapshots of one year’s answers rather than targets. And in 2025 DORA dropped the four levels altogether, replacing them with seven team archetypes that also weigh burnout, friction and product performance. The finding that has held throughout is the one release planning rests on. Speed and stability go together, because small changes are easier to understand and to undo.

One finding is worth quoting to anyone designing a release board. In “Accelerate” (2018) Nicole Forsgren, Jez Humble and Gene Kim report that external approvals — by a manager or a change advisory board — were negatively correlated with lead time, deployment frequency and restore time, and had no correlation with change fail rate. Approval from outside the team slowed delivery and did not make it safer. The go/no-go of line 6.4 is therefore inside the team, fifteen minutes long, and about what the pipeline cannot know.

A caveat worth carrying: these metrics are diagnostic, not a scoreboard. Deployment frequency is easily gamed by splitting one change into five, and a team measured on change fail rate will simply stop calling things failures. Read them together, and read them against your own trend rather than someone else’s benchmark.

Where it degrades

• A release board that approves deployments — the research says it adds delay and no safety.
• Batching a month of changes into one release, so that every failure is hard to attribute.
• “Roll back” written as the mitigation for a release that runs an irreversible migration.
• Feature flags with no owner or expiry, which quietly become permanent branches in the code.
• A canary with no automated comparison — a slower big bang with a nicer name.
• A staged rollout of code while configuration and data still reach everyone at once.
• A freeze that also forbids switching things off.
• Deployment frequency set as a target, which produces more deployments rather than more value.

When to use it

Related pages

• ManagementGit Branching StrategyA Git branching strategy is the agreement on how a team splits its work into branches and rejoins them. The choice looks like a matter of taste and is not: the variable that predicts pain is how long a branch lives before it merges, and every model is a position on that number plus a rule for where releases and fixes come from. The page opens with six models on one screen and the five questions that choose between them, then covers branch lifetime, where each model came from and what its authors said later, release branches and the way a fix travels, what makes short branches possible, merge versus rebase versus squash, and where it degrades.
• ManagementChangelogA changelog is a curated, human-readable record of the notable changes in each version of software, written for the person deciding whether to upgrade. That reader is why a dump of commit messages, however complete, is not one. The page opens with a finished changelog, then covers the documents people confuse with it, what a version number promises, how to write an entry, deprecation, how far automation honestly goes, four releases where the changelog line was the problem, and where the practice degrades.
• InfrastructureCI/CDContinuous integration is not a server that runs your tests — it is the habit of merging into one shared branch often enough that conflicts stay small, and the automation exists to make that habit safe. Continuous delivery is the second half: one artifact, built once, promoted through environments without being rebuilt. This page covers how to order the stages, why a flaky test destroys the pipeline's authority, what the deployment strategies actually trade, and the four numbers that tell you whether any of it is working.