Metrics & KPI
A metric is a number a team steers by, and a KPI is the small set of them the team has agreed to be judged on. The craft is not picking impressive metrics — it is building a chain from something the team can influence this week to something the business cares about this year, and knowing which link you are looking at. This page covers that chain, Goodhart's law and guardrails, two framework checklists, and how to read a number without fooling yourself.
- Governing law
- Goodhart's law
- Per team
- 1 primary, 2–3 guardrails
- Frameworks
- HEART · AARRR
Inputs, outputs, outcomes
Three kinds of number get called metrics, and confusing them is the root of most disagreement about measurement. An input is something a team directly controls this week. An output is what that work produced. An outcome is the change in behaviour or business result that followed.
The trap sits at both ends. Measure only inputs and you get a busy team with no evidence it matters. Measure only the outcome and you get a number that moves for reasons nobody can attribute — marketing spend, seasonality, a competitor's outage — and a team that cannot tell whether its work helped. A working set has one outcome to aim at and two or three inputs the team can actually pull this week.
The same distinction is often described as leading and lagging indicators. A lagging indicator tells you what happened — revenue, churn, annual retention — accurately and too late to act on. A leading indicator moves earlier and predicts it imperfectly: how many new accounts completed setup this week, how many teams invited a second member. You steer by the leading ones and you are judged on the lagging ones, and the whole difficulty is that the link between them is a hypothesis you have to keep re-checking.
Goodhart's law and why it is not avoidable
The law states that when a measure becomes a target, it ceases to be a good measure. This is not cynicism about people — it is arithmetic. Any metric is a proxy for something you cannot measure directly, and optimising hard enough will always find the gap between the proxy and the real thing.
| Target | Gets you | Guardrail |
|---|---|---|
| Time in app | Interfaces that waste time on purpose | Task completion rate |
| Signups | Accounts that never return | Week-4 retention |
| Tickets closed | Fast closes, repeat contacts | Reopen rate |
| Deployment count | One change split into five | Change failure rate |
The practical defence is the right-hand column. A guardrail is a metric you commit not to worsen while chasing the primary one, and it should be chosen specifically as the thing that would break if you gamed the target. Every primary metric deserves the question: if someone optimised this ruthlessly and cared about nothing else, what would they damage?
It is worth being precise about who does the gaming, because the law is usually explained as if it were about dishonest people. It is not. Most metric distortion is done by conscientious people making locally reasonable choices — a support agent closing a ticket because the queue timer is red, an engineer splitting a change because the weekly deployment number is watched. The behaviour follows the measurement whether or not anyone intends it to, which is why the answer is a better measurement rather than a talk about values.
Two frameworks worth knowing
HEART — for a feature
Happiness, Engagement, Adoption, Retention, Task success. From Google research, and useful because it forces you past adoption to whether the thing actually worked.
AARRR — for a funnel
Acquisition, Activation, Retention, Referral, Revenue. Maps the user lifecycle and shows which stage is leaking before you optimise the wrong one.
Both are checklists for finding candidates, not sets to adopt whole. A team tracking all five HEART dimensions for every feature is generating a report, not steering. Pick the one or two that would change a decision.
The North Star metric, honestly
A single company-wide metric is genuinely useful for alignment and genuinely dangerous alone. It works when it captures delivered value rather than activity — nights booked rather than searches run, messages read rather than sent. It fails when a team three layers down cannot influence it, at which point it stops being a goal and becomes weather.
The fix is not a smaller North Star but an input tree beneath it. Break the metric into the factors that multiply into it — for nights booked: how many travellers search, what share find something acceptable, what share of those complete a booking, how often a booking is cancelled — and give each team the factor it can move. The company keeps one number; each team gets one it can actually change by Friday.
Reading a number without fooling yourself
Most bad metric decisions are not caused by the wrong metric. They are caused by the right metric read the wrong way, and four habits prevent almost all of it.
Cohorts, not snapshots
A snapshot mixes people who joined yesterday with people who joined three years ago, so a change in the mix looks like a change in behaviour. A cohort groups users by when they arrived and follows that group over time, which separates "the product got better" from "we acquired different people". A retention number without a cohort behind it usually reflects last month's marketing rather than this month's product.
The retention curve is the single most informative chart most products can draw. What matters is not where it starts but whether it flattens: a curve that levels off at 30% means you have a product a third of arrivals genuinely keep using, and growth compounds on that floor. A curve that keeps descending to zero means every new user eventually leaves, and no amount of acquisition fixes it — you are filling a bucket with a hole in the bottom, faster.
The average is the least useful statistic
Averages hide everything interesting. An average page load of 800 ms can mean everyone waits 800 ms, or that 90% wait 300 ms and 10% wait five seconds — and only the second is a problem, and only the second explains the support tickets. Read the median and the 90th or 95th percentile together; for anything a user waits for, the tail is the experience people talk about.
The same applies to user behaviour, where distributions are usually far from symmetric. "Average sessions per user: 4" in a product where a small group of power users does forty sessions and everyone else does one describes nobody at all. Segment first, then average within the segment — never the reverse.
What an A/B result does and does not say
Two rules save most of the damage. First, decide the sample size and the duration before starting, and do not look at the result to decide when to stop — checking repeatedly and stopping when it looks good will produce a "win" from pure noise more often than not. Second, run at least one full week so that weekday and weekend behaviour are both represented.
And a "not significant" result is not a result of no difference. It means this test could not distinguish the effect from noise — which is the expected outcome when the effect is small, the sample is modest, or both. The honest reading is "we still do not know", and the decision then rests on judgement, cost and reversibility, exactly as it did before the test.
How this shows up in real delivery
A metric earns its place by changing a decision. The test is direct: for each number on the dashboard, name the decision that would go differently if it moved by 20%. Most dashboards lose two-thirds of their contents to that question, and the team is better off for it — a wall of numbers nobody acts on trains everyone to ignore the ones that matter.
Where it degrades
- Vanity metrics — totals that only go up, so they can never signal a problem.
- A primary metric with no guardrail, which is an open invitation to game it.
- No written definition, so two teams argue about a trend in two different numbers.
- Snapshots instead of cohorts, so acquisition changes are read as product changes.
- Averages hiding the distribution — a median and a 90th percentile tell different stories.
- Stopping an A/B test the moment it looks good, which manufactures wins out of noise.
- Metrics tied to individual performance, which corrupts the number within one review cycle.
When to use it
Use it when
- The team can influence the number within its own scope of decisions.
- Each metric has a decision attached to it that would actually change.
- You can pair every primary metric with a guardrail.
- The measurement is trustworthy enough that people will believe a bad number.
- Every metric has a written definition everyone reads the same way.
Avoid it when
- The number will feed a performance review — it will be optimised, not improved.
- Nobody can influence it, which makes it a weather report rather than a goal.
- Instrumentation is unreliable, so the debate will be about the data, not the product.
- Adding a fifteenth number to a dashboard nobody currently reads.
Found this useful?
Share it with someone who is working on the same problem.