Computer vision

Twelve field failures, and the check for each

Here is the whole list first, because a detector rarely fails in the lab and very often fails in one of these twelve ways in its first months in the field. Each row pairs what you will see with what is actually wrong and the check that would have caught it before deployment. The rows are numbered so a review can say "we have no check for row 8", and the sections below refer to them by number.

Inference happens inside a budget, and the budget is not GPU memory

Model serving in a data centre optimises for throughput per dollar. On a device the constraints are different in kind, not in degree, and three of them have nothing to do with computation. The general serving material lives in the MLOps pages; what follows is only what changes.

Row 4 of the opening table deserves a picture, because it is where a modelling metric quietly becomes a positional one. The benchmark most teams quote is inference alone, on a cool bench. The number that matters is shutter to decision, on the real device, after it has been in the sun.

Row 5 belongs here too, because it is a budget decision that looks like a modelling one. Shrinking a model to fit the device — quantisingquantisationfewer bits per numberStoring a model’s weights — or an index’s vectors — with fewer bits per number, typically 8-bit integers instead of 32-bit floats, so they take less memory and run faster. The cost is some accuracy, and whether that cost is acceptable has to be measured on your own data, not assumed.All terms it to 8-bit integers, compiling it for an accelerator, resizing inputs to what the hardware likes — changes its outputs. Sometimes by nothing, sometimes by enough to flip detections near the threshold. A fixed set of golden images run through the whole on-device pipeline and compared with the reference outputs on every build is the cheapest test on this page, and it catches a colour-channel swap before a flight does.

The data problem, which no amount of training fixes

A detector learns the distribution it was shown. Deployed, it meets a different one, and the gap is not random noise — it is systematic, it is large, and it is predictable enough to be planned for. This is the single biggest source of field failures in vision systems, rows 2, 3 and 6, and it is almost never a modelling mistake.

• Sensor shift. A model trained on colour imagery does not transfer to thermal, and one trained on one thermal camera often does not transfer to another. The sensor is part of the training data, not a detail of it.
• Season and weather. Foliage, snow, mud, rain on the lens, low sun straight into the sensor. A model validated in summer has been validated on a quarter of the year.
• Geometry. Altitude, angle and focal length change the number of pixels on an object by an order of magnitude, and a detector trained at one scale quietly fails at another.
• Deliberate concealment. Unlike almost every other domain, someone is actively working to make detection fail — with cover, with decoys, with disruptive patterns. The distribution is not merely different; it is adversarially chosen, which is row 12.
• Class imbalance in the extreme. The thing you care about may appear in one frame in fifty thousand, so a model that always says "nothing here" scores 99.998% and is worthless.

Two failures on the list are about the test itself rather than the world, and they are the ones that make every other number lie. Row 1 is leakage: frames from one flight are nearly identical, so a split made frame by frame puts near-copies on both sides and the model is graded on scenes it has memorised. Row 11 is the shortcut: something in the image that has nothing to do with the object — the background, a watermark, which camera took it — correlates with the label, and the model learns that instead. The fix for both is the same discipline. Split by flight, place and date, and hold out a whole source the model never sees until the end.

The practical response is not a better architecture. It is a data pipeline that closes the loop: every deployment records real frames, a review process labels the interesting ones — especially the misses and the false alarms — and those enter a permanent evaluation setevaluation setthe test suite for a modelA fixed collection of inputs with a way to judge each output, run before and after every change to a prompt, a model or a retrieval step, and reported as rates rather than pass or fail. Its job is to fail: a set built from cases the system already handles reports a high score forever and detects nothing.Full article →All terms organised by condition. That is the same discipline as the regression suite described in the QA section, applied to a model, and it is what turns field experience into something the next version inherits rather than something an engineer remembers.

The threshold is a policy decision with a named owner

A detector does not output "yes" or "no". It outputs a number, and somebody turns that number into a yes or a no by choosing where the line goes. That choice is usually made by whoever was tuning the model, on a Tuesday, using the validation set — and it is the single most consequential decision in the system.

Both failure directions have consequences that leave the model behind. Too many false positives and the operator stops trusting the alerts — row 7 — not through negligence but through arithmetic, because a stream that is mostly noise cannot be attended to. That failure looks like a human problem and is caused by a number. Too many misses and nothing at all happens — row 8: no alert, no record, no feedback, and the system quietly reports excellent precision while the thing it exists to find goes past. Of the two, the second is worse precisely because it produces no signal that anything is wrong.

The arithmetic behind row 7 is worth seeing once, because it is what makes a respectable-sounding error rate unusable. When the object is rare, false alarms are drawn from almost every frame and real detections from almost none, so even a small rate multiplies into a wall of noise.

One more property belongs in that policy and is usually absent — row 9: the system needs a way to express "I do not know". A detector forced to answer with a binary on every frame will answer confidently on the frames it has never seen anything like, which is the exact opposite of what you want. An explicit uncertain band — below the alert threshold, above the ignore threshold, routed to review rather than to an operator — costs little, and it is where the interesting failures show up first.

And a detection is rarely acted on alone: frames are joined into tracks, and row 10 is where that joining goes wrong. If the tracker keys an object on its class, every time the classifier changes its mind the object starts again with no history, and its path can never be predicted. The label has to be an attribute of a track that can be revised, not the thing that defines it — the same rule the C2C2Command and control — command and control softwareSoftware that turns reports from many sources into one shared picture people can act on, and carries the decisions made from it. Its central job is correlation — deciding whether two reports describe one thing or two — which is why it is harder than the map it is usually drawn as.Full article →All terms page states for track numbers.

What has to be recorded for a wrong answer to be investigable

A detection that cannot be reconstructed cannot be reviewed, and a system whose mistakes cannot be reviewed does not improve — it accumulates opinions about itself. Recording is constrained here by storage and by the fact that the device may not come back, so it has to be designed rather than switched on.

• The frame, or a crop of it

  Without the image there is no investigation, only argument. If storage forbids keeping everything, keep the detections and a sample of the silent frames rather than a uniform sample of nothing in particular.

• The model identity

  Which model version, which weights, which threshold, which preprocessing. Six months later this is what distinguishes a model regression from a changed environment.

• The conditions

  Sensor, altitude, angle, time, temperature. These are what let you slice performance by condition afterwards, which is the only useful way to read it.

• What the human did

  Confirmed, rejected, ignored. This is free labelled data of exactly the right distribution, and most systems throw it away.

The last card is the one worth acting on this week. Operator decisions are a labelled stream produced by the exact deployment you care about, at no annotation cost, and they are the shortest path from "the system is wrong sometimes" to a case in an evaluation set. Capturing them requires the interface to make confirm and reject equally easy, which is a design decision made in the C2 layer rather than in the model.

Seven failures, and the row that covers each

Defence programmes rarely publish their failures, so the documented cases here are civilian. The mechanisms transfer exactly: a detector does not know what it has been deployed for, and each of these is a failure of data, threshold, tracking or oversight rather than of the model architecture.

Where it degrades

The honest limit of every check on this page is that it measures the detector against conditions somebody thought to collect. An evaluation set sliced twelve ways still contains only those twelve slices; the thirteenth condition — a new kind of cover, a sensor fault, weather nobody recorded — arrives unannounced, and the numbers stay green until it does. The uncertain band of row 9 and the review of silent frames in row 8 are the only practices here aimed at what nobody anticipated, which is why they are worth more than their cost suggests.

• One accuracy figure on a slide, averaged across conditions nobody lists.
• A test set split frame by frame from the same flights as the training data.
• A benchmark run on a workstation and quoted for a device in a sealed box in the sun.
• A threshold nobody owns, set on a Tuesday by whoever tuned the model.
• "Human review" of an alert stream that is ninety-six per cent noise.
• Precision reported with no look at what was missed.
• A class removed from the product because the model got it wrong, with the data left as it was.
• An evaluation set collected only from cooperative scenes, for a system whose input someone else chooses.
• A match score presented to a person as an identification.

When to use it

Related pages

• DeftechDrone softwareDrone software is the flight code on an uncrewed aircraft and the ground software commanding it, built to stay safe when the link, position or battery fails. An uncrewed aircraft is a distributed system in which one node falls out of the sky if it misses a deadline, and that one fact decides which code runs where, what may block and what may crash. The page opens with the twenty-five-line flight-readiness checklist, then covers the stack as a hierarchy of deadlines, the failsafe ladder as the real specification, the return decision as an energy calculation, autonomy as a list of decisions, simulation as the test environment, and seven documented accidents.
• DeftechC2 systemsA command and control (C2) system is software that turns reports from many sources into one shared picture people can act on, and it is usually pictured as a map. It is better understood as a correlation engine with a map attached, whose central job is deciding whether two reports are one thing or two. The page opens with the twenty-five-line shared-picture checklist, then covers correlation, the two times and the clock every record carries, what happens when two nodes lose contact, interoperability as a requirement, the person in the loop as properties you can test, and seven documented failures.
• InfrastructureModel ServingModel serving is putting a trained model behind an interface that answers requests. Once a model is trained, serving it is a latency and cost problem before it is a machine learning problem. The inference itself is usually the smallest part of the response time, the hardware is throughput hardware being asked to do a latency job, and the thing most teams monitor — accuracy — is the one thing they cannot see live. This page covers the request path, the three shapes of serving and why the cheapest usually wins, batching and the right scaling signal, how to ship a new version without betting on it, and what to watch when the truth arrives weeks late.