C2 systems

Correlation is the operation, and it is never certain

Three vocabulary terms make the rest of this page readable, and they are worth separating because systems that blur them produce a picture nobody can reason about. A REPORT is what one source said at one moment: raw, attributable, never modified. An ENTITY is the real thing in the world that a report may or may not be about. A TRACK is the system’s current best belief about an entity, assembled from reports and always revisable. Reports are facts about sources; tracks are hypotheses about the world.

Correlation is the step that assigns a report to a track, and it is a judgement under uncertainty every single time. Two sensors reporting positions eighty metres apart may be seeing one object with a position error or two objects that happen to be close. The system has to pick, and both mistakes are expensive in different directions: merging two entities hides one of them, and splitting one entity produces a picture with a thing in it that does not exist. Neither error is recoverable if the reports were discarded after merging.

Two fields that have to exist from the first day

Retrofitting either of these is a migration across every consumer of the data, so they belong in the first schema rather than in the second version. Both look like details and both decide whether the picture can be trusted.

• Event time and receipt time

  When it happened and when you learned about it are different numbers, and the gap can be minutes. A report that is forty seconds old is not a report about now, and a display that cannot show the difference will present stale information as current.

• Confidence, as a value

  Every report and every track carries how sure it is and on what basis. Rounding uncertainty away at the edge of the system is the most common way a picture becomes confidently wrong.

• Provenance

  Which source, which sensor, which operator, which version of which classifier. This is what makes a disagreement investigable rather than a matter of opinion.

• Clock discipline

  Nodes that cannot reach a time source drift. Record the clock source and its quality with the timestamp, or ordering across nodes becomes a guess dressed as a fact.

Confidence deserves one extra note because it is where good intentions fail. A number between zero and one is only meaningful if everyone producing it means the same thing by it, and they do not: a sensor’s 0.9 is a calibrated probability, a classifier’s 0.9 is a softmax output that means very little, and a person’s 0.9 is a shrug rendered as a number. Either agree what the scale means and calibrate against it, or carry the raw evidence and let the consumer decide. What does not work is averaging three incompatible 0.9s into a 0.9 that describes nothing.

When two nodes lose contact

This is where the constraint from the overview becomes concrete. Two nodes are disconnected for an hour, both keep receiving reports, both update their tracks, and both are correct about what they saw. When the link returns they disagree — not because either is broken, but because they saw different things. The general distributed-systems material on this lives elsewhere in these pages; what is specific here is that the correct resolution is almost never to pick a winner.

One practical consequence: because reconciliation happens over reports rather than over tracks, the sync protocol is exchanging an append-only log of immutable facts, not a diff of mutable state. That is a considerably simpler thing to make correct, it is idempotent by construction, and it survives being interrupted halfway — which matters when the contact window is measured in seconds.

The interface is where the engineering ends up

All of the above exists so that a person can make a decision, which means the display is not a presentation layer over the real system — it is the last stage of it, and the place where careful data modelling is most easily thrown away. A track carrying age, confidence and provenance, rendered as an identical icon to every other track, has had all three deleted at the last step.

• Age has to be visible without being asked for. A track based on a report from four minutes ago should not look the same as one updated two seconds ago, and the operator should not have to click to find out.
• Uncertainty should be rendered as uncertainty — an area rather than a point, a hollow marker rather than a solid one. A precise symbol is a claim, and drawing one over a guess is a lie the interface tells on the system’s behalf.
• Every element should be able to answer "why do you think that" in one interaction, showing the reports behind it and their sources.
• Degraded modes need a design, not a blank panel. When a feed is stale or a node is isolated, the display has to say so plainly — silence reads identically to "nothing is happening".
• Alerting follows the same rule as anywhere else on this site: an interruption that a person cannot act on trains them to ignore the ones they can.

Anti-patterns worth naming

• Collapsing reports into a track and discarding them, which makes every correlation error permanent and unauditable.
• One timestamp per record, so nobody can tell whether a track is current or forty minutes old.
• Last-writer-wins reconciliation, which resolves genuine disagreements by comparing two clocks that were never synchronised.
• Averaging confidence values from sources that mean different things by the number, producing a figure that describes nothing.
• A display that renders every track identically, deleting age, confidence and provenance at the last possible step.

When to use it

Related pages

• DeftechOverviewDefence technology software is ordinary engineering under four constraints the rest of this site assumes away: there is no reliable network, the attacker is funded and patient, a wrong output costs far more than a bad quarter, and the user is not at a desk with time to learn. Almost everything else on this site assumes the opposite of all four. Remove them and the engineering does not become exotic — it becomes strict. This page names the four constraints, shows what each one replaces, and covers the parts of the working environment that surprise people arriving from product engineering.
• DeftechData platformsA data platform is the stack that collects, stores and moves an organisation's records, and one built for a reliable network sends first and stores as a side effect. Invert that assumption and the architecture inverts with it: the node writes locally and always, sending becomes an opportunistic background activity that may not happen today, and the interesting design decision is which records leave first when the contact window is forty seconds wide. This page covers that queue, the receiving side that expects to be interrupted, and the three properties every record has to carry.
• DeftechDrone softwareDrone software is the flight code that runs on an uncrewed aircraft together with the ground software that commands it, and an uncrewed aircraft is a distributed system where one of the nodes falls out of the sky if it misses a deadline. That single fact organises everything: which code runs where, what may block, what is allowed to crash, and why the interesting part of the specification is not the mission but what happens when the link, the position fix or the battery goes. This page covers the stack, the failsafe ladder, autonomy as a set of specific decisions, and why simulation is the main test environment rather than a nice extra.