Drone software

The flight-readiness checklist

Here is the whole checklist first, because it is what a flight-readiness review asks and because every line on it has already cost somebody an aircraft — seven of those accidents are at the bottom of this page. It is numbered so a line can be cited: "we have never run 2.4" is a work item, where "the failsafes need a look" is a mood. It is written for the team that builds or integrates the software, on PX4PX4open-source autopilotAn open-source autopilot for drones, planes and other uncrewed vehicles, hosted by the Dronecode Foundation under the Linux Foundation and licensed BSD. It runs on the flight board and handles stabilisation, state estimation, flight modes and failsafes; the permissive licence is one reason companies build products on it.All terms, ArduPilotArduPilotopen-source autopilotAn open-source autopilot for multirotors, planes, helicopters, boats and ground vehicles, licensed under the GPLv3. It covers the same ground as PX4 — stabilisation, navigation, modes, failsafes — with a wider range of vehicle types and a copyleft licence that matters the day you ship modified firmware.All terms or an autopilot of its own. It is not the operator’s pre-flight card, though half of group 5 ends up printed on one.

Do groups 2 and 6 first. The failsafe configuration is what the aircraft actually does when something goes wrong, and until a fault has been injected in simulation, nobody has seen that configuration run.

1. 1.The stack and its boundaries

   1. 1.1The flight controller runs control and state estimation and nothing else. A feature does not go there because it is "closer to the hardware".
   2. 1.2The companion computer reaches the autopilot through one message interface, as an untrusted client: every setpoint it sends is range-checked and may be refused.
   3. 1.3The autopilot watches a heartbeat from the companion computer and has a configured action for when it stops.
   4. 1.4No control decision on board waits for the ground link to complete.

2. 2.The failsafe ladder

   1. 2.1Every condition has a configured action: control link lost, position lost, energy low, boundary reached, companion computer silent, estimator unhealthy.
   2. 2.2Each action is enforced at the lowest tier that can enforce it — the boundary and the energy reserve in the autopilot, never only in the ground station.
   3. 2.3Combinations are tested, not only single conditions: link lost with low energy and a headwind is its own case.
   4. 2.4The last rung — land in place, parachute, flight termination — has actually been exercised, and what it does over people, water and roads is written down.

3. 3.Energy and the way home

   1. 3.1Return is triggered by the energy needed to get home plus a landing reserve, computed from distance and wind — not by a fixed battery percentage.
   2. 3.2Remaining capacity is estimated from the charge consumed as well as the voltage, and the estimate is known to read high in the cold.
   3. 3.3The lost-link route avoids people and ends somewhere a forced landing is acceptable — not in a loop flown until the fuel runs out.
   4. 3.4A link-loss grace timer lets routine drops pass without ending the mission, and its value is a reviewed decision rather than a default.

4. 4.Position and state

   1. 4.1Estimator health is monitored, and an estimate that stops being trustworthy is a failsafe condition of its own, not a warning on a screen.
   2. 4.2Losing satellite positioning leads to a defined behaviour: hold on inertial and visual estimation for a stated number of seconds, then descend or return.
   3. 4.3Every sensor that can end a flight is cross-checked against another: a satellite fix or an altitude reading that disagrees with the rest is distrusted, not obeyed.
   4. 4.4Every measurement carries a timestamp that is checked, because a stale reading presented as fresh does more damage than a missing one.

5. 5.Ground station and configuration

   1. 5.1Parameters live in version control per airframe, and a change to them is reviewed like a change to code.
   2. 5.2Before flight, the parameters on the aircraft are compared with the reviewed set, and any difference blocks launch.
   3. 5.3A setting loosened for a test or a demonstration has an owner and an expiry date.
   4. 5.4Firmware, parameters and mission files are confirmed on each aircraft: a fleet is N confirmations, not one upload.
   5. 5.5Control cannot pass between consoles or operators while the two disagree on a control position, and a safety-critical fault has an alarm of its own.

6. 6.Tests and evidence

   1. 6.1Software-in-the-loopsoftware in the loopreal firmware, simulated worldTesting in which the real control software runs on an ordinary computer against a simulated vehicle, sensors and world. It is fast enough to run on every commit, which makes it a regression suite; what it cannot find is anything that depends on the real processor’s timing.All terms simulation runs on every merge, and each simulated mission asserts on the recorded log.
   2. 6.2Hardware-in-the-loophardware in the loopreal board, simulated worldTesting in which the real controller board runs its real firmware, wired to a simulator that feeds it sensor data in place of the physical world. It finds timing faults, processor load and driver bugs that pure simulation cannot, at the cost of keeping the hardware on a bench.All terms runs on a schedule, because timing faults and driver bugs are invisible to pure simulation.
   3. 6.3Faults are scripted cases in the suite: cut the link, freeze a sensor, shift a timestamp, drop the battery reading, stop a motor.
   4. 6.4Every flight log is kept and can be replayed, and a fault seen once in the air becomes a test case that runs forever.

The stack is a hierarchy of deadlines

It is tempting to read onboard software as layers of abstraction, the way a web application is read. It is more useful to read it as layers of how long each part may take. The innermost control loop runs hundreds of times a second and has a hard deadline: miss it and the aircraft becomes unstable within a fraction of a second. Everything above it has progressively more slack, and the amount of slack decides what may live there.

One detail the diagram simplifies, and it matters when you read PX4 or ArduPilot code. The first two tiers are usually the same board running the same firmware. The rate controller, the state estimator, the navigation code and the mode logic are tasks on one microcontroller, separated by priority and loop rate rather than by a cable. So the boundary between them is a scheduling decision, and adding work to a fast task is exactly how a feature steals time from stabilisation. The companion computer is the first tier that is genuinely a separate machine, which is why it can be allowed to fail.

MAVLink deserves one sentence of its own, because almost every open-source ground station and companion library speaks it. It is a compact message protocol between the vehicle and everything outside it: a heartbeat once a second so each side knows the other is alive, telemetry streams, commands with acknowledgements, and a mission protocol in which the vehicle requests the waypoints one at a time and acknowledges the upload as a whole. The acknowledgements are the part to respect. A command that was sent is not a command that was accepted, and ground software that treats the two as the same is how an aircraft ends up flying a mission nobody finished uploading.

The failsafe ladder is the specification

A mission plan describes what should happen. The failsafe configuration describes what will happen, and it is the part that decides whether a vehicle can be flown at all. A failsafe is a behaviour configured in advance for one specific failure: the condition that triggers it, and the action the vehicle then takes on its own, with nobody asked. Each entry answers one question — this has happened, what now — and every answer is a trade with no free option.

Two properties of that table are worth extracting, and they are lines 2.2 and 2.3. First, every behaviour has to be enforced at the lowest tier that can enforce it. A boundary implemented in the ground station is a suggestion, because the ground station is exactly what disappears. Second, failsafes interact, and the interactions are where the surprises live. Link lost plus low energy plus a headwind is a different situation from any of the three alone, and a configuration that handles each one individually can still produce an aircraft that flies confidently away from home.

This is also why availability is the wrong target for anything that flies. A percentage of uptime describes a service that is either up or down. An aircraft spends its interesting minutes in the partial states — flying without the link, without satellite position, without the companion computer — and the useful specification names what still works at each level of loss. The failsafe table is that specification, written down.

Return is an energy calculation, not a percentage

The most common battery setting in the field is some version of "return at 30%". It reads as caution, and it is wrong at both ends of the flight. The energy an aircraft needs to get home grows with the distance to home and with the wind it will fly into on the way. A fixed percentage knows neither.

Take an illustrative multirotor that uses about 4% of its battery per kilometre in still air and wants 10% in reserve to land. One kilometre out, it needs 14% to get home, so a 30% trigger sends it back with 16% of the battery never used. Five kilometres out, it needs 30% — the threshold is exactly right at one distance and nowhere else. Now add a headwind that halves its ground speed. Every kilometre takes twice as long and costs roughly twice the energy, so five kilometres out the same aircraft needs about 50%, and the 30% trigger fires when it can no longer make it.

Both major open-source autopilots give you more than one number to work with. ArduPilot has a two-stage battery failsafe, low and critical, each triggered by voltage or by capacity consumed, and each with its own action. PX4 has low, critical and emergency levels and, separately, a failsafe for exactly this problem: it estimates whether the remaining battery is only just enough for a return, and engages Return mode when it is. Whichever you use, the reviewed decision is line 3.1 — what the trigger is computed from — not which percentage to type in.

The same reasoning applies to the route home, which is line 3.3. A lost-link route is a flight plan that runs with nobody watching, so it gets the review a mission gets and more: where it crosses people, what altitude it holds over the terrain it actually crosses, and where it ends if the link never returns. "It flies a loop until contact comes back" is not an ending. It is a decision about where the aircraft comes down when the energy runs out, left unmade.

Autonomy is a set of decisions, not a level

Autonomy is usually discussed as a scale from remote control to full independence, which is a useful summary and a poor engineering tool. The question a design actually has to answer is narrower, and it repeats for every capability: which decisions does this vehicle make alone, on what evidence, within what limits, and which does it hold and ask about. Answering that list is the design. The level is a label you can apply afterwards.

Two questions sort every decision: is its deadline shorter than a round trip to a person, and can a wrong call be undone. Three of the four answers are familiar. Short and reversible — stabilising, avoiding an obstacle — is decided on board, because nobody could answer in time and a mistake costs little. Long and reversible — adjusting a route, pointing a camera — can also be decided on board, as long as it is reported afterwards. Long and irreversible is where the vehicle holds and asks.

The fourth quadrant is the one designs miss: no time to ask and no way to undo. A motor fails low over a road; the position estimate collapses a few metres above the ground; the flight termination condition fires. Nobody can be consulted, and whatever happens next is final. The answer there is neither autonomy nor a human in the loop. It is a person deciding in advance, on the ground, which way the vehicle errs, and writing that decision into the configuration where it can be reviewed. The Zurich parachute is exactly this quadrant: the decision to terminate had been made long before, and it was the right one. What failed was the mechanism that carried it out.

The right-hand column has a consequence people miss: holding a decision requires the vehicle to be able to wait safely. A system that must ask a person but cannot loiter, cannot store the request and cannot behave sensibly if no answer comes has not implemented the requirement — it has written it down. Waiting is a capability with its own energy budget, its own timeout and its own failsafe. The operator’s side of a held decision — what they are shown, how long they have, how a refusal is recorded — belongs to the C2C2Command and control — command and control softwareSoftware that turns reports from many sources into one shared picture people can act on, and carries the decisions made from it. Its central job is correlation — deciding whether two reports describe one thing or two — which is why it is harder than the map it is usually drawn as.Full article →All terms page. The vehicle’s side is this one.

Simulation is the test environment, not a convenience

Every failure case that matters here is one you cannot reasonably produce on a real aircraft: losing the position fix at two hundred metres, a motor stopping in a turn, the link going at the far end of the route, a battery reporting a wrong value. Flying those tests is expensive, dangerous and slow, and the ones you survive you cannot repeat identically. There is no production traffic to learn from either. The feedback a web team gets from live telemetry arrives here as a log file after landing — if the aircraft lands. So the primary test environment is a simulator, and the maturity of a team’s simulation setup is a good proxy for the maturity of the team.

Two practices turn that ladder into a regression suite rather than three places where people sometimes test. Faults are scripted cases, not experiments — line 6.3 — so cutting the link at the far waypoint is a test that runs on every merge and fails on a specific line. And every flight log is a test input — line 6.4. A fault seen once in the air is replayed in simulation and stays in the suite forever, which is the same rule as any regression suite and the one that makes the suite better every time something goes wrong. The Ingenuity timestamp fault below is the kind a scripted case catches cheaply: offset one input’s timestamps and assert that the estimator notices.

The release cadence follows from all of this, and it is the part of web practice that transfers worst. Deploying several times a day and fixing forward assumes the next fix can reach the running system in minutes. An aircraft in the field may be out of reach for weeks, so the version it carries has to be safe on its own for weeks. Releases here are fewer, staged across a fleet, confirmed per aircraft — line 5.4 — and reversible on the vehicle itself.

Seven accidents, and the line that covers each

Every one of these has a public investigation or a documented account. In most of them the aircraft did exactly what its software and its configuration told it to, and the investigation put the cause in a design decision rather than in bad luck. None of them is about the mission logic — the part every team demonstrates.

Where it degrades

The honest limit of everything on this page is that a failsafe answers only the failures somebody imagined. Every one of the seven cases was, in hindsight, a condition nobody had written a row for: a lever with two meanings, rain on a lidar, sand with nothing to see. Simulation reproduces the faults you script, and the dangerous one is the fault nobody thought to script. That is not an argument against the ladder. It is the argument for keeping every flight log, replaying it, and treating each surprise as a new row rather than as bad luck.

• A failsafe table that exists as a document and has never been executed, in simulation or anywhere else.
• A boundary enforced in the ground station, the component guaranteed to be absent when it matters.
• "Return at 30%" copied from a forum into every airframe, whatever its range and whatever the wind.
• Parameters edited on the field laptop and never brought back into version control.
• A simulation suite that flies the happy mission hundreds of times and injects nothing.
• A flight log kept only for incident review, never replayed as a test.
• A "human in the loop" requirement for a decision the vehicle cannot wait for.
• A demonstration flown in good weather, on a full battery, with a strong link and the author standing next to it, presented as evidence of anything.
• A release process copied from web engineering — deploy often, fix forward — for a unit that may be out of reach for weeks.

When to use it

Related pages

• DeftechCybersecurityCybersecurity for a defence system is the set of design decisions that keep it trustworthy against a funded, patient adversary who may hold the device itself. Ordinary application security assumes an opportunist on the far side of a network boundary and a patch that ships on Tuesday; here the boundary eventually fails completely, and the patch may take months to arrive. The page opens with the twenty-five-line survivability checklist, then covers the threat model and what it accepts, the captured device, the boot chain and the update path, the supply chain, twenty years in service, eight documented compromises, and where the practice turns into theatre.
• DeftechComputer visionComputer vision is the practice of turning camera images into detections a machine or a person can act on, and the model is rarely the hard part of it. What makes it hard here is that it runs on a device with a power budget and no network, on images unlike the ones it was trained on, feeding a decision where the two kinds of error are paid for by different people. The page opens with twelve field failures, the cause behind each and the check that catches it, then covers the budget on the device, the data problem, the threshold as a policy, what has to be recorded, and seven documented failures.
• DeftechC2 systemsA command and control (C2) system is software that turns reports from many sources into one shared picture people can act on, and it is usually pictured as a map. It is better understood as a correlation engine with a map attached, whose central job is deciding whether two reports are one thing or two. The page opens with the twenty-five-line shared-picture checklist, then covers correlation, the two times and the clock every record carries, what happens when two nodes lose contact, interoperability as a requirement, the person in the loop as properties you can test, and seven documented failures.