AI Agents

Workflow or agent: decide this first

Most systems described as agents are workflows, and most systems built as agents should have been. The distinction is precise and worth applying deliberately: in a workflow your code decides what happens next and calls the model for the steps that need language or judgement; in an agent the model decides what happens next. Here is the decision table first, because it is the part that saves money, and because by the time the loop is built the decision has been made by default.

The loop, and the arithmetic that governs it

Mechanically an agent is small. You describe a set of tools — each a name, a description and a parameter schema — and send them with the user’s request. The model answers either with text or with a request to call one of the tools and the arguments to use. Your code runs the tool, appends the result to the conversation, and calls the model again. That repeats until the model answers with text instead of a tool call, or until you stop it. There is no planner and no state machine: the loop is the architecture, and that is exactly why building one is not the hard part.

Two things about that loop are not obvious until you run one, and both are arithmetic rather than judgement. The conversation grows on every iteration, because each tool result is appended and everything is resent — so step twelve carries all eleven results before it, and cost per step rises as the task goes on. And accuracy compounds downward: every step has to be right for the outcome to be right, so a step that is right ninety-five per cent of the time gives a ten-step task that is right about sixty per cent of the time.

The tool surface is the design

Every tool is a capability you have granted, so grant the fewest that let the task be done, scoped to this user’s permissions rather than the service’s. The descriptions are part of the prompt and are read by the model on every call — a description that says when not to use the tool prevents more mistakes than any instruction elsewhere. And every argument the model produces is untrusted input: validate it in your own code, exactly as you would a request body from the internet, because a model that has read an injected instruction will produce a well-formed, entirely wrong call.

Granularity is where the compounding arithmetic becomes a design rule. Mirroring an internal API one endpoint at a time feels like the neutral choice and is not: it turns one unit of work into five chances for the model to choose wrong, and it moves the sequencing, the validation and the transaction out of code and into a loop that cannot be tested. A tool called create_invoice leaves one decision and keeps all three inside code where they belong. The test to apply is whether a tool corresponds to something a person would name as a task, rather than to something a database would name as an operation.

Two details of the tool contract matter more than they look. A tool result is read by the model and acted on, so a failure has to come back as an explicit, readable result — "no invoice matches that number" rather than an exception the harness swallows or an empty string the model interprets as an empty list. And every tool that changes state needs an idempotency keyidempotency keya request id that makes a retry safeA unique value a client sends with a request that changes something — a payment, an order — so the server can recognise a repeat and return the first result instead of acting twice. The server has to store the keys and decide how long to keep them; a retry that arrives after its key has expired is treated as new.Full article →All terms checked server-side, because a retry, a timeout that succeeded anyway, or a model that simply calls it twice are all ordinary events in a loop, and none of them should send the second email.

Five failure modes that only exist in a loop

Each of these is specific to handing control flow to the model. None of them appears in a workflow, which is the clearest statement of what the transfer actually costs — and the right-hand column is what has to be written in code, not requested in a prompt.

What stands between the model and the effect

Start every agent read-only. An agent that can search, fetch and summarise but cannot change anything is useful on its first day and cannot cause an incident, and it earns the write permissions one at a time as its trajectories prove trustworthy. Where a write is genuinely needed early, make it reversible or gate it behind a person — and make the human step meaningful, because an approval dialog that appears forty times an hour is clicked without reading and is worse than no gate at all, since it manufactures the appearance of oversight.

1. 1.Scope

   1. 1.1The agent starts read-only, and every write permission is earned by a record of trajectories.
   2. 1.2There is a written list of what it is allowed to do, and it is shorter than the list of what it could do.
   3. 1.3Every task whose steps can be enumerated has been moved into a workflow first.

2. 2.Hard limits, enforced in code

   1. 2.1A maximum number of steps per task, enforced by the runner rather than requested in the prompt.
   2. 2.2A token and spend cap per task that aborts the run rather than logging a warning.
   3. 2.3A wall-clock timeout, and a way for a person to stop a run that is already going.

3. 3.Tools

   1. 3.1Each tool does one meaningful unit of work rather than mirroring one API endpoint.
   2. 3.2Every tool description says when not to use the tool, not only what it does.
   3. 3.3Every argument is validated in your code as untrusted input, exactly like a request body.
   4. 3.4Every state-changing tool takes an idempotency key, checked server-side.

4. 4.Permissions

   1. 4.1Tools run with this user’s permissions, never the service account’s.
   2. 4.2Anything irreversible sits behind a human confirmation rare enough that it is actually read.
   3. 4.3Production data and production credentials are not reachable from a development or test run.

5. 5.Observability

   1. 5.1The full trajectory is recorded: every call, every argument, every result, every decision.
   2. 5.2The user can see what the agent is doing while it does it — the tool, the target, the result.
   3. 5.3The trajectory is shown afterwards, so a wrong outcome can be understood rather than merely reported.

6. 6.Evaluation

   1. 6.1Fixed scenarios run in a sandbox with seeded data and tools that record instead of acting.
   2. 6.2Two things are scored separately: did it reach the right end state, and did it do anything it should not have.
   3. 6.3Cost and step count per task are tracked as distributions, because the average hides the run that ended a budget.

One line deserves expanding, because it is the one people argue about: 4.2, making confirmation rare. The instinct when an agent does something alarming is to add a confirmation step, and the second instinct is to add another. Past a certain frequency the dialog stops being a control and becomes a keystroke — and it is then strictly worse than nothing, because the audit log now contains a human approval for every action, which is exactly the evidence somebody will produce to show the process worked. If confirmations are frequent, the fix is to narrow what the agent may do, not to ask more often.

Evaluate trajectories, not answers

Two runs can reach the same result with wildly different paths, and the one that took nine steps and read a file it had no business reading is the one that will cause an incident. So record the whole trajectory — every call, argument, result and decision — and score two things separately: did it reach the correct end state, and did it get there without doing anything it should not have. A run that reached the right answer by the wrong path is not a success with a note attached; it is the next incident, currently passing.

Fixed scenarios in a sandbox make that repeatable: seeded data, tools that record the call instead of performing it, and the same starting state every run. Because the trajectories vary, run each scenario several times and read the spread rather than one result — a scenario that succeeds four times in five is a different thing from one that succeeds every time, and the average of a single run tells you neither. Track cost and step count the same way, as distributions, because the mean hides the one run in two hundred that spent a month’s budget.

Finally, the interface decides how the failures land. Show the user what the agent is doing while it does it — the tool, the target, the result — because a spinner over a two-minute run gives them nothing to judge and nowhere to intervene. Let them stop it. Show the trajectory afterwards, so a wrong outcome can be understood rather than merely reported. An agent whose work is visible gets corrected early by the person best placed to notice; an agent that works in silence gets switched off after the first incident.

Five runs, and what each one cost

These span three years of the technology getting substantially better, and the failures do not change shape: long trajectories go sideways, limits written in prompts are not limits, and the interesting damage is done by an agent that is working exactly as built. Two of them are published by the people who ran the agent, which makes them the most useful entries in the table.

Where it degrades

The honest limit is that an agent trades a bounded system for an unbounded one, and no amount of engineering buys the bound back — it only decides how much damage the unbounded part can do. You cannot know in advance what a run will cost, how long it will take, or which tools it will use, and every practice on this page is a way of making that acceptable rather than a way of making it false. A team that needs those three things knowable is describing a workflow, and should build one.

• An agent where a workflow would do, paying in cost, latency and debuggability for flexibility nobody needed.
• Limits written in the prompt, which are requests made of the component whose judgement you are trying to bound.
• No hard limit on steps or spend, so one task can run until somebody notices the bill.
• Tools that mirror an internal API one call at a time, turning one job into ten chances to go wrong.
• Tool arguments trusted because the model produced them, with no validation in your own code.
• Non-idempotent side effects, so a retry sends the second email nobody asked for.
• The service account’s permissions instead of the user’s, so the agent can reach what the user cannot.
• Production credentials reachable from a development run, with a code freeze announced in a prompt.
• Confirmation prompts frequent enough to be reflexive, which manufacture oversight without providing it.
• Evaluating only the final answer, so a run that reached the right result by the wrong path looks like a success.

When to use it

Related pages

• DevelopmentLLM FundamentalsA large language model (LLM) scores every possible next token against the text so far, lets a sampler pick one, and repeats that until the answer ends. Chat, tool calls, reasoning and agents are all that single operation run many times. The page opens with twelve things a model does that look like defects and are not, each with the mechanism behind it and the fix, then covers the loop itself, why the same sentence costs more in Ukrainian, what the context window really holds, the limits that cannot be patched out, five documented failures, and what all of it costs in production.
• DevelopmentRAG ArchitectureRetrieval-augmented generation (RAG) answers questions about data a model was never trained on: find the relevant passages first, then put them in the prompt. Requiring the answer to come from those passages makes RAG a search system with a language model on the end, which is why almost every wrong answer is a retrieval failure rather than a generation one. The page opens with the twenty-four-line build checklist, then nine symptoms and where each is actually fixed, the two pipelines, how to measure retrieval on its own, the permission problem that turns a demo into an incident, the alternatives, and five documented failures.
• DevelopmentLLMOpsLLMOps is the practice of running a feature built on a language model in production, where the same input does not reliably produce the same output. It exists because everything that makes software releasable — a test that fails, a diff that explains a change, a rollback that restores yesterday — assumes that it does. The page opens with the twenty-four-line production checklist, then covers what each practice is replaced by, why the deployed unit is four things rather than one, evaluation as the test suite, what a trace has to contain, how cost and latency behave under real traffic, and five documented incidents.