Fine-tuning

Eighteen gates before a training run

A training run is a command that takes a few hours, and that cheapness is exactly what makes this page necessary: the expensive part is everything the run commits you to afterwards. Here is the whole list of gates first, numbered so a line can be cited in a planning meeting. A project that cannot answer 1.1 and 6.1 is not ready regardless of how good the data is, and those two are the ones most often skipped — the first because nobody wants to admit the prompt was never measured, the second because it is somebody else’s problem until it is nobody’s.

1. 1.Have the cheaper rungs been tried?

   1. 1.1The best prompt you can write has been measured on a held-out set, and you know its number.
   2. 1.2Retrieval has been tried for everything that is a fact rather than a behaviour.
   3. 1.3A stronger base model has been tried and priced, because it changes in an afternoon.

2. 2.Is this the kind of problem it fixes?

   1. 2.1The problem is behaviour, form or judgement — not facts, prices, policies or anything that must be current.
   2. 2.2The requirement is stable enough that it will not change weekly, since each change costs a training run.
   3. 2.3Volume is high enough that a shorter prompt pays back the training and the maintenance.

3. 3.Can you build the dataset?

   1. 3.1Labelling rules are agreed in writing before anybody labels anything.
   2. 3.2Examples come from real production input, including the messy ones and the ones that should be refused.
   3. 3.3Several hundred genuinely good examples exist or can be made, and somebody owns making them.
   4. 3.4A held-out slice is set aside before training starts and is never trained on.

4. 4.Are you allowed to train on it?

   1. 4.1Personal data is removed or masked before training, not filtered at the output.
   2. 4.2You have the right to train on this content — customer data under a processing agreement, licensed corpora, another provider’s outputs.
   3. 4.3You know whether your provider trains on what you submit, which is a different question for a dataset than for a chat message.

5. 5.Can you prove it worked?

   1. 5.1The comparison is against the best prompt on the base model, on the same held-out cases — not against nothing.
   2. 5.2The evaluation setevaluation setthe test suite for a modelA fixed collection of inputs with a way to judge each output, run before and after every change to a prompt, a model or a retrieval step, and reported as rates rather than pass or fail. Its job is to fail: a set built from cases the system already handles reports a high score forever and detects nothing.Full article →All terms keeps a handful of general cases, not only the task you trained for.
   3. 5.3The gap is large enough to justify everything in group 6.

6. 6.Who owns it afterwards?

   1. 6.1There is a named owner for retraining, by name rather than by team.
   2. 6.2There is an agreed cadence for re-running it when the provider ships a better base model.
   3. 6.3There is a documented path back to the base model plus a prompt, in case the tuned model has to be abandoned.

The four rungs, and the rule about climbing

There are four ways to make a model behave the way your product needs, and they differ by roughly an order of magnitude each in cost, in the time to try something, and in how much you have to own afterwards. The rule that keeps teams out of trouble is simple: never climb a rung you have not been pushed off. Most projects that fine-tuned early discover afterwards that a better prompt and a retrieval step would have got them the same result in an afternoon — which is what gate 1.1 exists to find out before, rather than after.

• Prompting

  Minutes to change, nothing to own. Fixes format, tone and most behaviour. Costs input tokens on every call, forever.

• Retrieval

  Days to build. The only way to give the model facts it can cite, update and have restricted per user.

• Fine-tuning

  Weeks, mostly spent on data. Moves consistent behaviour into the weights and shortens every prompt that follows.

• Continued pretraining

  Months and a serious budget. Reserved for a domain whose language the base model genuinely does not speak.

There is one asymmetry in the ladder worth naming, because it changes how the decision ages. The lower rungs get better on their own: a new base model arrives, your prompt and your retrieval pipeline inherit every improvement in it for the cost of changing one string. The upper rungs do the opposite — a tuned model is pinned to the base it was trained on and inherits nothing, so the gap between "what we have" and "what is available" widens every quarter until somebody repeats the exercise. The ladder is therefore not a static picture of cost; it is a picture of which decisions keep paying and which start charging.

What the methods differ on

Two distinctions cover almost everything you will meet. The first is how much of the model you actually change. Full fine-tuning updates every weight, needs the hardware to hold optimiser state for the whole model, and produces a complete new model to store and serve. Parameter-efficient methods — LoRA is the one you will meet by name — freeze the original weights and train a small set of additional ones alongside, typically a fraction of a per cent of the total.

The second distinction is what you show the model. Supervised tuning shows it input and the correct output, and it is the right tool whenever a correct answer can be written down — a classification, an extraction, a document in your house format. Preference tuning shows it two answers and which one a human preferred, and it exists for the cases where you cannot write the right answer but can recognise it: tone, helpfulness, how firmly to refuse. Preference data is much more expensive to collect and is worth reaching for only after supervised tuning has taken you as far as it goes.

The dataset is the project

The training run is a command that takes a few hours. Everything that determines whether it worked happens before it, in the dataset, and teams consistently budget this the wrong way round. A few hundred genuinely good examples beat tens of thousands of scraped ones, because the model learns exactly what you show it — including the inconsistencies. Two annotators who disagreed about the same case teach the model to be inconsistent in precisely that place, so gate 3.1, agreeing the labelling rules in writing before anyone labels anything, is not process for its own sake.

The dataset also has to look like production, which sounds obvious and is routinely violated. Real input is messy: typos, mixed languages, pasted email signatures, empty fields, questions the system should refuse. If every training example is a clean, well-formed request, the model learns that the world is clean and behaves unpredictably the first time it is not. Include the awkward cases and the refusals deliberately, in roughly the proportion they occur.

Proving it worked

Hold examples back before you start, and never train on them. Without a held-out set you have no way to distinguish a model that learned the task from one that memorised your examples, and the second looks perfect on everything you measured. That much is standard practice. The part that is not standard practice, and is the reason this section exists, is what you compare the held-out score against.

The comparison that matters is not the tuned model against nothing — it is the tuned model against the best prompt you managed on the base model, on the same held-out cases. Against nothing, every tuned model is a success, which is why almost every internal report reads like one. Against a serious prompt, a large share of them turn out to have bought two or three points and a maintenance obligation, and knowing which kind you have is the difference between a decision and a purchase.

One practical note on the size of the gap. A difference of a point or two on fifty held-out cases is one or two cases, which is noise — the same set run twice on the same model will move that much on its own. If the decision rests on a small gap, either grow the held-out set until the gap is bigger than the noise, or accept that you cannot tell them apart and take the option with less to own. That second outcome is a real and respectable result, and it is much cheaper to reach here than in the quarter after the base model changes.

What it costs elsewhere, and who pays it later

A model tuned hard on a narrow task gets worse at things it used to do. Ask a model trained only to emit classification labels to explain its reasoning and you may get a label. That is the expected consequence of pushing the weights toward one distribution, and the uncomfortable part is that an evaluation set containing only the task you trained for cannot see it: the number on your report keeps improving while something nobody is measuring goes the other way.

That trade-off has a safety-shaped version worth knowing about, because it surprises people who assumed alignment is a property of the base model rather than of its current weights. Published work has shown that a small number of adversarial examples and a few dollars of fine-tuning can strip most of a model’s refusal behaviour, and that even entirely benign fine-tuning degrades it measurably. If your product relies on the base model declining certain requests, that reliance does not automatically survive your training run, and gate 5.2 — keeping general cases in the evaluation set — is what turns it from an assumption into a check.

The bill people forget is the one that arrives later. A tuned model is pinned to the base it was trained on, so when the provider ships a better base — which now happens several times a year — your tuned model does not inherit any of it. You are choosing between staying on an ageing base and repeating the whole exercise: retrain, re-evaluate, re-deploy. Gate group 6 is the cheapest possible version of that conversation: a name, a cadence, and a written way back to the base model plus a prompt if the answer turns out to be that the gap has closed.

Serving splits into two very different situations. A hosted provider takes your dataset, returns a model id, and the rest of your code is unchanged — you pay a premium per token and you are done. Self-hosting an open-weights model with your adapter gives you control over data residency and unit cost at high volume, and hands you a GPUGPUGraphics Processing Unit — a processor for parallel mathsA processor built from thousands of simple cores that apply the same operation to large arrays of numbers at once. That shape happens to be exactly what training and running models needs, which is why GPUs long ago stopped being only about graphics.All terms fleet to run, with capacity planning, batching, quantisationquantisationfewer bits per numberStoring a model’s weights — or an index’s vectors — with fewer bits per number, typically 8-bit integers instead of 32-bit floats, so they take less memory and run faster. The cost is some accuracy, and whether that cost is acceptable has to be measured on your own data, not assumed.All terms trade-offs and an on-call rota. That second path is a platform commitment, not a deployment step, and it is worth naming as such before the decision is made by default.

Five training runs and what they taught

Two of these are older than the current generation of models and are here because the mechanism has not changed: what is in the data is what comes out, and nothing downstream repairs it. The other three are about the decision rather than the data — what imitation actually copies, how fast a domain-trained model is overtaken, and what a training run does to behaviour nobody was training.

Where it degrades

The honest limit is that fine-tuning is a way of buying consistency, and consistency is not the same as correctness. It will make a model do reliably whatever the dataset showed it, including the parts nobody meant to show it, and it will do so with no way to cite, update or explain any individual behaviour afterwards. Everything below is a version of expecting it to do something else.

• Fine-tuning before a serious attempt at prompting and retrieval, which usually would have been enough.
• Training on documents to teach facts, producing fluent answers with quietly wrong details.
• No held-out set, so a memorised dataset is indistinguishable from a learned task.
• Comparing the tuned model against nothing instead of against the best prompt on the base model.
• Treating a two-point gap on fifty cases as a result, when the same set run twice would move that much.
• Inconsistent labels, which teach the model to be inconsistent in exactly those places.
• Clean training data for a messy production input, so the first malformed request behaves unpredictably.
• Measuring only the trained task, so the abilities the run cost you never appear on any report.
• No plan and no owner for retraining, leaving the tuned model pinned to a base nobody wants to be on.

When to use it

Related pages

• DevelopmentRAG ArchitectureRetrieval-augmented generation (RAG) answers questions about data a model was never trained on: find the relevant passages first, then put them in the prompt. Requiring the answer to come from those passages makes RAG a search system with a language model on the end, which is why almost every wrong answer is a retrieval failure rather than a generation one. The page opens with the twenty-four-line build checklist, then nine symptoms and where each is actually fixed, the two pipelines, how to measure retrieval on its own, the permission problem that turns a demo into an incident, the alternatives, and five documented failures.
• DevelopmentPrompt EngineeringPrompt engineering is the work of writing and maintaining the instruction a language model acts on, as a specification rather than an incantation. The page opens with a twenty-four-line review checklist you can run against your own prompt today, then the diagnosis that decides whether the prompt is even the right thing to edit, what a prompt is made of, the techniques that genuinely move the output, the evaluation set without which none of it is measurable, injection as an architectural boundary, and five documented failures.
• DevelopmentLLMOpsLLMOps is the practice of running a feature built on a language model in production, where the same input does not reliably produce the same output. It exists because everything that makes software releasable — a test that fails, a diff that explains a change, a rollback that restores yesterday — assumes that it does. The page opens with the twenty-four-line production checklist, then covers what each practice is replaced by, why the deployed unit is four things rather than one, evaluation as the test suite, what a trace has to contain, how cost and latency behave under real traffic, and five documented incidents.