# The adaptation ladder

> The order in which to adapt a model to a task, what each rung fixes and costs, and why the evaluation set comes first.

- Web page: https://www.alesanalytics.com/docs/the-adaptation-ladder
- Markdown: https://www.alesanalytics.com/docs/the-adaptation-ladder.md
- Last updated: 2026-11-08
- Publisher: Ales Analytics

Technical page. Prompting, retrieval, fine-tuning, then preference tuning. Published 2026-11-08. Last reviewed 2026-11-08. Reviewed by Ales Analytics engineering.

## In one paragraph

The adaptation ladder is the order in which to try making a model fit a task: better prompts and context first, then retrieval, then supervised fine-tuning, then preference tuning or reinforcement learning. Each rung costs more than the last, so a rung is climbed only when evaluation shows the one below falls short. Retrieval adds knowledge; fine-tuning changes behaviour.

## How it works

| Rung                           | Fixes                                                                     | Costs                                                         |
| ------------------------------ | ------------------------------------------------------------------------- | ------------------------------------------------------------- |
| Prompt and context engineering | Unclear instructions, wrong format, missing examples                      | Hours; no training                                            |
| Retrieval                      | Missing or changing knowledge                                             | An index and a pipeline to keep it current                    |
| Supervised fine-tuning         | Consistent behaviour the prompt cannot produce                            | Labelled examples, training runs, a model to version          |
| Preference tuning or RL        | Choosing between acceptable and better outputs; passing computable checks | Preference pairs or reward functions, more careful evaluation |

The evaluation set is built before the first rung. Without it there is no way to tell whether a rung helped.

![Figure 1. The adaptation ladder.](https://www.alesanalytics.com/diagrams/ma-ladder.svg)

## Design choices and trade-offs

- **Knowledge in weights or in retrieval.** Knowledge that changes, or that must be cited, belongs in retrieval. Fine-tuning facts into weights makes them hard to update and impossible to cite.
- **How many examples.** A few hundred well-chosen examples often move behaviour noticeably with parameter-efficient methods; noisy examples move it the wrong way.
- **Forgetting.** Fine-tuning can degrade abilities the task did not exercise; parameter-efficient methods tend to forget less.

## When not to use it

When the base model already passes the acceptance tests with a good prompt, stop at the first rung.

## How we evaluate it

Each rung is compared against the previous one on the same frozen test set, with a separate check on general abilities the system still relies on, so a gain on the task is not paid for elsewhere.

## Sources

- Biderman et al. (2024), [LoRA Learns Less and Forgets Less](https://arxiv.org/abs/2405.09673)
- [Model adaptation and evaluation](https://www.alesanalytics.com/engineering/model-adaptation)
- Explainer: [Tuning and routing](https://www.alesanalytics.com/explainers/fine-tuning-and-routing)

## Contact

solutions@alesanalytics.com, or https://www.alesanalytics.com/contact.

- Every page as Markdown: https://www.alesanalytics.com/llms.txt