Evaluation sets

In one paragraph

An evaluation set is a fixed collection of real questions, paired with answers that have been checked and agreed as correct, used to score an AI system every time something about it changes. Written by the people who actually do the work the system supports, it turns "does it still work well" from an impression into something you can measure. Run again after every change, from a small adjustment to a full model upgrade, it shows whether quality held, improved, or slipped.

Why it matters

An AI system can seem to work well in everyday use and still get quietly worse after a change, because the cases that go wrong are often the unusual ones nobody happens to try by hand. A fixed set of real questions with reviewed answers catches that: run it before and after any change, and a drop in quality shows up as a number rather than a feeling that something is off. Because the questions come from the people who do the work, the set reflects what actually matters to get right, rather than a generic idea of what a good answer looks like.

How it works

• The questions are real ones, drawn from the actual work the system supports, not invented examples.
• The answers are reviewed and agreed by the people who do that work, so there is a clear standard to score against.
• Every change to the system, a tuning pass, a model upgrade, a change to how it retrieves information, is scored against the same set before it is treated as an improvement.
• A regression, where the system gets something wrong that it previously got right, shows up clearly when the set is run again, rather than being noticed only once someone complains.
• The same set, kept and reused, becomes the evidence a regulator or an auditor asks for when they want to know how a system's quality is checked and tracked.

What it looks like in practice

Someone looks after an assistant that answers questions from an internal knowledge base. Before making any change to how it works, they gather a set of questions their own people actually ask, with answers checked and agreed by someone who knows the material well. After a change, such as pointing the assistant at an updated model, the same set is run again. Where the answers match the agreed ones, the change is confirmed to be safe. Where they do not, the team knows exactly which questions to look at before the change goes any further.

How this connects to our work

Evaluation sets are how every change is scored under managed AI and continuous improvement, and how a task is shown to be safe to move to a smaller model during cost and model optimisation, before either is treated as an improvement.