Choosing an open-weight model

In one paragraph

Choosing an open-weight model is a filter before it is a contest. A candidate must be licensed for the intended use, fit the hardware at the required speed, and handle the client's languages and document types; only then is it scored on the client's own test set. Public leaderboards are a way to build a shortlist, never a way to make the decision.

How it works

1. 1Licence. Permissive licences such as Apache 2.0 and MIT cover families including Qwen3.5, Gemma 4, Mistral 3, DeepSeek-V4 and gpt-oss at the time of writing. Custom licences such as the Llama 4 Community License add conditions. Check the current licence file for each model and size; this is not legal advice.
2. 2Shape and memory. Dense models use every parameter per token. Mixture-of-experts models activate a few experts per token, so compute follows active parameters but memory follows total parameters.
3. 3Context and modality. Maximum context length, whether it reads images, and how well it handles the client's languages.
4. 4Serving support. Whether the chosen serving engine supports the architecture and the quantisation formats needed.
5. 5Scores on the client's test set, per task, with confidence intervals.

Design choices and trade-offs

When not to use it

When data sensitivity, volume and terms all allow a hosted model, and no task needs to run inside the boundary, self-hosting adds operational work without a matching benefit.

How we evaluate it

Every shortlisted model runs the same frozen test set under the same prompts, at the quantisation level that would be deployed. Results are reported per task with bootstrap confidence intervals, alongside measured latency and memory on the target hardware.

Sources

• Qwen3.5 (opens in a new tab), Gemma 4 (opens in a new tab), Mistral 3 (opens in a new tab), DeepSeek-V4 (opens in a new tab), gpt-oss (opens in a new tab), Llama 4 (opens in a new tab)
• Explainer: Language models and open weights