Model gateways

In one paragraph

A model gateway is a single point that every AI request in an organisation passes through on its way to a model. Instead of each application holding its own connection to its own model, they all ask the gateway, and the gateway decides which model answers, checks that the request is allowed, and writes down what happened. It is the same idea as putting one door on a building rather than a door on every room: not to slow anybody down, but so there is one place where the rules live and one place that knows who went where.

Why it matters

Without a gateway, the choice of model gets made once per application, usually by whoever built it, and then stays made. Changing it later means finding and editing every place that choice was written. Working out what the organisation is spending means adding up several bills. Working out who asked what means asking several teams. And a policy about which kinds of work may leave your environment has nowhere to live except in people's memory.

With a gateway, those become settings rather than projects. A model can be swapped for a newer one in configuration. A class of request can be pinned inside your own environment because the data it carries is sensitive. Spend can be broken down by team without anybody reconciling anything. And the record of what was asked, by whom, and which model answered exists by default rather than being assembled when somebody asks for it.

How it works

• Applications send their requests to the gateway rather than to a model directly, using one interface regardless of what ends up answering.
• The gateway authenticates the request against your own identity provider, so every call is attached to a person or to a named system identity.
• A routing policy decides which model handles this kind of request: a small model running inside your environment, a larger open-weight model on your own capacity, or a frontier model reached over an API.
• Rules can pin particular kinds of request to particular destinations, which is how a policy such as "anything containing customer records stays inside our environment" becomes a setting rather than a guideline.
• The gateway records the request, the identity behind it, the model chosen, the reason it was chosen, and what it cost, which is what makes both the bill and the audit trail possible.
• A fallback is configured in advance for the cases where the first choice is unavailable, so a provider having a bad afternoon does not become an outage in your business.

What it looks like in practice

An internal assistant, a document summariser and a workflow automation are built by three different teams at three different times. Each one calls the gateway. Routine questions over internal documents are routed to a small model running on the organisation's own hardware. Long contract summarisation goes to a larger open-weight model on the organisation's own cloud capacity. A small number of unusual drafting tasks are allowed out to a frontier API, and the policy says which ones. When a better small model appears, it is scored against the organisation's own test set and then switched on in configuration, and all three systems get it without any of the three teams changing a line.

How this connects to our work

The gateway is the centre of the private AI environment we build, because it is what makes the model choice reversible and the record automatic. It is also what the routing layer is configured in, and what makes replacing a withdrawn model a configuration change during ongoing improvement rather than a rebuild.