Hybrid with a gateway

Reference pattern. This page describes a deployment pattern; it does not describe any client system. Components are named by role, and products are examples.

Context and constraints

Most work can be done by open-weight models inside the client's environment, but a few approved tasks benefit from a hosted frontier model, for example drafting from non-sensitive material. The client wants one place where that choice is made, enforced and recorded.

Diagram

Components (by role)

Data flow and trust boundaries

The trust boundary is the gateway's outside route. A request may cross it only if its task and data class are on the approved list. Everything else is served inside. Responses from the outside route are treated as untrusted input like any other model output.

Identity and permissions

Applications authenticate to the gateway with their own scoped keys; the gateway alone holds the outside provider's key. Users never hold provider keys.

What is logged

Every request: identity, application, task, data class, route chosen and why, model version, tokens and cost. Requests that crossed the boundary are reportable separately.

Failure modes

• Misclassification. A sensitive request is labelled as safe. Mitigations: classify by source system rather than by content where possible, default to the inside route, and review a sample of outside-routed requests.
• Provider outage or change. The outside model is withdrawn or changes behaviour. Every outside route has an inside fallback, and is re-scored on the test set when the provider changes the model.
• Library compromise. The gateway is a high-value component. Pin and verify its dependencies (see security controls).

Alternatives

Inside-only, with either on-premises or cloud hosting, when no task justifies the outside route.

Related decisions

Which tasks may use the outside route; whether redaction is required; budget limits; who approves changes to the policy.