Single on-premises GPU server

One server, one workload, data that stays in the building

Published: 8 November 2026
Last reviewed: 8 November 2026
Re-checked every: 180 days
Reviewed by: Ales Analytics engineering
Used in: Private AI infrastructure

Reference pattern. This page describes a deployment pattern; it does not describe any client system. Components are named by role, and products are examples.

Context and constraints

One team, one workload, data that must stay in the building, and a modest number of users: for example, a document-processing or knowledge system for a few dozen staff. The firm has a server room or a rack in a colocation facility, and an IT lead who can look after one more server.

Diagram

Reference pattern: one GPU server inside the building. The package mirror is the only outbound route.

Components (by role)

Role	Notes
GPU server	One or two data-centre or workstation GPUs, sized from the model and concurrency (see sizing a self-hosted deployment)
Serving engine	Serves one open-weight model, possibly quantised
Model gateway	Single entry point for every model call: keys, routing, logging
Application and workflow engine	The user interface and the process logic
Stores	Relational database, object store for documents, search index
Identity	The firm's existing directory, through single sign-on

Data flow and trust boundaries

Documents enter through ingestion jobs that read from source systems with a read-only service account. Everything after that stays on the server. The only trust boundary crossed at query time is between the user's browser and the reverse proxy. There is no route to a hosted model.

Identity and permissions

Users sign in through the existing directory. Group membership is read at query time, so a user retrieves only passages from documents they could open in the source system. Administrative access to the server is separate from application access.

What is logged

Who asked, when, which documents were retrieved, which model version answered, latency and token counts. Prompt and response content is not logged by default.

Failure modes

Single point of failure. One server means one place to fail. Mitigations: spare parts, a documented rebuild from configuration, and backups of the stores tested by restoring them.
Capacity. Load beyond the sizing shows up as rising time to first token and queueing. The fix is a second GPU or a smaller model, decided from measurements.
Updates. Model and software updates come through an internal mirror and are tested on the frozen test set before rollout.

Alternatives

A private cloud tenancy avoids owning hardware. A hybrid gateway adds a hosted model for approved tasks. An air-gapped deployment removes even the update route.

Model choice and licence, quantisation level, whether to keep a cold spare, backup retention. See how we record decisions.

Should you have any questions regarding this page, please contact us at solutions@alesanalytics.com.