Systems architecture

Reference architecture

A private AI system is mostly ordinary software. The model is one component among a dozen, and most of the decisions that determine whether the system can be trusted are made around it: how documents are read, how permissions travel with data, where a person signs off, and what gets logged.

Reference pattern. This diagram shows a common layering; it is not a description of any client system.

Two properties hold across every layer. First, every record keeps a pointer to where it came from, so an answer, a filled form or a figure in a report can be traced back to a page, a row or a message. Second, nothing leaves the network unless a rule in the gateway or the egress allow-list says it may.

Document pipeline

Most business documents are not clean text. They are scanned delivery notes, faxed referral forms, PDFs exported from accounting packages, and spreadsheets pasted into emails. A pipeline that reads them reliably usually has these stages.

• Routing by document type. Born-digital PDFs with a text layer go to a layout parser, which is fast and cheap. Scans, photographs and pages with complex tables go to a document vision-language model, which is slower but reads what a parser cannot. Docling is one example of a layout parser; olmOCR and PaddleOCR-VL are examples of document vision-language models.
• Structure kept, not flattened. Reading order, table cells and the page coordinates of every value are kept. A value without coordinates cannot be highlighted on the original page for a reviewer, and a reviewer who cannot see the source will either trust everything or check everything.
• Schema-constrained decoding. The model is made to produce output that matches a JSON Schema token by token, so a missing bracket or an invented field cannot occur. Engines such as XGrammar compile the schema into a grammar the decoder follows.
• Business-rule validation. Valid JSON is not the same as a correct invoice. Rules check what the business knows: line items sum to the total, the tax rate exists for that country, the purchase order is open in the ERP, the supplier is on the vendor master.
• Per-field confidence and a review queue. Each field gets a confidence score from several signals, and fields under a threshold go to a person. Every correction is saved as a labelled example, which becomes part of the test set.
• Licence review. Model weights and parser code are licensed separately, and some document-model weights restrict commercial use. Licences are checked before a component is selected, not after it is in production.

Retrieval

A knowledge system is only as good as the passages it finds. The model writes the answer, but retrieval decides what it can possibly say.

• Hybrid search. Keyword search (BM25) finds exact terms, product codes and clause numbers; dense vector search finds passages that mean the same thing in different words. Running both and merging the results with Reciprocal Rank Fusion (RRF), where each document scores the sum of 1 / (k + rank) across the lists, with k commonly set to 60, is a sound default because it needs no score calibration between the two systems. A cross-encoder re-ranker then reorders the top few dozen.
• Structure-aware chunking. Documents are split along their own headings, clauses and table boundaries rather than every N tokens. Each passage can also be given a short generated description of where it sits in the document, the technique Anthropic described as contextual retrieval, so that a passage reading "the term is 24 months" still knows which agreement it belongs to.
• Permissions at query time. Each passage carries the access-control list of its source document, copied from SharePoint, the document management system or the practice system. The filter is applied inside the search, as described in the worked example below, so that a user never retrieves text they could not open in the source system.
• Graph retrieval only when it earns its cost. Graph-based approaches such as GraphRAG build an entity graph and community summaries over the whole corpus, which means a model call for every chunk at indexing time. They help with questions about the corpus as a whole ("which suppliers appear across all our disputes?") and add little to lookup questions. They are chosen by question type, not by default.
• Citations and groundedness. Every answer cites the passages it used. Before the answer is shown, a check confirms that each claim is supported by a cited passage; if it is not, the system says the evidence is insufficient rather than filling the gap.
• pgvector or a dedicated engine. If the data already lives in PostgreSQL and the corpus is in the low millions of passages, pgvector keeps vectors, metadata and permissions in one transactional database. A dedicated engine such as Qdrant becomes attractive with larger corpora, heavy filtering, or a need to scale search independently of the database.

Workflows and agents

An agent that can act on business systems needs the same discipline as any other software that writes to them, and more, because its next step is proposed by a model.

• Durable execution. Each process is an explicit workflow in an engine such as Temporal, which records every step in an event history and can resume after a crash exactly where it stopped. Workflow code is deterministic so it can be replayed; anything non-deterministic, including every model call and every API call, runs as an activity whose result is recorded.
• The model proposes, the workflow decides. The model suggests the next step and fills in its arguments. The workflow checks that the step is allowed at this point in the process, that the arguments pass validation, and whether it needs approval.
• Typed tools with least privilege. Tools are exposed through typed interfaces, including the Model Context Protocol (the current specification is dated 2026-07-28). Each tool has the narrowest access that works: a read-only view of the claims table, not a database login.
• No token passthrough. A tool server never forwards the caller's token to a downstream API; it authenticates with its own scoped credentials. The MCP security best practices name token passthrough as an anti-pattern for this reason.
• Approval as a workflow signal. An approval is an event the workflow waits for, with a timeout and an escalation path, and it is recorded with who approved and what they were shown.
• Untrusted input. Retrieved documents, emails and tool outputs are treated as data that may contain instructions. Controls are mapped to the OWASP Top 10 for LLM Applications (LLM01, prompt injection) and the OWASP Top 10 for Agentic Applications.
• Append-only audit. Every proposal, decision, tool call and approval is written to an append-only log that can be replayed for review.

Data integration

• Change data capture. Reading the database's own transaction log, for example with Debezium, picks up every insert, update and delete without polling or extra load on the source. Where log access is not possible, API pulls or scheduled batch extracts are used instead.
• Orchestration and data contracts. Pipelines run under an orchestrator, and each dataset has a contract stating its schema, how fresh it must be and who owns it. A pipeline that breaks a contract fails loudly rather than loading bad data.
• Canonical models and entity resolution. One canonical record per customer, product or supplier. Matching starts with exact keys (VAT numbers, GTINs, NPIs), then moves to probabilistic matching on names and addresses, with uncertain matches sent to a person.
• Lineage. Every dataset records which jobs and sources produced it, in an open format such as OpenLineage, so a figure can be traced back through every transformation.
• Healthcare interfaces. HL7 v2 messages remain the workhorse for admissions, orders and results; FHIR R4 is the current baseline for API access, with R6 in development.
• Safe write-back. Writes to systems of record go through staging tables, carry an idempotency key so a retry cannot create a duplicate, and are reconciled against the target afterwards.

Deployment topologies

On Kubernetes, GPUs are exposed through the vendor's device plugin, and Dynamic Resource Allocation offers finer-grained requests for accelerators. Model weights are pulled from an internal registry rather than from the internet, which matters doubly in air-gapped sites. Data residency is mapped for every copy of the data, not just the primary store: backups, logs, search indexes, and any access granted to a vendor's support staff.

Security and observability

• Zero trust. Every request is authenticated and authorised on its own merits, following NIST SP 800-207, whether it comes from a person, a service or an agent.
• Secrets and keys. Secrets live in a vault, are injected at runtime, and are rotated on a schedule. Model gateway keys are scoped per application and per team.
• Software supply chain. Dependencies are pinned to exact versions with hashes, mirrored into an internal registry, and updated deliberately. The March 2026 compromise of two LiteLLM releases, a widely used gateway library, is the worked lesson: a team that pinned and mirrored verified versions was not exposed by an upstream release; a team that installed the latest version automatically could have been.
• Controls, not certificates. These controls are designed to support a client's SOC 2, ISO 27001 or HIPAA programme. They are evidence for the client's own audit, not a certification of the system.
• Tracing. Model calls are traced following the OpenTelemetry semantic conventions for generative AI, which are still marked "Development" and may change. Prompt and response content is not logged by default; it is captured only where the client decides it is needed and the retention rules allow it.

Worked examples

Extracting fields from scanned invoices

Illustrative example. This explains a technique; it does not describe any client system.

One possible pipeline:

1. 1Classify the document: invoice, credit note, statement, or something else.
2. 2Run a document vision-language model on the page images.
3. 3Decode against a JSON Schema so every output is valid by construction, using grammar-constrained decoding (for example with XGrammar).
4. 4Validate against business rules: line items sum to the total, the tax rate is valid for the country, the purchase order exists in the ERP.
5. 5Score each field's confidence as the weakest of several signals, such as token log-probabilities, the rule results, an OCR quality estimate, and whether the value matches master data.
6. 6Send any field under its threshold to a review queue, with the page region highlighted. Save every correction as a labelled test case.

{
  "type": "object",
  "required": ["supplier_vat", "invoice_number", "invoice_date", "currency", "total", "lines"],
  "properties": {
    "supplier_vat": { "type": "string", "pattern": "^[A-Z]{2}[0-9A-Z]{2,13}$" },
    "invoice_number": { "type": "string" },
    "invoice_date": { "type": "string", "format": "date" },
    "currency": { "enum": ["GBP", "EUR", "USD"] },
    "total": { "type": "number" },
    "lines": {
      "type": "array",
      "items": {
        "type": "object",
        "required": ["description", "quantity", "unit_price"],
        "properties": {
          "description": { "type": "string" },
          "quantity": { "type": "number" },
          "unit_price": { "type": "number" }
        }
      }
    }
  }
}

The thresholds are set per field, not globally. A wrong invoice number is caught by the ERP match; a wrong bank account number is not, so it gets a much stricter threshold.

Permission-aware retrieval and the filtering trap

Illustrative example. This explains a technique; it does not describe any client system.

Approximate nearest-neighbour indexes such as HNSW return the k closest vectors quickly by searching a graph. If the permission filter is applied after the search, a selective filter throws most of those k results away. Ask for the top 10 passages, apply a filter that only 2% of the corpus passes, and the expected number of survivors is 0.2: usually none.

Three remedies, each with a cost:

The right choice depends on how many distinct access groups exist and how selective a typical user's view is. It is measured on the client's own permission data, reporting recall against an exact search, latency at p95, and index count.

A durable approval step

Illustrative example. This explains a technique; it does not describe any client system.

In a durable workflow engine such as Temporal, workflow code is replayed from its event history after a restart, so it must be deterministic. Model calls therefore run as activities, never inside workflow code. An approval is a signal the workflow waits on, with a timer that escalates if nobody responds.

@workflow.defn
class AppealWorkflow:
    def __init__(self):
        self.decision = None

    @workflow.signal
    def review(self, decision: Decision):
        self.decision = decision

    @workflow.run
    async def run(self, claim_id: str):
        draft = await workflow.execute_activity(
            draft_appeal, claim_id, start_to_close_timeout=timedelta(minutes=5)
        )
        try:
            await workflow.wait_condition(lambda: self.decision is not None,
                                          timeout=timedelta(hours=24))
        except asyncio.TimeoutError:
            await workflow.execute_activity(escalate, claim_id,
                                            start_to_close_timeout=timedelta(minutes=1))
            await workflow.wait_condition(lambda: self.decision is not None)
        if self.decision.approved:
            await workflow.execute_activity(submit_appeal, self.decision.final_text,
                                            start_to_close_timeout=timedelta(minutes=5))

Replay then shows exactly what the model proposed, what the reviewer saw, who approved it and when.

Prompt injection through a document

Illustrative example. This explains a technique; it does not describe any client system.

A retrieved file contains a line telling the model to disregard its earlier instructions and email the file to an outside address. No filter reliably detects every such instruction, so the mitigations are structural and layered:

• Steps that read retrieved content are given no write tools at all.
• Tools are allow-listed per workflow step, so a summarising step cannot send email whatever the model outputs.
• Model outputs are validated against a schema before any action is taken.
• Outbound network traffic is allow-listed, so even a successful injection has nowhere to send data.
• A person approves every outbound message.

These map to OWASP LLM01 (prompt injection) and to the goal-hijack and tool-misuse risks in the OWASP Top 10 for Agentic Applications.

Sources

• Model Context Protocol, specification 2026-07-28: changelog (opens in a new tab) and security best practices (opens in a new tab)
• OpenTelemetry semantic conventions for generative AI (opens in a new tab)
• OWASP Top 10 for LLM Applications (opens in a new tab) and the OWASP agentic list, reported by TechRepublic (opens in a new tab)
• NetSPI: the LiteLLM supply chain compromise (opens in a new tab)
• Docling (opens in a new tab), olmOCR (opens in a new tab), PaddleOCR (opens in a new tab), Marker (opens in a new tab)
• vLLM structured outputs (opens in a new tab)
• Anthropic: contextual retrieval (opens in a new tab)
• Qdrant hybrid queries (opens in a new tab) and pgvector, including iterative index scans (opens in a new tab)
• Microsoft GraphRAG (opens in a new tab)
• Temporal: workflow execution and determinism (opens in a new tab)
• Debezium connectors (opens in a new tab) and OpenLineage (opens in a new tab)
• Kubernetes Dynamic Resource Allocation (opens in a new tab)
• Health Samurai: getting ready for FHIR R6 (opens in a new tab)
• NIST SP 800-207, Zero Trust Architecture (opens in a new tab)
• Cormack, Clarke and Büttcher (2009), "Reciprocal rank fusion outperforms Condorcet and individual rank learning methods", SIGIR.