The Enterprise Data Foundation Layer for AI

TL;DR

• The binding constraint on enterprise AI is data debt, not model quality. On ungoverned corpora, retrieval accuracy collapses to roughly 45–60% against 85–92% on governed data, a gap of thirty to forty-five percentage points. Around 80% of enterprise retrieval projects hit critical production failures, and roughly 73% of those failures originate at the retrieval stage rather than in the model. Clean, mapped, governed data is the non-negotiable base layer, and there is no model good enough to compensate for its absence.

• Naive document retrieval is the wrong default for the enterprise and should be demoted rather than deployed as the primary access pattern. It structurally cannot answer analytical or aggregation questions, and it leaks access control. The successor is a routed, layered access architecture in which a query router dispatches to structured paths (text-to-SQL over a governed semantic layer), graph paths (ontology-grounded traversal), live-tool paths, cached-context paths, and, only for genuinely unstructured prose lookup, semantic retrieval.

• Two components carry the whole design. A data atlas: a continuously-maintained, machine-readable registry of where what data is stored, who is authoritative for each attribute, how good it is and how sensitive it is. And a governed access fabric: the routed layer that consumes the atlas and enforces policy before content reaches a model. Built once and amortised across every subsequent use case, these are what make "governed data in weeks" true for the second use case and every one after it.

• Enterprise-grade extraction is system-specific and genuinely non-trivial. SAP requires the supported provisioning paths rather than raw table reads. Salesforce needs a three-way strategy across its REST, bulk and change-event APIs. Relational systems need log-based change capture. A generic connector does not exist. A certified connector taxonomy does.

• Access control must move from the document to the query, and be enforced at retrieval time. A policy checkpoint that propagates source-system row, column and document security into every AI context, with those labels surviving into the embeddings, is mandatory rather than an enhancement. Organisations using real-time access controls saved materially on average breach cost; 97% of AI-breached organisations said they lacked proper access controls.

Key Findings

1. Data debt is quantifiable and it is the ceiling. Poor data quality has been estimated to cost organisations an average of $12.9 million a year. Data teams spend the majority of their time finding and cleaning data rather than analysing it. Every rupee spent on model tuning is capped by the quality of the data feeding it, and governed data raises the same retrieval pipeline from 45–60% to 85–92%.

2. Retrieval's enterprise failure is architectural, not tunable. The published taxonomy of seven failure points, combined with benchmark evidence, shows the failures are inherent to the pattern: chunking destroys table and section structure; embedding similarity is not the same thing as relevance; multi-hop and aggregation questions fail categorically. On the FRAMES benchmark, single-step naive retrieval scores around 0.40 accuracy against 0.66 for multi-step and 0.73 with oracle documents. On multi-hop question answering it can score as low as 8.2 exact-match.

3. Structured-first access dramatically outperforms document retrieval for the questions enterprises actually ask. Grounding text-to-SQL in a semantic layer took accuracy on one standard benchmark from 20% to 87% and on another from 50% to 83%. A knowledge-graph representation lifted enterprise question-answering accuracy from 16.7% to 54.2%. Graph-based retrieval beat vector retrieval on comprehensiveness win-rates of 72–83% while using over 97% fewer tokens for root-level summaries.

4. The map has to come before the pipeline. No unification effort succeeds without first answering "where is what." That means automated crawling, profiling, personal-data classification, and an entity-centric index recording which system is authoritative for which attribute of customer, order or invoice, kept current by events rather than by an annual audit.

5. There is a well-documented enterprise cliff on realistic schemas. On benchmarks built from real enterprise schemas rather than tidy academic ones, bare frontier models drop to roughly 17–21% execution accuracy. Accuracy collapses under massive scopes, thousands of columns, abbreviated names, and knowledge scattered across a million-plus tokens of documentation. This is precisely why the semantic layer, and not the raw schema, must be the grounding surface.

6. Access control must be enforced in the retrieval flow, before content reaches the model. Academic work recommends embedding the enforcement point inside the retrieval path to strip anything exceeding the user's permissions before the model sees it, with entity-level filtering and permission re-verification. The alternative, filtering the answer afterwards, is not a control, because the model has already read the data.

Part A, What "Governed" Actually Means

Enterprise AI adoption fails at a predictable point. The demonstration works on a curated slice; production fails because the real estate is scattered across SAP, Salesforce, other ERPs, departmental databases, spreadsheets, shared drives, PDFs and email, ungoverned, unmodelled, siloed, with no unified access path, no semantic model, no lineage and no consistent access control.

Stated sharply: with clean data, AI produces reliable results; with dirty data, it hallucinates fluently and erodes user trust faster than any accuracy gain can rebuild it.

The economics justify treating data as the base rather than as a preliminary. If the marginal AI use case is dominated by data-wrangling cost, the only way to make the second use case cheap is to pay that cost once, centrally, and amortise it. That is the entire strategic case for a foundation layer: the first use case pays to build the atlas and the fabric; every subsequent use case reuses them.

"Governed" is not a vague virtue. It decomposes into seven measurable properties, each of which becomes a component specification later in this report:

• Discoverable, every asset is registered and findable.
• Modelled, raw tables and columns are mapped to business entities and metrics.
• Authoritative, for each entity attribute, one system is the source of record and conflicts are resolved.
• Access-controlled, source-system permissions propagate into every AI context.
• Traceable, column-level lineage from source to answer.
• Fresh, declared staleness limits, with detection when they are breached.
• Quality-scored, measured on defined dimensions, with gates before data reaches a model.

If you cannot say where an attribute lives, which copy wins, and who may see it, you do not have a data problem you can solve with a better model. You have an information architecture problem.

Part B, The Ingestion Layer

B.1 Reference architecture

Five tiers. Raw sources feed certified connectors, which land data in a lakehouse using an open table format, which is normalised into a canonical model, catalogued and mapped into the semantic layer, and exposed through the access fabric.

• Source tier. SAP, Salesforce, other ERPs, relational databases, file shares, SaaS APIs, email and collaboration.
• Ingestion tier. Log-based change capture for relational systems; system-specific extractors for SAP and Salesforce; metadata-driven batch for file and API sources.
• Landing tier. An open lakehouse table format. Iceberg is the safer long-term default for a vendor-neutral, multi-engine estate because it is specification-first and engine-independent; Delta where the estate is already committed to it; Hudi where high-frequency streaming upserts dominate. All three now converge on transactional guarantees, schema evolution and time travel, so the choice tracks the dominant workload and the platform rather than raw benchmarks.
• Processing tier. Stateful transformation with a bronze-silver-gold layering.
• Serving tier. The governed access fabric described in Part D.

B.2 Connector taxonomy

SAP is the hardest and usually the most valuable. SAP data cannot be reliably read straight from the underlying tables, they are unsupported as an interface and cluster and pool tables are opaque. The correct extraction spine is the operational data provisioning framework, which exposes CDS views, extractors and BW objects as delta-capable providers. From recent releases onward, CDS-view change data capture uses database triggers to capture deltas including deletes. Note that SAP restricts third-party clients from calling the provisioning API directly and now steers customers toward its own replication flows and data products. The usable decision: prefer SAP-managed data products and replication flows for support and licensing compliance; use CDS views with change-capture annotations where data products are absent; fall back to extractors only where no CDS view exists. Watch two constraints, very large CDS views hit in-memory limits and must be split, and a structure change mid-load forces a full restart.

Salesforce is tri-modal. Three APIs serve three traffic patterns and must be combined: the REST API for instant operations on small volumes; the bulk API for large historical backfills; and change data capture or platform events for near-real-time deltas that do not consume conventional API limits, respect field-level security, and cover create, update, delete and undelete. For incremental sync the system modification timestamp is the watermark, with the documented trap that back-dated records and some formula-derived fields do not update it, requiring a periodic full re-sync. Deletes must be captured inside the recycle-bin window, which is why change events are the robust path for deletion.

Relational systems. Log-based change capture reading the transaction log is the production default. Each event carries the operation, the before and after images, and a source envelope with log position, transaction identifier and commit timestamp. End-to-end exactly-once delivery does not exist; correctness is achieved at the sink through idempotent upserts, deterministic primary keys, offset versioning and merge-based reconciliation. Watermark polling is acceptable only for low-change-rate sources, and building streaming capture for data that changes weekly is a common and expensive over-engineering trap.

File shares, SaaS and email. Metadata-driven batch ingestion: a framework reads a source registry, connection, object list, extraction mode, schedule, and generates the extraction jobs, so onboarding a new source is a configuration entry rather than new code. Documents require structure-aware parsing that preserves tables and headings before anything downstream touches them.

B.3 Keeping it correct over time

Schema drift is handled in four steps: enforce a schema registry with compatibility rules at the producer; reject incompatible payloads before they enter the stream; run statistical drift detection for the semantic drift a registry cannot catch, a transaction-amount field whose mean jumps from one distribution to another is a breakage the schema will happily accept; and route genuinely breaking changes through an adapter layer and a human review gate.

Data contracts are the machine-enforceable interface between each source domain and the foundation layer: a specification covering schema, quality rules, service levels and ownership, validated in continuous integration against live sources, with breaking changes detected before they ship. Without them, a federated data architecture is just decentralised chaos with better vocabulary.

Freshness limits are declared per asset, orders within fifteen minutes, the general ledger within twenty-four hours, and a monitor compares last-successful-load timestamps against the limit and raises an alert on breach. A stale source is the most common cause of a confidently wrong answer.

Part C, The Data Atlas

Definition. A continuously-maintained, machine-readable registry answering, for the entire enterprise, where what data is stored. It sits above the catalogue and below the semantic layer, and it is the precondition for every component downstream.

Where a catalogue records that a table exists, the atlas records which real-world entities and attributes live in which systems, who is authoritative, how good the data is, how sensitive it is, and how to reach it under policy.

Why build it rather than buy a catalogue. The mature catalogues each solve part of the problem: strongly-typed metadata graphs with column-level lineage and a policy engine; pull-based ingestion with built-in profiling and quality; lightweight search. None of them natively produces an entity-centric authority index, which system is the record of truth for each attribute of customer, order or invoice, nor cross-system conflict and duplication resolution, nor an access-path descriptor a routing layer can consume. The atlas is the union of catalogue, discovery graph, entity resolution, master-data authority and classification, expressed in one schema and kept live. It can be built on an existing catalogue as the metadata substrate. It is a method and a schema, not a competing store.

The five phases.

Crawl. Automated, credentialed, read-only scanners per system class register every reachable asset. Output: a raw asset inventory. Crawling is incremental and event-driven wherever possible so the atlas never goes stale.

Profile. Each asset is profiled: per-column cardinality, null rate, minimum, maximum and quantiles, data-type inference, a value-distribution signature, format regularity and last-modified time. Profiling works from a sample rather than a full scan so it does not overload the source system.

Classify. Automated personal-data and sensitivity detection: pattern and checksum recognisers for structural identifiers, named-entity recognition for contextual ones, and context-based confidence enhancement. Every column receives a label and a confidence score. Detection is assistance; a named steward decides.

Relate and resolve. Discovered relationships between columns, value overlap, primary and foreign key candidates, joinability and unionability, form a relationship graph. Entity resolution then links records to golden identifiers using a blocking-then-matching approach: cheap candidate generation over pre-computed embeddings, then an accurate cross-encoder matcher. Published results for this method reach an F1 of 96.5% on the kind of large, messy company datasets an enterprise actually has, and reach prior state-of-the-art with at most half the labelled data.

Publish authority. For each attribute of each canonical entity, the atlas records the candidate systems, marks one authoritative, states the rule, and names the steward who owns the decision. This is the artefact that makes an access request, a correction or an erasure answerable in hours rather than weeks.

Part D, Why Document Retrieval Is the Wrong Default

The claim that retrieval-augmented generation disappoints in the enterprise is substantially correct for the questions enterprises most want answered, and the nuance matters for the design. "Retrieval" here means the naive pattern: chunk documents, embed, retrieve the top few by vector similarity, stuff into the prompt.

D.1 The documented failure modes, with mechanisms

1. 1Chunking destroys structure. Fixed-size chunking severs tables, lists and the relationship between a heading and its content. A financial table split across two chunks is semantically meaningless in both.
2. 2Embedding similarity is not relevance. Dense retrieval surfaces topically similar but not answer-bearing passages. Hybrid keyword-plus-vector retrieval with rank fusion improves accuracy by roughly 15–30% and does not solve it.
3. 3Aggregation and analytical questions fail categorically. There is no aggregation operator in similarity search. It retrieves a few chunks, not a population. This is the single most important enterprise limitation, and it is structural rather than a tuning failure.
4. 4Multi-hop reasoning collapses. Chaining facts across documents scores around 0.40 against 0.73 with oracle documents on published benchmarks, and far worse on harder multi-hop sets.
5. 5Freshness rot. Embedded snapshots go stale as policies and metrics change; the index lags the source, silently.
6. 6Access-control leakage. Once documents are chunked and embedded, source permissions are lost unless explicitly propagated. Retrieval can then surface content the requester may not see. Security teams block these systems precisely because the retrieval component has the data-access profile of a privileged system and the governance posture of a development tool.
7. 7Context pollution and cited hallucination. Irrelevant retrieved chunks degrade generation and produce fluent, cited, wrong answers, the most dangerous mode, because the citation manufactures confidence.
8. 8Evaluation difficulty and per-query cost. Faithfulness and context-precision metrics are noisy, and every query pays both retrieval and long-context generation cost.

The deeper point: enterprise questions are disproportionately structured, counts, sums, joins, as-of states, who-is-authoritative, and relational, multi-hop. Those are exactly the classes similarity search cannot serve. Its legitimate residual role is narrow: unstructured knowledge lookup, where the answer is a passage of prose rather than a computed value.

D.2 The successor: a routed access layer

The replacement is not a better retrieval technique. It is a query router that dispatches each question to the access pattern that can actually answer it, all of it grounded in the atlas and the semantic layer.

The routing decision, as a specification:

1. 1Is the answer a computed value over a population, a sum, count, average, group-by, or an as-of state? → Structured path: text-to-SQL over the semantic layer, executed against the authoritative system the atlas names.
2. 2Does it require multi-hop reasoning over entities and relationships? → Graph path: ontology-grounded traversal.
3. 3Does it need the current operational state of a record? → Live-tool path: a governed call to the system of record.
4. 4Is it a lookup over a bounded, stable prose corpus, one manual, one policy set? → Cached-context path: the whole corpus preloaded, retrieval skipped entirely.
5. 5Is it open-ended lookup over large, changing unstructured prose? → Semantic retrieval, the residual role: structure-aware chunking, hybrid retrieval, reranking.
6. 6Hybrid, numbers plus narrative? Run structured and graph in parallel and merge.

On every path: access control is evaluated at retrieval or execution, and lineage and citations are attached to the answer.

This is the concrete answer to "what replaces retrieval-augmented generation": it is demoted to branches four and five of a router whose default branches are structured and graph access grounded in the semantic layer and the authority index.

The evidence for each branch is in Key Findings above, and it is not close. Grounding in a semantic layer rather than a raw schema is what separates 20% accuracy from 87%.

Part E, The Governed Access Fabric

Certified connectors. "Certified" means a connector meets a fixed specification before it is allowed into production: it uses the system's supported extraction path; it captures deltas including deletes; it preserves the source schema and emits schema-change events to the registry; it propagates source access-control metadata into the descriptor; it emits lineage events; it declares a freshness limit; and it is idempotent at the sink. Seven properties, checked, or it does not ship.

Automated normalisation. Land raw, standardise types and encodings, canonicalise source columns to a canonical model via statistics-grounded schema matching, resolve entities to golden identifiers, conform against business rules and reference data, then publish the gold tables and write the mappings back to the atlas. The canonical model is entity-centric, customer, product, order, invoice, employee, with a stable surrogate key per entity and a crosswalk to every source key.

The semantic layer. Built bottom-up and top-down at once: bottom-up mining of the normalised model and the relationship graph proposes candidate entities, relationships and metrics; top-down, a business glossary and steward review confirm the canonical names, definitions and metric formulas, active user excludes internal employees, recurring revenue treats refunds thus. The atlas says where each attribute lives and which is authoritative; the semantic layer says what it means and how it connects. Models consume it three ways: as the grounding schema for text-to-SQL, as the graph for ontology-grounded retrieval, and as tool descriptions with business-meaningful names.

Access control at retrieval time. A policy checkpoint sits in the access path between every AI system and every data source. Before any content reaches a model, it evaluates attribute-based policies over subject, resource, action and environment, and enforces row, column and document-level decisions, allow, redact or deny. Source-system security propagates by metadata chaining: classification and entitlement tags applied at ingestion are attached to derived objects including embeddings, so labels survive into the vector store and retrieval filters by the requester's current entitlements. The system runs under the asking human's propagated identity, not a broad service account, with entitlements evaluated at runtime.

A service account with universal read access is the single most common architectural flaw we encounter, and it makes every other control cosmetic.

Lineage. Column-level lineage captured automatically by parsing compiled SQL and physical plans, emitted to an open standard so it is portable. This answers "if we change this column's type, what breaks" in milliseconds, and supplies the provenance chain from source column through transformation to answer that makes model output auditable.

Freshness monitoring. Each asset declares a maximum staleness; the monitor compares last-successful-load timestamps against it, detects breaches and alerts, and flips the asset's status in the atlas so the router can decline to use it rather than answering from stale data.

Part F, Honest Failure Modes of This Design

Semantic layer drift. If the glossary and metric definitions are not maintained, the structured path degrades. Mitigate with stewardship commitments and contract tests.

Entity-resolution errors. A false merge creates a wrong golden record that poisons every downstream answer. Mitigate with human review of low-confidence matches and merges that can be reversed.

Router misclassification. Sending an analytical question to the retrieval branch reproduces exactly the failure the router exists to prevent. Mitigate with a conservative default toward structured and graph paths, and confidence thresholds that fall back to "I cannot answer that reliably."

The policy checkpoint as a bottleneck. Centralised policy evaluation can become a latency and availability problem. Mitigate with policy caching and attribute pre-computation.

Extraction fragility. Vendor restrictions and structure-change restarts can break pipelines. Mitigate by standardising on the vendor-supported paths even where an unsupported one is easier.

Over-engineering. Building streaming change capture for data that changes weekly wastes months. Mitigate with a per-class sync decision, and a bias toward starting simple, measuring, and evolving.

Ontology over-ambition. An attempt at a boil-the-ocean enterprise ontology stalls. Mitigate by scoping the ontology to the active domain and growing it one use case at a time.

Recommendations

Stage 1, Prove the base before touching models (weeks 0–8). Do not start with a retrieval proof-of-concept. Start by building the atlas over your two or three highest-value systems, almost always an ERP plus a CRM or a warehouse. Deliverable: the entity-centric authority index for customer, order and invoice, plus a personal-data and quality map. Threshold to proceed: an authoritative source identified for at least 90% of the target entities' key attributes, with quality scores measured rather than asserted.

Stage 2, Build the routed access layer, not a retrieval app (weeks 6–16). Stand up the semantic layer for the first domain and the query router with the structured and live-tool paths first, because they serve the majority of real questions. Add graph and cached-context paths next. Deploy semantic retrieval last and narrowest, only for unstructured prose lookup. Threshold: structured-path execution accuracy of at least 80% on a held-out set of real business questions. If you are near bare-schema numbers, you have not grounded properly.

Stage 3, Enforce governance at retrieval time before go-live (in parallel). No AI access path reaches production without the policy checkpoint enforcing source-propagated permissions, label propagation into embeddings, and column-level lineage on every answer. Threshold: a security review passes because retrieval is a governed data-access system rather than a development tool.

Stage 4, Industrialise and amortise (ongoing). Convert onboarding into configuration. Track the marginal cost and time-to-value of each new use case; the whole thesis is validated only when use case N ships in weeks. Benchmark that would change the plan: if onboarding a new domain still takes months after three domains, the foundation is not productised, stop adding use cases and fix the connector and normalisation automation.

What would change these recommendations. If your corpus is genuinely small, stable and purely unstructured prose, skip most of this and use cached long context directly. If your questions are overwhelmingly single-system operational lookups, prioritise the live-tool path over the lakehouse. If you are already committed to a single vendor's platform, adopt its native semantic layer rather than assembling one.

Caveats

Vendor versus peer-reviewed evidence. The strongest structured-path uplift numbers come from a mix of vendor research and peer-reviewed work. The academic benchmark papers are the most citable. The graph-retrieval win-rates are LLM-judged comparisons rather than ground-truth accuracy. Treat single-vendor accuracy claims as directional rather than audited.

The 45–60% versus 85–92% and "80% of retrieval projects fail / 73% at retrieval" figures come from industry context-layer research and are directionally consistent with the peer-reviewed failure literature, but the precise percentages are not from a single controlled academic study. Use them as well-supported estimates, not laboratory constants.

The "most of data-science time goes on data preparation" statistic traces to a 2016 survey putting it near 80%; more recent surveys put it closer to 45%. The qualitative point is robust; the exact fraction is contested, and we would rather say so than pick the flattering number.

This is a reasoned architecture, not an empirically validated one. It is engineered from documented failure modes and published component results, and it carries the open risks enumerated in Part F.

We have a commercial interest in these conclusions, since this is the layer we build. The counter is that the primary sources are cited and independently checkable throughout, and that Part F states the ways our own design fails.

This report is published as research and is not a commercial proposal. If it is useful, the two-week diagnostic produces the first slice of the atlas for your own estate, the authority index for one entity, and the honest quality scores underneath it.