Table of contents
In this article
Technical page · Systems
Data foundations and integration
One reliable record per entity, with lineage
- Published
- 8 November 2026
- Last reviewed
- 8 November 2026
- Re-checked every
- 90 days
- Reviewed by
- Ales Analytics engineering
In one paragraph
Most systems built on company data fail on the data, not the model. A data foundation gives each customer, product, supplier and matter one reliable record, joined across the ERP, CRM, practice and accounting systems, kept current automatically, and traceable back to its sources. Reports, dashboards, search and agents then all read from the same governed data.
How it works
- Extraction. Change data capture reads the source database's transaction log (Debezium is a common option). Where that is not possible, API pulls or scheduled batch extracts are used.
- Orchestration and contracts. Pipelines run under an orchestrator. Each dataset has a contract for schema, freshness and ownership; a breach stops the pipeline rather than loading bad data.
- Canonical models. One model per entity, with source keys kept so every record can be traced.
- Entity resolution. Exact keys first (VAT number, GTIN, NPI), then probabilistic matching on names and addresses, with uncertain matches reviewed by a person.
- Lineage. Every job emits lineage events (OpenLineage is an open standard), so a figure can be traced through every transformation.
- Write-back. Changes go back to systems of record through staging tables, idempotency keys and reconciliation.
Design choices and trade-offs
| Choice | Trade-off |
|---|---|
| CDC vs batch | CDC is near real time with low source load but needs log access; batch is simpler but stale between runs |
| Central warehouse vs operational store | A warehouse suits reporting; an operational store suits applications that read and write |
| Automatic vs reviewed matching | Automatic is fast; review is slower but avoids merging two different customers |
When not to use it
When one system already holds the data cleanly and nothing else needs it joined, read from that system directly.
How we evaluate it
Data quality checks on every load (completeness, validity, uniqueness, freshness), entity-resolution precision and recall on a labelled sample of matches, and reconciliation of totals against the source systems.
Sources
Should you have any questions regarding this page, please contact us at solutions@alesanalytics.com.