# Data foundations and integration

> Change data capture, data contracts, entity resolution, lineage and safe write-back across ERP, CRM, practice and accounting systems.

- Web page: https://www.alesanalytics.com/docs/data-foundations-and-integration
- Markdown: https://www.alesanalytics.com/docs/data-foundations-and-integration.md
- Last updated: 2026-11-08
- Publisher: Ales Analytics

Technical page. One reliable record per entity, with lineage. Published 2026-11-08. Last reviewed 2026-11-08. Reviewed by Ales Analytics engineering.

## In one paragraph

Most systems built on company data fail on the data, not the model. A data foundation gives each customer, product, supplier and matter one reliable record, joined across the ERP, CRM, practice and accounting systems, kept current automatically, and traceable back to its sources. Reports, dashboards, search and agents then all read from the same governed data.

## How it works

- **Extraction.** Change data capture reads the source database's transaction log (Debezium is a common option). Where that is not possible, API pulls or scheduled batch extracts are used.
- **Orchestration and contracts.** Pipelines run under an orchestrator. Each dataset has a contract for schema, freshness and ownership; a breach stops the pipeline rather than loading bad data.
- **Canonical models.** One model per entity, with source keys kept so every record can be traced.
- **Entity resolution.** Exact keys first (VAT number, GTIN, NPI), then probabilistic matching on names and addresses, with uncertain matches reviewed by a person.
- **Lineage.** Every job emits lineage events (OpenLineage is an open standard), so a figure can be traced through every transformation.
- **Write-back.** Changes go back to systems of record through staging tables, idempotency keys and reconciliation.

![Figure 1. Where data foundations sit in the reference architecture.](https://www.alesanalytics.com/diagrams/sa-reference.svg)

## Design choices and trade-offs

| Choice                                 | Trade-off                                                                                                |
| -------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| CDC vs batch                           | CDC is near real time with low source load but needs log access; batch is simpler but stale between runs |
| Central warehouse vs operational store | A warehouse suits reporting; an operational store suits applications that read and write                 |
| Automatic vs reviewed matching         | Automatic is fast; review is slower but avoids merging two different customers                           |

## When not to use it

When one system already holds the data cleanly and nothing else needs it joined, read from that system directly.

## How we evaluate it

Data quality checks on every load (completeness, validity, uniqueness, freshness), entity-resolution precision and recall on a labelled sample of matches, and reconciliation of totals against the source systems.

## Sources

- [Debezium connectors](https://debezium.io/documentation/reference/stable/connectors/index.html)
- [OpenLineage](https://openlineage.io/)
- [HL7 FHIR R4](https://hl7.org/fhir/R4/)
- Explainer: [System of record](https://www.alesanalytics.com/explainers/system-of-record)

## Contact

solutions@alesanalytics.com, or https://www.alesanalytics.com/contact.

- Every page as Markdown: https://www.alesanalytics.com/llms.txt