# Document intelligence pipeline

> How a document pipeline classifies, reads, extracts to a schema, validates and routes doubtful fields to review, with each value linked to its page.

- Web page: https://www.alesanalytics.com/docs/document-intelligence-pipeline
- Markdown: https://www.alesanalytics.com/docs/document-intelligence-pipeline.md
- Last updated: 2026-11-08
- Publisher: Ales Analytics

Technical page. From scanned page to checked record. Published 2026-11-08. Last reviewed 2026-11-08. Reviewed by Ales Analytics engineering.

## In one paragraph

A document intelligence pipeline turns invoices, forms, delivery notes and scans into structured records that another system can use. It classifies each document, reads it with the right tool for its type, forces the output into a fixed schema, checks the result against business rules and master data, and sends anything doubtful to a person with the source region highlighted. Every value keeps a link to the page and position it came from.

## How it works

1. **Classify.** Decide what the document is (invoice, credit note, bill of lading, referral form) and whether it has a usable text layer.
2. **Read.** Born-digital files go to a layout parser that recovers text, reading order and tables. Scans, photographs and complex tables go to a document vision-language model that reads the page image directly.
3. **Extract to a schema.** The model is constrained to produce JSON matching a schema as it generates, so the output is always well-formed.
4. **Validate.** Rules encode what the business knows: totals add up, dates are plausible, the purchase order exists, the supplier is on the vendor master.
5. **Score confidence per field.** Several signals are combined, and the weakest one usually decides.
6. **Review.** Fields under their threshold go to a queue where a person sees the value next to the highlighted page region. Corrections are saved as labelled examples.
7. **Write.** Accepted records are written to the target system through a staging step, with an idempotency key so a retry cannot duplicate them.

![Figure 1. The pipeline from incoming document to checked record.](https://www.alesanalytics.com/diagrams/sa-document-pipeline.svg)

## Design choices and trade-offs

| Choice                                        | Trade-off                                                                                                                              |
| --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| Layout parser vs vision-language model        | Parsers are fast and cheap on clean digital files; vision models handle scans and tables but cost more GPU time per page               |
| One general model vs a fine-tuned small model | A general model needs no training data; a small model fine-tuned on corrections is cheaper to run at volume once enough examples exist |
| Thresholds per field vs one global threshold  | Per-field thresholds take more setup but put review effort where errors are costly                                                     |
| Licences                                      | Some document-model weights restrict commercial use; check before selection                                                            |

## When not to use it

When documents already arrive as structured data (EDI, an API, a supplier portal export), parse that instead. When volumes are a few documents a week, the setup and review effort may not pay back.

## How we evaluate it

A frozen test set of the client's own documents, labelled field by field. Report field-level precision, recall and F1 by document type, the share of fields sent to review, and the error rate on fields accepted without review. The last number matters most, because it is the error that reaches the system of record.

## Sources

- [Docling](https://github.com/docling-project/docling), [olmOCR](https://github.com/allenai/olmocr), [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR), [Marker](https://github.com/datalab-to/marker)
- [vLLM structured outputs](https://docs.vllm.ai/en/latest/features/structured_outputs.html)
- See also [Systems architecture](https://www.alesanalytics.com/engineering/systems-architecture), which has a worked example of this pipeline.

## Contact

solutions@alesanalytics.com, or https://www.alesanalytics.com/contact.

- Every page as Markdown: https://www.alesanalytics.com/llms.txt