An agent that extracts data from documents

Last reviewed 27 July 2026

A document agent takes an unstructured file, works out what kind of document it is, pulls the fields a schema demands, and checks the result against arithmetic the document itself asserts. Every field carries a confidence score, and anything under the threshold the owner set goes to a person rather than into the ledger.

What starts it

A file arrives in the shared inbox, the scanning folder, or the upload endpoint that a downstream system needs as structured fields rather than as an attachment nobody opens.

What it does

  1. 01

    Classify the file before reading it. An invoice, a credit note, a statement and a delivery note need different schemas and different validity checks.

  2. 02

    Split multi-document scans at the page where one document number ends and the next begins, before any field is extracted.

  3. 03

    Extract header fields and line items as two separate passes, because line-item tables run across page breaks and header blocks do not.

  4. 04

    Recompute the arithmetic the document asserts about itself, meaning line items plus tax against the stated total, and quantity times unit price against each line.

  5. 05

    Match the record against the source of truth, which is the purchase order and the goods receipt for a payable, and the load or job record for a delivery note.

  6. 06

    Route by confidence into post, review, or reject, and keep the page image attached to whatever gets posted so a reviewer can see the original.

What it connects to

  • An OCR layer that returns geometry as well as text, so a value can be tied to its position on the page (Amazon Textract, Google Document AI, or Azure AI Document Intelligence)
  • A vision-capable model for what OCR alone cannot resolve, such as which of the four numbers in the bottom right is the amount actually due
  • The system of record the output posts into, with duplicate detection on document number so the same invoice cannot enter twice

How it goes wrong

  • Inventing a plausible value for a field the document does not contain. A missing purchase order number comes back as a correctly formatted purchase order number, and the invoice posts against the wrong job.
  • Transcribing OCR noise with high confidence. The model reports certainty because the text it was handed was unambiguous. The text was simply wrong, and the confidence score cannot see that far back.
  • Dropping the continuation rows of a line-item table that breaks across a page. The extracted lines are short, and the arithmetic check does not fire because the stated total was read off the same last page.
  • Reading a credit note as an invoice, which flips the sign and turns a refund owed to you into a payable owed by you.
  • Misreading decimal convention on a European document, where 1.234,56 becomes 1.23 and a four figure invoice is posted as small change.

When it hands back

Any document whose arithmetic does not reconcile, any vendor not already on the master file, any bank detail that differs from the one on record, any amount above the owner's threshold, and every credit note regardless of confidence.

Extraction is four jobs, not one

Treating this as “read the PDF” is why most implementations disappoint. There are four distinct stages, and they fail differently. Classification decides what the document is. Extraction pulls fields into a schema. Validation checks the extracted record against arithmetic and against a source of truth. Routing decides whether the record is trustworthy enough to post unattended.

Skipping validation is the common shortcut, and it is the expensive one, because extraction errors do not announce themselves. A wrong total looks exactly like a right total until something downstream reconciles.

What a page costs

Two costs stack, and both are published.

Amazon lists Textract’s Analyze Expense API at $0.01 per page for the first million pages a month in US West (Oregon), and its Analyze Document forms feature at $0.05 per page over the same tier. Plain text detection is $0.0015 per page. So a thousand single-page invoices costs about ten dollars through Analyze Expense, or about fifty dollars if the work needs full forms analysis.

On top of that, a model pass that reads the extracted text plus a schema and returns a structured record might use 3,000 input tokens and 500 output tokens. Priced against Anthropic’s published Claude Haiku 4.5 figures, one dollar in and five dollars out per million tokens, the input side is $0.003 and the output side $0.0025, so $0.0055 a document. Combined, a single-page invoice lands somewhere near one and a half cents through OCR plus a model pass. Substitute your own page counts and rates and the arithmetic still holds.

Neither figure includes the exception queue, and the exception queue is where the real money goes. A pipeline that posts most documents unattended and sends the rest to a person still needs the person.

Confidence bands, and what each one means

Confidence is not a single number applied uniformly. Field criticality differs: the invoice total, the tax, the invoice number and the vendor identity carry consequences that a description line does not, so they deserve tighter thresholds. A practical arrangement is three bands. High confidence on all critical fields, with the arithmetic reconciling, posts automatically. Middle confidence goes to a clerk with the extracted values pre-filled and the page image beside them. Low confidence, or a failed reconciliation, goes to a full manual queue where the record is keyed from scratch.

The threshold is a business decision, not a technical one. It is the exchange rate between review labour and the cost of one bad posting, and the owner sets it.

Why a model’s own confidence is not enough

A language model reading clean OCR output has no way to know the OCR was wrong. If the scanner produced a plausible but incorrect string, the model transcribes it fluently and reports high certainty, because from where it sits the input was unambiguous. Model confidence measures agreement with the input, not agreement with the paper.

That is why the validation stage cannot be a second model asked whether the first was right. It has to be arithmetic and lookups: does the sum of lines plus tax equal the stated total, does the vendor exist on the master file, has this document number been seen before, does the amount fall within the tolerance the purchase order allows. Those checks catch errors a confidence score never will, and they are cheap because they are just code.

The documents that break every pipeline

Faxed and rescanned pages, where a document has been through two generations of compression. Rotated pages from a phone camera. Handwritten annotations that change the meaning of a printed line, such as a crossed-out quantity with a new one written above it. Statements listing dozens of invoices where the agent must decide whether it is processing one document or forty. Tables that continue across a page break with no repeated header. Each of these is worth testing explicitly during setup rather than discovering in production, because the failure mode is silent output rather than an error.

Where this job differs

Apply