Order the stages of an invoice extraction pipeline from upload to approved record

Order steps·Easy·4.0 · 0·~1 min·Asked atIBM Siemens Turing

Attempt it

1Write approved records to the accounting system
2Send page images to the VLM with a JSON schema for vendor, line items, and totals
3Route low confidence extractions to a human review queue
4Run validation rules: totals reconcile, dates parse, currency is consistent
5Render each page to an image at a resolution where line items stay legible
6Ingest the uploaded PDF and split it into single pages

An invoice extraction pipeline is a small assembly line. Each stage has one job, and the order matters because every stage depends on the previous one. First, ingestion. The uploaded PDF arrives, and you split it into single pages so the rest of the pipeline can process one page at a time. Multi-page invoices are common, and treating the document as a list of pages keeps the downstream code simple. Second, rasterization. PDFs are vector containers, but the VLM consumes pixels, so each page is rendered to an image at a resolution where line items and small print remain legible. Third, model extraction. The page images go to a multimodal model along with a JSON schema describing the fields you want, vendor, line items, totals, dates, currency. Structured output prevents free-form drift. Fourth, validation. Cheap deterministic rules catch the most common errors. Totals should reconcile with line items. Dates should parse. Currency should be consistent. Fifth, human review. Anything that fails validation or falls below a confidence threshold goes to a queue where a human approves or corrects it. Sixth, write. Only after validation and any required review does the record land in the accounting system.

The six-stage pipeline is ordered by data dependency. Each stage consumes the output of the previous one and could not run earlier. 1. **Ingest and split.** Accept the upload and break multi-page PDFs into single pages so downstream stages have a uniform unit of work. 2. **Rasterize.** Convert each page to an image at a resolution that keeps line items and footers legible. Too low and the model misreads; too high and per-page cost balloons. 3. **Extract with structured output.** Send the page images to the VLM with a JSON schema covering vendor, line items, totals, dates, currency, and tax. The schema constrains the response shape and gives downstream stages something parseable. 4. **Validate deterministically.** Run cheap rules before any human looks at the output: line items sum to subtotal, subtotal plus tax equals total, dates parse to real dates, currency is consistent across the document. These rules catch a large fraction of model errors with zero LLM cost. 5. **Route to human review.** Anything that fails validation, scores below a confidence threshold, or exceeds a value threshold goes to a review queue. Most documents skip this stage; the queue is sized for the exceptions. 6. **Write to the accounting system.** Only validated, approved records land in the system of record. This is intentionally the last step because every prior stage exists to protect the books.

TL;DR

Ingest, rasterize, extract, validate, escalate, write. Each stage assumes the previous one ran, and the accounting system is the last possible destination.

Memory aid

Easy to grasp

Imagine you run a tiny accounting office. The mail carrier drops a stack of envelopes. First you open each envelope and lay the pages flat. Then you put them on the scanner because you want clear pictures of each page. Then you ask a smart assistant to read every page and write down vendor, items, and totals on a structured form. Before filing anything, you double-check the math, the dates, and the currency. Anything that looks fishy goes to your senior bookkeeper. Only the cleanly checked, approved forms get carried into the official filing cabinet at the end. The same six steps run every day in production invoice systems, just at machine speed.

Key concepts

Multimodal AI

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Invoice extraction is the canonical multimodal pipeline because it touches every stage you need in a real document-AI system: ingesting raw uploads, rendering them for a vision model, getting structured output, validating that output, escalating exceptions to humans, and writing approved records to a system of record. Teams that get the stage ordering wrong end up either booking bad data, burying a review team in noise, or rerunning expensive model calls on the same pages.

This walkthrough explains why the six stages sit in the order they do, what each one protects, and how senior teams reason about stage boundaries, validation depth, and review capacity. The order itself is straightforward once you see it as a chain of data dependencies; the interesting work is at the boundaries between stages.

Why ingest and split come first

An invoice can be one page or fifty pages. Treating the document as a flat blob forces every downstream stage to do its own page handling, which produces inconsistent behaviour. The pipeline becomes simpler if the very first thing you do is split the upload into single pages and assign each one a stable id.

The page id becomes the unit of caching, retry, and metric reporting for the rest of the pipeline. If the model misreads page 7 of a 12-page invoice, you can rerun page 7 alone instead of the whole document. If a vendor changes its layout, you can compare extraction rates on page 1 across vendors to detect the regression.

Ingestion also covers prosaic but important hygiene: validating that the upload is actually a PDF, rejecting password-protected documents with a clear error, normalizing orientation so the rasterize stage does not see a rotated page, and recording the original filename and upload metadata for audit. Skipping any of these creates failure modes that surface much later as mysterious model errors.

Why rasterize sits between split and extract

Why validation comes before human review

Why the accounting system write is last and idempotent

Production realities and 2026 model lineup

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

AWS Textract paired with Claude Sonnet 4.6 for layout-aware extraction is a common 2026 stack for invoice pipelines, with Textract handling forms and tables and Claude handling unusual layouts
Google Document AI feeds Gemini 3.1 Pro extractions into a structured schema then routes low-confidence pages to humans through a managed review console

What an interviewer would ask next. Try answering before peeking at the approach.

QWhere would you add caching, and how do you keep it from leaking incorrect extractions?

Cache by content hash of the page image, key the cache value to the schema version, and invalidate on any schema or prompt change. Cache hits still go through validation so a stale extraction cannot reach the accounting system unchecked.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Writing extracted fields straight to the accounting system before validation or human review, then spending months reconciling bad bookings caused by misread totals.

60 second bullets to scan on the way to the call.

Why ingest and split must come before rasterization
How resolution choice in the rasterize stage trades cost against legibility

Primary sources. Browse if you want the original framing.