Scale invoice extraction to a million documents a month without burning the budget

Short answer·Hard·4.0 · 0·~3 min·Asked atFlipkart Mistral AI Vellum

Attempt it

Your invoice extraction pilot works on a frontier VLM, but finance says the unit economics fail at the contracted volume of one million documents a month. Design the production system: cost structure, accuracy assurance, human review, and how you keep quality from drifting.

Free · 2 AI evals / day

At a million documents a month, the system has to do two things at once: keep accuracy high enough that finance trusts the books, and keep the cost low enough that the contract is profitable. The single biggest lever is routing. Most invoices come from a repeat set of vendors with predictable layouts. A small, cheap multimodal model handles the bulk; only documents that fail validation rules or come back with low confidence are escalated to the expensive frontier model. The second lever is structural cost reduction on every call: render pages at the resolution where line items are legible, not the maximum supported; use batch APIs where end-to-end latency does not matter; cache extractions by content hash so reruns are free; and lean on the fact that the same vendor templates repeat month after month. Accuracy comes from layered validation, not from picking the smartest model. A strict JSON schema constrains the output shape. Deterministic rules like totals reconciling and dates parsing catch most arithmetic and formatting errors before anything else looks at the document.

The system has four pillars: cost structure, accuracy assurance, human review, and drift control. **Cost structure** starts with model routing. A cheap default model, often a smaller open VLM or a provider's mini tier, handles the high-volume clean majority. Documents that fail deterministic validation, score below a confidence threshold, or come from never-before-seen vendor templates escalate to a frontier model like Claude Sonnet 4.6 or Gemini 3.1 Pro. The frontier model sees the tail, not the traffic. On top of routing, three orthogonal levers stack: tune rasterization resolution to the legibility floor rather than the model maximum; use batch APIs for non-urgent volume since they price at 30 to 50 percent of synchronous; and cache extractions by page-image content hash, with template-aware prompts that exploit the long tail of repeat vendors who send near-identical layouts every cycle. **Accuracy assurance** is layered. Structured output against a strict JSON schema constrains the response shape. Deterministic business rules catch errors that no model self-confidence score can match: line items sum to subtotal, subtotal plus tax equals total, dates parse, currency is consistent, tax IDs match country patterns. Rule failures are the primary escalation signal. **Human review** is a budgeted queue sized from finance's error tolerance. Confidence thresholds are calibrated so the daily queue is humanly processable; dollar value adds an override so a misread ten-thousand-dollar invoice always gets eyes regardless of confidence. Reviewer corrections are captured as labeled data for the next fine-tune round.

TL;DR

Route cheap model first, escalate on validation failure or low confidence, layer deterministic rules, size the review queue from error tolerance, and gate releases on a golden set plus production audits.

Memory aid

Easy to grasp

Imagine a busy mailroom sorting a million letters a month. Most letters are simple: the address is clear, the stamp is right, an intern handles them. A few letters are weird: handwriting smudged, foreign postage, unusual addressee. Those go to the senior sorter who is slow and expensive. The intern handles the river of normal mail, the senior sorter handles the exceptions, and a supervisor spot-checks both. The mailroom also keeps a binder of tricky old letters and tests every new sorter against it before letting them work alone. Run the same pattern with two model tiers, deterministic checks, a budgeted human queue, and a golden test set, and a million invoices a month becomes affordable.

Key concepts

Multimodal AI

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

A pilot that works on a frontier multimodal model at a thousand documents a month is a different engineering object than a production system at a million documents a month. The unit cost stays the same; the volume goes up three orders of magnitude. Finance does the multiplication and the system is unworkable.

This answer lays out the production design across the four dimensions that determine whether the system survives the contract: cost structure, accuracy assurance, human review, and drift control. The central insight is that the frontier model should see the tail of the document distribution, not the traffic. Everything else falls out of that decision.

Cost structure: route, then squeeze

The dominant cost lever at volume is model routing. A two-tier setup works for most invoice domains. The cheap tier is a smaller multimodal model, either a provider mini tier or an open VLM like LLaVA-NeXT or Qwen2-VL hosted on your own infrastructure. The frontier tier is Claude Sonnet 4.6, GPT-5.5, or Gemini 3.1 Pro reserved for documents that the cheap tier could not handle.

The escalation rule is deterministic. If the cheap model's structured output passes the JSON schema, if the business rules reconcile, and if the model's self-reported confidence on the riskiest fields is above threshold, ship it. Otherwise escalate. In a healthy production system, escalation runs at 5 to 15 percent of volume, and that is where the frontier spend lives.

On top of routing, three independent levers stack. Resolution: render pages at the legibility floor for the typeface your vendors use, not the model maximum. This often saves 30 to 50 percent of vision tokens on the cheap tier. Batching: most invoice pipelines do not need sub-second latency, and the batch APIs from major providers price at roughly half of synchronous, with response windows of a few hours. Caching: every page image gets a content hash, and identical pages reuse the previous extraction. Template-aware prompting amplifies cache hits because the same vendor often sends the same layout every cycle.

\text{cost}_\text{month} \approx N \cdot \left[(1-r) \cdot c_\text{cheap} + r \cdot c_\text{frontier}\right]

Where r is the escalation rate and the two costs include all per-call optimizations. Dropping r from 100 percent to 10 percent is typically a 5x to 10x reduction in total spend.

Accuracy: structured output plus deterministic rules

Human review: a budgeted, value-aware queue

Drift control: golden set, audits, and dashboards

Production realities and 2026 model lineup

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Ramp and Brex run AP automation at this scale by routing most invoices through smaller in-house extractors and reserving frontier models like Claude Sonnet 4.6 for unusual layouts
Stripe Tax handles invoice and receipt extraction across hundreds of jurisdictions using a similar tiered approach, with deterministic rules around currency and VAT formats as the primary escalation signal

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you decide when to fine-tune the cheap model versus when to keep relying on the frontier escalation path?

Look at the escalation rate by vendor over a few months. When a specific vendor's documents consistently escalate to the frontier model, that vendor's accumulated reviewer corrections are good fine-tuning data. Fine-tune when the projected reduction in frontier calls pays back the fine-tuning cost within a quarter.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Sending every document to a frontier model because the pilot did, then watching unit cost stay constant while volume grows linearly, instead of routing the cheap model first and reserving the expensive one for the tail.

60 second bullets to scan on the way to the call.

How to identify which document portfolio segments are clean enough for the cheap model
Which deterministic rules give the highest signal-to-noise for invoice extraction

Primary sources. Browse if you want the original framing.