ProjectsAugust 1, 2026

Reading 17 Auto Glass Catalogs with a Vision Model

The problem

A glass distributor had 17 NAGS catalogs (National Auto Glass Specifications), about 500 MB of PDF, and wanted one Excel file out of them. Seven columns per part:

Marca	Modelo	Ano inicio	Ano fin	Tipo de cristal	Numero NAGS	Caracteristicas
Acura	ILX 4 Door Sedan	2021	2022	FW	FW04187	Solar, Electrochromic Mirror, Acoustic

Their hard requirement was not accuracy. It was reusability. NAGS ships a new edition twice a year, and a script that only works on the 2024 Spring edition is worthless in six months. Each page is two columns. Vehicle header printed once, then implicit across every row under it. Part numbers with superscript footnote markers glued to them. Dot leaders. Line drawings taking up a third of the page.

Why the obvious approach fails

The PDFs have an embedded text layer, so the first instinct is pdfplumber plus regex. I tried it. The text layer is pre-baked OCR from a scan, and it is bad in a way that is worse than being absent:

FW05943   ->  FWOS943      digit 0 read as letter O
FW05681   ->  FWts681      digits read as letters
FD28886-87 -> FD18886-87   2 read as 1, still matches the regex, silently wrong
35.0x56.0 ->  35.0156.0    the x became a 1, dimension destroyed
FW04187   ->  FW0418'79d   superscript footnotes absorbed into the part number

That third line is the killer. FD18886-87 passes any part number regex you write. You cannot detect the error from the string alone. And the 2023 Winter edition fragments text at the word level while 2024 Spring reconstructs lines, so I already needed two code paths for two catalogs. Extrapolate to 17 editions and you own a maintenance problem forever. So: throw away the text layer, render pages as images, and read them with a vision model. Layout-agnostic by construction.

Methodology

I did not build the pipeline first. I ran eleven isolated pilot experiments, each one answering a single question against real catalog pages with a scored pass or fail, before writing any production code. Each experiment lives in its own folder with a findings file, a timestamped run log holding every image and API response, and a config block to write back. Total pilot spend was under $2.

ID	Question	Verdict
D5	What DPI and tiling?	250 DPI, gutter-split
D6	How to find the column gutter?	Whitespace-run detector with depth guard
P1	Where do we cut tall columns?	At whitespace valleys, not fixed heights
P2	Can we detect skipped rows?	Count-then-emit with retry
P3	How do rows keep their vehicle header?	Prompt carry-forward
P4	Which pages are data pages?	Text heuristics, 98.7% without an API call
P5	How do we prove completeness?	Reconcile against the catalog's own index
P6	Normalization	Config tables, zero LLM calls
P7	Confidence routing	Row score plus a page-level coverage gate
V2	Does state survive page breaks?	Yes
V3	Row-level crop coordinates	Partial, one approach ruled out

What we found

1. Sending the whole page does not work, and not for the reason you think

Claude downscales images whose long edge exceeds 1568 px. Send a 250 DPI page and it gets squeezed back to roughly 143 effective DPI before the model ever sees it. Every whole-page config in the D5 sweep froze at 34.6% recall no matter what DPI I rendered at. Tiling is not an optimization here, it is the only way to get real resolution to the model. Scored against 32 hand-verified part numbers on page 13:

DPI	Mode	Recall	Hallucinations
150	tiled	65.6%	1
200	tiled	56.2%	1
250	tiled	100% (32/32)	0
300	tiled (blind bands)	12.5%	0
any	whole	34.6%	-

2. The 300 DPI collapse taught me more than the 100%

12.5% at higher resolution made no sense until I looked at the tiles. Blind equal-height banding was slicing straight through text rows:

Look at the top line. Assist) with nothing above it. The row's first half went to the previous tile, the model never saw a complete row, and it dropped it. Six bands per page at 300 DPI means six chances to orphan a row. The fix was to make cuts content-aware. Build an ink projection profile per column, find the horizontal whitespace valleys between rows, and cut in the middle of one. The threshold self-calibrates per column, because vertical table rules run the full page height so a true gap is not zero ink (floor sits around 11 px, text rows around 145).

The green lines are the chosen cuts. The wiggly trace down the left is the ink profile driving the decision. Zoomed in, every seam lands in clean white space:

Result: 300 DPI went from 56% to 96.9%, and 250 DPI stayed at 100%. The single remaining 300 DPI miss was a glyph confusion (FQ read as PQ) on a row that was fully intact, so it belonged to a different stage.

3. Finding the gutter is a signal processing problem

Same idea on the horizontal axis. Scan the central 30 to 70% of the page, build a column ink profile, find runs below an adaptive threshold, and add a depth guard so a uniformly inked single-column page cannot fake a gutter. Return None when nothing qualifies. Page 13's gutter landed at x=1002, page 14 at x=912. All 208 pages of the Spring edition are dual column, so I validated the single-column path against synthetic images instead of pretending I had tested it.

4. The hardest bug is invisible in the output

Split a page into four tiles and each tile is a valid extraction on its own. The problem is that the vehicle header only appears once. On page 13, one model's header sits at the bottom of the top-left tile and all ten of its part rows sit at the top of the bottom-left tile. Extract tiles independently and those ten rows come back with no vehicle attached. 31.2% of page 13. On page 14 it is 47.2%. The worst case is the column transition, because the catalog does not repeat the header across the gutter:

Six rows sitting under a bare (cont.) marker with no header of their own. I tested three strategies. Union of sets (the baseline) got 100% part recall but only 68.8% row recall, which is the precise reason part-level metrics lie. Post-hoc propagation worked on these pages but breaks in cases I could construct on paper. The winner was prompt carry-forward: each tile reports its open_block and the next tile's prompt is prefixed with it.

Strategy	Part recall	Row recall	Header loss
A, union of sets	100%	68.8%	31.2%
B, carry-forward	100%	100%	0%
C, post-hoc merge	100%	100%	0%

Carry-forward also handles the case where it should be ignored: a tile that received a stale carry from the previous model saw a fresh header at the top of its own content and used that instead. Cost of the extra context was about 3% more tokens per tile.

5. Make the model check its own work

The prompt asks the model to state a count first, then emit exactly that many rows. The original harness parsed the list and never compared the two. On a dense tile the model declared 13 and emitted 12, and the missing row was invisible in every metric. Now a mismatch triggers one retry, and if the retry still disagrees the tile is split in half and both halves are sent. Real trigger on page 13 tile 2: declared 10, emitted 9, retry self-corrected to 9 and 9. Overhead is roughly 5 to 10% at the page level.

6. The catalog grades its own extraction

Every NAGS catalog ends with a Numerical Part Listing, a flat index of every part in the book. That index is a built-in answer key. Parse it with a cheap text-only model, call it G, and reconcile the vision output E against it. Missing is G minus E, extra is E minus G. This is what makes completeness provable instead of asserted, and it is what a black box PDF parser cannot give you.

The testing story worth telling

The reconciliation first reported 40.6% recall with every miss being a door or quarter glass part. That reads like a clean diagnosis: the extractor skips certain glass types. It was wrong, and it was my harness that was wrong. Bug one. NAGS packs consecutive parts into range notation. FD29020-21 means FD29020 and FD29021. The extractor captured them faithfully in that compact form, but the cleaning regex rejected any string containing a hyphen, so it returned None. Door and quarter glass are almost always ranges. Windshields are almost always singletons. So the discarded rows correlated perfectly with glass type and manufactured a fake coverage gap. The rows were sitting in the output file the whole time. Bug two. Exact string matching, against an index that is itself OCR of the same bad scan. The index listed one part number, the vision read a one-digit variant of it. Same physical row, one digit of drift, and the index is only about 82% reliable so it is not obvious which side is wrong.

Method	Recall	Hallucinations
Exact match	40.6%	0
Plus range expansion	76.6%	0
Plus edit-distance snap	~98.4%	0

Two lessons went straight into the design. Range expansion is a required normalization stage, not a reconciliation nicety. And normalization must run before reconciliation, otherwise the re-extraction loop spends real money chasing phantom misses.

Then I made the loop find that by itself

Since the pipeline has to run unattended across 17 editions, I built an auto-tuning loop that mines correction rules on one page and only keeps a rule if it improves recall on a held-out page.

Rule	Train	Held out	Decision
R0, baseline	42.5%	37.5%	baseline
R1, expand ranges	72.5%	83.3%	keep, +45.8 pp
R2, plus OCR snap	75.0%	91.7%	keep, +8.3 pp

The loop rediscovered both of my manual findings, automatically, with an overfit guard. It also drew its own boundary: the residual one-versus-two-digit drift cases are invisible to self-supervision, because the index cannot say which side is right. That is exactly the set a human should label once during the build, and the loop identified it without being told.

Other testing worth naming

Page classification resolves 98.7% of pages with text heuristics and no API call. The 2023 Winter index pages have such garbled headers that the score-based detector only caught 5 of 22. A literal string override for the index page's title took it to 22 of 22.
Confidence routing scores each row 0 to 100 from three real signals and buckets it. On the pilot sample: 92 auto-accept, 9 spot-check, 0 reject. The 9 were exactly the known OCR digit confusions.
The finding that changed the design: per-row confidence is structurally blind to a missing row. A row that was never extracted has nothing to score. So there is a separate page-level coverage gate, and it still flagged the pilot page at 76.6% recall while every individual row on it looked fine.
A ruled-out approach. For row-level crop coordinates I tried matching part numbers back to raw PDF word boxes. 78.1% against a 95% bar, and only a small fraction of extracted words matched the pattern at all. Documented as a dead end rather than tuned.

The pipeline as built

[1] Render      PDF page -> 250 DPI PNG                        (D5)
[2] Classify    data / index / TOC / ad / blank, no API        (P4)
[3] Index parse Cheap text model reads the catalog index -> answer key G  (P5)
[4] Extract     gutter-split -> content-aware bands -> vision model,
                carry-forward across tiles and pages           (D6/P1/P3)
[5] Normalize   range expansion, year pivot, make aliases      (P6)
[6] Reconcile   E vs G, range-aware, edit-distance tolerant    (P5/V1)
[7] Route       row confidence + page coverage gate            (P7)
[8] Export      Excel + QA HTML report

Every page is checkpointed into SQLite in WAL mode, so an interrupted run resumes without re-billing a single API call. I found out this mattered when a run died on exhausted credits mid-catalog and picked up exactly where it stopped. Normalization rules live in config tables, not code. Makes, aliases, glass types, features, century pivot for two-digit years. A new edition with a new brand is a config edit.

Where it stands

First consolidated run, 17 catalog pages of the 2024 Spring edition:

Metric	Value
Parts extracted	868
Recall against catalog index	92.6%
Missing	42
Hallucinations	2 of 868
Cost, this run	$0.89
Same run on the batch API	~$0.77

Projected cost is about $2.84 for a full 250-page catalog and roughly $49 for all 17. Two honest caveats. Precision reads 61%, but that metric compares against index entries scoped to the pages processed so far, so parts attributed to an adjacent page count against it. It understates. And a meaningful share of rows carry an OCR-snap flag, meaning the extraction and the index disagree by one or two characters and neither source is authoritative. That is the residual a human labeling pass exists for.

What I would carry to the next one

Score against a golden set scoped to the exact edition and page range. My first pilot run reported 61% failure because the reference file mixed rows from two different catalog editions. Part-level recall can hit 100% while a third of your rows are unusable. Measure the thing you are shipping. When a metric fails in a suspiciously tidy pattern, suspect the measurement before the system. "It misses one whole category" turned out to be two harness bugs wearing a trench coat. And use the document's own index. The catalog shipped with an answer key in the back. Nothing else in this project bought as much confidence for as little money.

Related projects

Automated Daily AI Report, Built for a Non-Technical Owner

A self-serve daily reporting tool for an Amazon seller: Google Sheets in, an AI-written summary out, delivered by email on a schedule she controls herself. Built end to end on Cloudflare Workers for $5/month.

Read case study

Kasb: A Desktop Earnings Tracker for a Fellow Freelancer

A local-first desktop app that turns a messy Upwork CSV export into real profit and loss: gross earnings, fees, connects ROI, and client breakdowns, with zero cloud and zero recurring cost.

Read case study

The Legal Layer: AI Contract Review for a Legal Directory Platform

Backend engineering on a members-only AI tool that explains a contract in plain English before you sign it, live at justisconnection.com/legal-layer.

Read case study

Daily Property Listing Feed from a Major Online Marketplace

An Apify actor that pulls new property listings from a large online marketplace every day, enriches each one with council-level data, and feeds a live Google Sheet with zero duplicate rows.

Read case study

24/7 AI Bot Platform with SaaS Dashboard and Multi-Bot Workflows

Designed and built a full-stack SaaS dashboard and backend in Python to manage a 24/7 AI bot platform. Integrated OpenAI with Google Sheets, WordPress, and custom rewriting tools.

Read case study