On this page 5 sections
A reader asked a practical question: could her team extract fields from supplier invoices on employee laptops instead of sending documents to a hosted API? Legal preferred the documents never leave the machine. I spent a week finding out.
The models are fictional stand-ins for the size classes you can download today: a 3B model from a lab we’ll call Kestrel, a 7B from one we’ll call Northwind, and a 14B we’ll call Alder. All ran 4-bit quantized through a local runtime.
The bench
Two machines, both the kind a company actually issues:
- Laptop A: 32 GB unified memory, recent mid-range chip.
- Laptop B: 16 GB memory, two years old.
The task: read the text of 200 real (anonymized) invoices and return JSON with supplier name, invoice number, date, currency, total and VAT. A field counted as correct only on an exact match after normalizing whitespace and date format.
{ "supplier": "Hakuba Industrial Supply", "invoice_no": "HIS-20931", "date": "2026-03-14",
"currency": "JPY", "total": 482300, "vat": 43845 }Results
| Model | File size | Laptop A tok/s | Laptop B tok/s | Field accuracy |
|---|---|---|---|---|
| Kestrel 3B | 1.9 GB | 58 | 34 | 88.4% |
| Northwind 7B | 4.1 GB | 31 | 17 | 94.1% |
| Alder 14B | 8.3 GB | 16 | 3 (swapping) | 96.0% |
| Hosted baseline | n/a | n/a | n/a | 97.8% |
The 7B model was the clear middle. Six points better than the 3B, under two points behind the 14B, and comfortably fast on both machines. A typical invoice needed about 120 output tokens, so at 31 tokens per second the answer arrived in roughly four seconds including prefill.
The 14B model on Laptop B was not a result so much as a warning. With a browser, a chat app and a spreadsheet open, the model no longer fit in memory and the operating system started swapping. Throughput fell to three tokens per second and the fan never stopped.
Where the errors were
Accuracy averages hide the pattern, so I sorted the failures by field:
- Dates caused most 3B errors. European day-first dates were read as month-first about one time in ten.
- VAT tripped every model when an invoice listed two rates. The 14B model handled it best, but still missed 5 of 17.
- Supplier names were nearly perfect across the board. Copying text is the easy part.
Two cheap fixes closed much of the gap. Adding three worked examples of day-first dates to the prompt lifted the 3B model from 88.4% to 91.7%. Validating that subtotal plus VAT equals the total, and retrying on mismatch, caught 60% of the remaining VAT errors for every model.
Battery and heat
Running the 7B model continuously on Laptop A for one hour, processing invoices back to back, drained 18% of the battery. In real use, with extraction triggered a few dozen times a day, the effect was not noticeable. Laptop B ran warm enough under sustained load that I would not ship continuous processing to it.
What I told the reader
Ship the 7B class model, with the date examples in the prompt and the arithmetic check in code. Set a minimum of 16 GB of memory and keep context under 8,000 tokens so the total footprint stays near 5 GB. Route the roughly 6% of invoices that fail validation twice to a human queue rather than to a bigger model. At her team’s volume of about 250 invoices a day, that queue is roughly 15, and nothing leaves the laptop.