Independent · Applied AI · Since 2024
Token math, eval harnesses, cache hit rates and KV-cache memory, worked through with real numbers and the code we used. Written for people who read the p95 before the demo.
Editors' picks
AgentsAn agent that is right 97% of the time still makes dozens of wrong moves a day. Reversibility, not accuracy, makes that tolerable. An argument, with a pattern.
Read the storyA design lead on why her team stopped chasing accuracy alone, the edit-rate metric they track instead, and the small interface changes that halved complaints.
Users judge a chat feature in the first second and a half. How to split that budget across network, guardrails, retrieval and the model, and what to show while they wait.
Cached prefixes are cheap to read and slightly expensive to write. We work out the break-even hit rate, the traps that keep it at zero, and a prompt layout that helps.
Over six weeks an internal help assistant got quietly worse with no model or prompt change. The causes were all in the retrieval layer, and all preventable.
Model cards are marketing and documentation at once. A practical checklist for separating the claims you can rely on from the ones you need to test yourself.
We ran 3B, 7B and 14B open-weight models on two ordinary laptops to extract invoice fields. Speed, accuracy, memory and battery, measured rather than guessed.
One essay every other Thursday on evals, retrieval, agents, latency or cost, with the arithmetic shown and the code attached.
Free · No sponsored posts · 18,400 readers