Q&A: designing AI features that users forgive when they are wrong
A design lead on why her team stopped chasing accuracy alone, the edit-rate metric they track instead, and the small interface changes that halved complaints.
9 stories
Latest from the desk
Token Budget is an independent, reader-funded publication about applied AI for the engineers, PMs and designers who ship and maintain AI features.
Latency budgets for chat interfaces: spend 1.5 seconds on purpose
Users judge a chat feature in the first second and a half. How to split that budget across network, guardrails, retrieval and the model, and what to show while they wait.
Prompt caching math: when it saves 75% and when it costs you more
Cached prefixes are cheap to read and slightly expensive to write. We work out the break-even hit rate, the traps that keep it at zero, and a prompt layout that helps.
Agents need an undo button before they need more autonomy
An agent that is right 97% of the time still makes dozens of wrong moves a day. Reversibility, not accuracy, makes that tolerable. An argument, with a pattern.
Postmortem: why our RAG answers drifted when nobody deployed anything
Over six weeks an internal help assistant got quietly worse with no model or prompt change. The causes were all in the retrieval layer, and all preventable.
How to read a model card skeptically, line by line
Model cards are marketing and documentation at once. A practical checklist for separating the claims you can rely on from the ones you need to test yourself.