On this page 4 sections
Rosa Achterberg leads design at Tallyhouse, a bookkeeping app for small businesses. (Both are fictional composites, built from several conversations with practitioners; the details and numbers are representative rather than reported.) Her team ships an assistant that categorizes transactions and drafts invoice reminders. We talked about mistakes, and why some of them cost trust while others do not.
The interview has been condensed and edited.
On accuracy and trust
You told me accuracy was the wrong goal. That is a strong claim for a bookkeeping product.
It is the wrong only goal. Our categorizer was right about 93% of the time when we launched, and people were furious about the other 7%. A year later it was right 95% of the time and complaints had fallen by half. Two points of accuracy did not do that. The way errors showed up did.
What changed?
Three things. We showed why each category was chosen, in one line: “Matched 14 earlier payments to this supplier.” We made correcting a category a single tap, with the correction applied to similar transactions if the user agreed. And we stopped auto-applying categories below a confidence threshold. Those transactions show as suggestions with a dotted outline instead.
On the metric they track
How do you measure whether people forgive the feature?
We track edit rate: the share of AI outputs a person changes before accepting. For categories it is a tap. For invoice reminders it is the share of the draft rewritten, measured by character difference. We log a small event for every accepted output.
{ "feature": "reminder_draft", "accepted": true, "edit_ratio": 0.18, "time_to_accept_s": 21 }An edit ratio under about 20% means the draft helped. Above 60%, the person would have been faster writing from scratch, and we treat that as a failure even if they never clicked thumbs down.
Why not just use thumbs up and down?
Fewer than 3% of users ever touch them, and the ones who do are angry. Edits come from everyone, and they tell you what the right answer was.
On interface choices
What design decisions made the biggest difference?
The dotted outline for low-confidence suggestions, honestly. Users accept that a suggestion can be wrong. They do not accept that a decision was wrong. Same model, same output, completely different reaction.
The second was tone. Our early reminder drafts were cheerful in a way that sounded like a bank. Small business owners write to clients they know personally. We let them pick from three tones and remember the choice per client. Edit ratio on reminders fell from 41% to 22% after that.
Anything you tried that did not work?
A percentage confidence score next to each category. People read “78%” as “this is broken one time in four” and second-guessed suggestions that were fine. We replaced it with the one-line reason and the outline, and users stopped asking what the numbers meant.
On advice for other teams
If a team is about to ship its first AI feature, what should it do first?
Decide what a mistake looks like before launch. Write down the three most likely wrong outputs, then design the screen where each one appears. If fixing it takes more than one action, change the design. If you cannot tell the user why the system did it, add a reason. Then instrument edits from day one, because that data is the only honest review your feature will get.