Retrieval and RAG
Postmortem: why our RAG answers drifted when nobody deployed anything
Over six weeks an internal help assistant got quietly worse with no model or prompt change. The causes were all in the retrieval layer, and all preventable.
Evals and reliability editor
Mara Ekström
Mara spent six years building evaluation pipelines for search and support products. She writes about evals, regressions and the postmortems nobody wants to publish. Her favorite metric is the one that disagrees with the dashboard.
Stories by Mara
Evals
What an eval suite actually catches (and the three things it never will)
A 240-case eval suite for a ticket summarizer, a cheaper model swap, and an honest accounting of which failures the suite caught and which reached users.