Start here
Node.js40 MB an hour: the Node memory leak hiding behind setMaxListeners(0)
A postmortem of a slow leak in a Node ingest service: how two heap snapshots, one retainer chain and a silenced warning led us to the bug.
Read the storyA group engineering blog
No listicles. Just the bug and the fix.
We write up the things that cost us a weekend so they don't cost you one. Real queries, real stack traces, and the part where our first guess was wrong.
Latest posts
Performance
How to read a flame graph without guessing
Width is time, height is call depth, colour means nothing. A practical guide to reading CPU flame graphs from Go's pprof, using a real regression we fixed.
Postgres
Renaming a column without downtime: expand, migrate, contract
ALTER TABLE ... RENAME COLUMN takes a millisecond and breaks every running pod. A step-by-step expand and contract rename in Postgres, with the trigger we use.
Testing
TIL: our trial countdown was wrong one day a year, and only in Europe
A date test that passed for months failed on a CI runner in Berlin. What we learned about DST, local midnight and testing code that reads the clock.
Tooling
Feature flags without a vendor: one table, one function, one rule
For most teams, feature flags are a Postgres table, a hashing function and a policy about expiry dates. Here is ours, and when we would pay for a product instead.
Postgres
The index that did nothing: reading an EXPLAIN plan line by line
We added an index to fix a 1.2 second query and the plan did not change. Here is how EXPLAIN ANALYZE told us why, and the partial index that worked.
Go
A retry helper with full jitter, in about 40 lines of Go
Exponential backoff without jitter turns one outage into synchronized waves of retries. Here is a small Go helper that gets backoff, jitter and cancellation right.
Browse by topic
- Postgres4Indexes, query plans, migrations and the locks behind them, written from the point of view of the team that owns the database.Browse
- Node.js3The Node runtime in production: memory, the event loop, testing and the tooling around TypeScript services.Browse
- Reliability3Retries, idempotency, safe deploys and the habits that keep a service up when its dependencies are not.Browse
- Performance3Finding where the time and memory go, with measurements first and opinions second.Browse
- Go2Go services and workers: profiling with pprof, concurrency, retries and small helpers worth copying.Browse
- Testing2Tests that catch real bugs: time, randomness, failure paths and the cases that only break in production.Browse