Aman Verma

On code samples that drift

· 2 min read · #docs #developer-tooling #go

The code sample in your README is wrong. Not catastrophically — just enough to make the next person reading it run go build, get an error, and wonder if the problem is them.

I’ve been bitten by this enough times to build a tool around it. But the tool isn’t the interesting part. The interesting part is how this happens, and what it says about the shape of your documentation.

How code samples go stale §

Almost always, the same loop:

  1. You write a tutorial or README and paste a snippet from your editor.
  2. The snippet is correct at the moment you paste it.
  3. Six months later, someone refactors the underlying function — renames a parameter, changes a return type, splits it in two.
  4. The refactor PR doesn’t touch the docs, because nothing in CI tells it to.
  5. The next reader finds a snippet that almost works.

Every link in that chain is reasonable. The problem isn’t carelessness — it’s that the snippet has no relationship to the code it pretends to describe. It’s a screenshot in text form.

What doesn’t work §

  • Vigilance. “Remember to update the docs” scales linearly with attention, and attention is the resource your codebase is already fighting for.
  • Doctests. Great for the snippets they cover, but most code in docs isn’t shaped like a doctest. The friction is too high; the coverage is too low.
  • Generating docs from source. Sometimes the right answer — but docs aren’t just an API reference. Prose around code matters, and the wrong granularity strips that out.

What does §

The snippet should not exist as a copy. It should be a reference that gets resolved at build time:

Here's how we handle the request:

<!-- snippet src/handler.go region=handleRequest -->

…where region=handleRequest corresponds to a tagged block in the actual source file. When you build the docs, the snippet is pulled in fresh. If the function disappears, the build fails. If the function changes, the snippet changes with it.

I built aquila to do roughly this for Markdown docs — pull tagged regions from source files at build time, fail loudly when a tag is missing. There are other tools in the same neighbourhood; the specific tool matters less than the shape.

The broader point §

Documentation lies in the same direction every time: in the direction of yesterday’s code. The fix isn’t to be more disciplined — it’s to remove the option of lying. Snippets that are references can’t drift, because there’s nothing to drift from.

Docs are code. Build them the same way.