Tweeted this in February. Two months later I can name what I meant.
Success is one bit of information. Green. The thing worked, probably, maybe for the reason you thought, maybe not. You ship a landing page, conversions go up, you assume it's the headline. Six weeks later you change the CTA color and they drop. Was it the headline? The CTA? The time of year? Success doesn't say.
Failure says exactly which assumption was wrong, in what order, and what to try next. A failing test on a specific line, specific error, specific call stack, is hundreds of bits.
Generalizes well outside code:
- •email with no reply: the segment doesn't recognize themselves in your subject line
- •demo where they ask the same question three times: your pitch is doing the wrong thing in the first ten seconds
- •feature nobody used after launch: the workflow you imagined isn't the workflow they run
- •partnership that died in week three: you didn't know your real walk-away conditions
The information density of a clean failure is much higher than a vague success.
People avoid it because it's emotionally expensive in a way success isn't. You can move on from a win. You can't move on from a loss without looking at it. So people stop running tests they might lose. Stop sending the email that might not get answered. Stop pitching the idea that might get rejected. The cost is that they also stop learning. The comfortable hypothesis stays unchallenged. The data that would correct it never gets collected.
The discipline isn't liking failure, nobody likes it. It's going looking for it on purpose because that's where the information lives.
Founder version: ship something small enough to fail cheaply. Fail in public if you can stand it. Read the failure carefully before you patch over it, the patch is usually wrong if you skipped the reading.
Failure is interesting because it's honest. Success is often a lie you tell yourself with numbers.
— Simon