Survivorship bias is quietly wrecking how you think about breaches

There's a story security people should know by heart, and most don't. During the Second World War, the Allies wanted to armour their bombers better. They studied the planes that came back, mapped where the bullet holes clustered, wings, tail, fuselage, and prepared to reinforce exactly those spots. It was obvious. It was data-driven. It was completely backwards.

A statistician named Abraham Wald saw what everyone else had missed. The holes weren't showing where planes were vulnerable. They were showing where a plane could be hit and still make it home. The planes shot in the other places, the engines, the cockpit, weren't in the data at all. They hadn't come back. The armour needed to go where the holes weren't.

I think about those planes constantly, because security has exactly the same blind spot, and almost nobody names it.

We only study the breaches that came back

Every breach report, every post-mortem, every "lessons learned" deck is a bomber that made it home. It's an attack that was detected. That got caught, investigated, written up. And from that pile of survivors, we build our entire mental model of what an attack looks like.

But think about what's missing from that pile. The attacks that worked so well nobody ever noticed. The intrusions that are quietly ongoing right now, in companies that will tell you with total sincerity that they've never been breached, because they genuinely don't know. The breaches so clean they never generated a report to learn from. Those are the planes that didn't come back, and they're the ones we most need to understand.

What this does to your defences

Here's where the bias bites, and it bites hard. Because our data is made entirely of detected attacks, we optimise for the attacks we detect. We get better and better at catching the kinds of intrusions we already catch, reinforcing the bullet holes in the wings.

Meanwhile the attacker who's genuinely good, the one who moves slowly, uses legitimate credentials, touches nothing that trips an alert, generates no report, teaches us nothing, because we never see them. Our whole industry pours its attention into the loud, clumsy, detectable attacks, not because those are the most dangerous, but because those are the only ones that show up in the data. [If you've ever found an intrusion that had been quietly sitting undetected for a long time, this is where that story lands — how long it had been home before anyone saw the holes.]

The metrics make it worse

And then we measure ourselves on it, which completes the trap. "We detected and stopped 10,000 attacks this quarter" sounds like strength. But it's a count of bombers that came home. It tells you precisely nothing about the ones that didn't, and a rising detection number could mean you're getting better at catching things, or it could mean you're getting attacked more by the clumsy ones while the quiet ones sail straight past. The number can't tell you which, because the dangerous attacks are, by definition, not in it.

This is why "we've never had a breach" is one of the least reassuring sentences in security. It can mean you're secure. It can also mean you have no idea where the holes are, because your planes stopped coming back a long time ago and you called the silence safety.

How to actually think past it

Wald's fix wasn't to collect more data on the survivors. It was to reason about the absence, to armour the places with no holes, specifically because there were no holes. Security needs the same move.

It means assuming the attacks you can't see exist, and asking where they'd be if they did. It means hunting proactively for the quiet intrusion rather than waiting for the loud one to announce itself. It means treating a suspiciously clean environment not as a victory but as a question: are we actually secure here, or are we just not looking where the fatal shots land? And it means holding your detection metrics loosely, remembering they describe the survivors and stay silent about everything that mattered most.

The planes that came home taught the Allies the wrong lesson until one person thought about the ones that didn't. Our breaches are teaching us the wrong lesson in exactly the same way, and the fix is the same too: pay the most attention to the damage you can't see.

Where in your environment are you reinforcing the wings because that's where the holes are? I'd genuinely love to hear what you think your invisible shots are.