Writing detection rules that survive contact with reality

A rule that's perfect in testing and useless in production is the default outcome, not the exception. Here's what actually makes the difference — and it's rarely the detection logic.

Anyone can write a detection rule that fires on the exact behavior they're targeting in a clean test. The hard part, the part that separates detection engineering from writing regexes, is writing a rule that's still useful three months into production — when the environment has drifted, legitimate activity has evolved, and the rule is either drowning the team in false positives or has quietly stopped firing on anything at all. Let's talk about what actually makes a rule survive.

The false positive problem is the whole problem

A rule that generates too many false positives doesn't just waste time — it actively degrades security, because it trains the team to ignore that rule, and a rule everyone ignores is worse than no rule, since it occupies the space where a working detection should be while providing false comfort that the behavior is covered.

This is the failure mode that kills most detection rules, and it usually traces to a rule written against a technical definition of the behavior rather than an understanding of how that behavior actually shows up in this specific environment, including all the legitimate reasons it might occur. The classic example: a rule detecting a sysadmin tool being used in a way that could indicate an attack, which also fires every single time an actual sysadmin does their actual job, because nobody checked what normal admin activity looks like here before shipping it.

What actually makes a rule durable

Baseline against real data before deploying. The single highest-value step, and the most skipped. Before a rule goes live, run it against historical production data. If it would have fired two hundred times last month, all benign, you've learned something crucial before inflicting it on the team rather than after. This one habit prevents most detection-rule failures, and it's skipped constantly because it's slower than just shipping.

Understand the legitimate causes of the thing you're detecting. Almost every suspicious behavior has benign explanations. Detecting large outbound transfers? Backups, migrations, legitimate bulk exports produce identical signatures. A durable rule accounts for the legitimate causes — through tuning, allowlisting known-good sources, or requiring additional context — rather than firing on the raw behavior and dumping the disambiguation on a human three hundred times.

Write for the environment you have, not the textbook one. Detection content from a vendor or a public ruleset is a starting point, not a finished rule, because it was written against a generic environment that isn't yours. It needs tuning against your baseline, your normal, your specific mix of legitimate activity. Deploying generic rules untuned is how teams end up with thousands of alerts nobody reads.

Add context at detection time, not investigation time. A rule that fires with enriched context — asset criticality, whether the user normally does this, related recent activity — produces an alert a human can triage in seconds. A rule that fires with just "this happened" forces the analyst to go gather all that context by hand, every time. The enrichment work done once in the rule saves the manual work done repeatedly by every analyst who touches the alert.

The maintenance problem nobody budgets for

Rules decay. This is the part that gets left out of every "how to write detections" guide. An environment isn't static — new applications, changed workflows, new normal patterns of activity. A rule perfectly tuned six months ago drifts out of alignment as the environment moves under it, and starts either over-firing on newly-legitimate activity or under-firing because the attack pattern shifted.

Detection engineering isn't a write-once activity, it's a maintenance discipline, and treating it as write-once is why so many detection libraries are full of rules nobody trusts. Rules need periodic review against current data, retirement when they've stopped earning their keep, and tuning as the environment evolves. Almost nobody budgets time for this, which is exactly why almost every mature SOC has a graveyard of rules that fire constantly and get universally ignored.

Where automation genuinely helps

Testing rules against historical data, automatically. Flagging rules whose firing patterns have shifted (a rule that fired five times a month now firing five hundred is telling you something changed). Measuring the true-positive-to-false-positive ratio over time so you can spot decay before the team starts ignoring the rule. This is all well-suited to automation and it makes the maintenance discipline actually sustainable instead of aspirational.

AI can also help draft the first version of a rule from a described behavior, which is genuinely useful for getting started — but the drafted rule still has to go through baselining and tuning against your environment, because the model drafting it has no idea what normal looks like for you. It's a faster first draft, not a finished detection.

What stays irreducibly human

Knowing what's normal here. The single most important input to a durable detection rule is a deep understanding of what legitimate activity looks like in this specific environment, and that's knowledge no tool has, because it's specific to your systems, your teams, your workflows. It's built by actually working in the environment and paying attention. That knowledge is the difference between a rule that survives contact with production and one that gets muted in week two — and it's the part of detection engineering that was always the actual craft.