Agentic goes purple: AI-driven detection validation with the Atomic Red Team MCP — Underground Labs…

A threat report lands. Somewhere in it are the techniques an actor used, and my job is to know whether my detections would catch them. The honest way to find out is to run each technique on a lab machine and watch what the detection sees. It is slow, so we read the report, nod, and move on, trusting that the rules we already have are enough.

So I built an agent that works from the detection rule itself: it reads a rule, runs the technique the rule claims to cover, and checks whether the rule fires.

Detected is not the same as seen

A rule that maps to a technique is making a claim: run this, and I will fire. Whether that claim is true is a separate question, and the answer only exists after the technique actually runs and you look at the telemetry.

When you look, there are three outcomes, not two. The rule fired. The rule did not fire, but the activity was there in the logs. Or there was nothing in the logs at all. The first two are what people expect. The third is the one that gets missed, because on a dashboard it looks identical to the second. Both show no alert.

But "the rule missed it" and "I never saw it" are opposite problems. One is a weak rule I can fix. The other is missing collection, a blind spot where no rule could have fired because the data never arrived. Atomic Red Team gives me a clean way to run a single ATT&CK technique on demand. The hard part is not running it. It is reading the result honestly enough to tell a gap from a blind spot.

What I built

An agent that reads a detection rule, picks the atomic that matches the technique it claims to cover, runs it, and checks Sentinel to see whether the rule fired.

The pieces: a LangChain agent, the Atomic Red Team MCP server (by CyberBuff), and an isolated Windows VM that ships its telemetry into Sentinel.

The VM runs in an isolated environment, managed with Terraform. The isolation matters because the atomics do real, sometimes destructive things. And because it is all code, a broken or dirty VM is not a problem: I tear it down and bring a clean one back up. Sysmon records the activity and the Azure Monitor Agent ships it to Sentinel, so the lab has honest telemetry to check against.

Implementation

Stack:

• LangChain, the agent and its tool calling
• Atomic Red Team MCP server, the emulation
• Azure: isolated Windows VM, Microsoft Sentinel
• OpenAI gpt-4o-mini, the model that picks the atomic

The agent and the MCP

The agent uses two of the server's tools, query_atomics and execute_atomic, and never touches generate_atomic. That choice matters: it runs tests the community already wrote and reviewed, it does not invent offensive code. The model's only job is to read the query_atomics results and name the atomic that matches the rule, not to write a command or drive the tools itself.

The verdict

After the atomic runs, the check asks the workspace two questions against the execution window, in order. They are two separate queries, and the order is the whole point.

The first asks whether any process telemetry arrived for this host in the window, with none of the rule's matching logic:

Event
| where TimeGenerated between (datetime(2026-11-01T08:00:00Z) .. datetime(2026-11-01T08:10:00Z))
| where Computer == "ai-purple-vm"
| where Source == "Microsoft-Windows-Sysmon" and EventID == 1
| count

Event
| where TimeGenerated between (datetime(2026-11-01T08:00:00Z) .. datetime(2026-11-01T08:10:00Z))
| where Computer == "ai-purple-vm"
| where Source == "Microsoft-Windows-Sysmon" and EventID == 1
| count

If that returns zero, the verdict is blind: the collection the rule depends on was not there, and no rule could have fired. There is nothing to match against.

If it returns rows, the check runs the rule's own query over the same window (the rule's ago(1h) is swapped for the exact execution window so an old time filter cannot hide a fresh event). If the rule's query returns nothing, the verdict is gap: the telemetry was there, the rule did not match it. If it returns rows, detected.

What each verdict tells you to do

The three verdicts are not just labels, they route the work to different places.

• Detected: the rule fired for this shape, as of today. I log it as validated coverage with a date, so it reads as evidence with a shelf life, not as trust that never expires.
• Gap: a rule problem. The data was there and the rule did not match it, so the fix is the detection logic. Broaden it, or add a rule for the shape it missed.
• Blind: a collection problem, not a rule problem. No rule against that table would have fired. The fix is upstream, in the data source or the DCR, and it belongs on the detection engineering backlog, not in the rule file.

The real run

One run, end to end, with nothing staged, came back gap. And it is a real one.

The rule matches powershell with -EncodedCommand, -enc or -e:

| where CommandLine has_any ("-EncodedCommand", "-enc", " -e ")

| where CommandLine has_any ("-EncodedCommand", "-enc", " -e ")

That line works. Here is a command the same rule caught cleanly, an encoded payload that decodes to Write-Output "hello from encoded":

"...\powershell.exe" -EncodedCommand VwByAGkAdABlAC0ATwB1AHQAcAB1AHQAIAAiAGgAZQBsAGwAbwAgAGYAcgBvAG0AIABlAG4AYwBvAGQAZQBkACIA

"...\powershell.exe" -EncodedCommand VwByAGkAdABlAC0ATwB1AHQAcAB1AHQAIAAiAGgAZQBsAGwAbwAgAGYAcgBvAG0AIABlAG4AYwBvAGQAZQBkACIA

The atomic the agent picked for the same technique runs powershell with -Command and encoded arguments, a different shape of the same idea. The technique ran, Sysmon recorded it, and the rule stayed silent because none of the strings it looks for are in that command line.

That gap is not a guess from reading the rule. It is what the endpoint actually recorded. This is not "I never saw it", the telemetry is right there. It is "I saw it and did not catch it". A coverage gap I can act on: the rule is narrower than the technique it claims.

The other shape: a detected

Run it again and the verdict flips. The agent picked a different atomic for the same technique, one that uses -EncodedCommand, and the rule fired.

The agent reads that from the telemetry in seconds. The deployed analytics rule, on its own schedule, then raised a real incident for the same execution. Same event, two views: the query matched now, the alert landed on the rule's cadence.

Same rule, same technique, opposite verdict. Coverage is not a property of a rule and a technique. It is a property of a rule and a specific shape of that technique.

From verdict to backlog

A gap printed to a console dies there. So the agent does not stop at the verdict.

On a gap or a blind spot it opens an issue in the detection-as-code repo the rule came from, carrying the whole case: rule, technique, atomic, window, counts, query. gap gets a detection-gap label, blind a collection-gap label, detected opens nothing. It is idempotent by title, so a re-run files no duplicate.

Where it breaks

• One run is one shape, not the technique. The model picks one atomic, so a run samples one variation. The same rule flipped gap and detected for exactly this reason. Real validation runs every atomic, not one pick.
• The atomic library is the ceiling. No atomic, no claim to check this way.
• The run is clean and isolated. A rule that fires on a single atomic can still miss the technique inside a messy real chain.
• It leans entirely on collection. If the table is not shipping, every rule against it reads blind, and blind is not safe.

Where this goes

The rule came from a detection-as-code repo, where the ATT&CK mapping is asserted in YAML and nothing tests it. The verdict is a cheap test, and a cheap test belongs in the pipeline:

• In CI: the mapping becomes a check that fails the build on a gap or a blind spot.
• Across the ruleset: a coverage map, detected, gap or blind per rule, monitored instead of audited once a year.
• On a schedule: a rule that flips to blind flags a collection that broke, before an incident does.

Final notes

This agent is a support tool, not an autopilot. It runs the technique and reads the telemetry back, but it does not decide what a gap means. Whether a gap matters, whether the rule should be broadened, whether a blind spot is acceptable for now, that is the engineer's call, and I think that boundary should stay exactly where it is. The model picks a test, the code reports what happened, a person decides what to do about it.

The value is not replacing the work. It is absorbing the repetitive setup that comes before it: spinning up the technique, scoping the window, running the query, filing the finding. At the scale of a ruleset that changes every week, that setup is where the hours go.

Picture a detection pipeline that already sees a stream of new rules and rule changes. Now imagine an agent picking each one up as it lands and handing the engineer a coverage verdict before anyone validates by hand: the technique run, the telemetry read, a detected, gap or blind, and a filed issue for the ones that need work. That is where this gets genuinely useful.

Repo: https://github.com/zdr4k/agentic-threat-emulation/tree/main