The Circuit Nobody Could Find: How Exact-Match Searching Nearly Cost Me the Audit

Somewhere in most organizations there is a circuit nobody understands. It shows up on a carrier invoice every month, it has a service ID and a price, and not one person currently employed can tell you what it connects or whether anything still runs across it. The people who ordered it are gone. The documentation, if it ever existed, left with them.

That was the problem on my desk last week, eight times over.

A carrier handed us an Ethernet services inventory: eight active circuits billed to us, each wrapped in the usual wall of identifiers. When I went looking for them in our own equipment — the firewall, the switch stack, any documentation at all — I found nothing. Not a partial match. Nothing. Which left me needing to prove a negative before I could call the carrier and dispute the bill: either these circuits genuinely have no footprint in our gear, or I was missing the footprint.

Proving a negative across two large network configs by hand is the kind of work that eats a day and still leaves you unsure. So I handed it to an AI. And what happened over the next few passes taught me something I keep coming back to: with this kind of analysis, the question you ask shapes the answer far more than the tool you ask it with.

The setup

I had three things to reconcile. The first was a firewall running config, around 15,000 lines, with five virtual routers — independent routing tables living inside the same box — and a few hundred address objects accreted over years. The second was a switch stack config, five members deep, every port labeled by hand across a decade of changes. The third was the carrier's inventory: sixteen rows and fifty-nine columns of dense carrier-side metadata covering eight circuits — service IDs, circuit IDs, the management addresses of the NTE (the network terminating equipment, the carrier's box that sits at our edge), gateway IPs, VPLS instance numbers, VLAN tags, location codes, and aggregation router details.

The plan was simple. Feed the model both configs, let it fully ingest them, then hand over the spreadsheet and ask one question: does anything in this inventory exist anywhere in our equipment? That question felt obviously correct. It was also, I would learn, the entire problem.

Round one: the exact-match pass

The first pass — running on Opus at the time — did competent senior-engineer work. It pulled every IP out of the spreadsheet and ran exact-match comparisons against both configs. Zero verbatim hits. Then it went a step further and compared at the /24 level — a /24 is a block of 256 addresses, roughly the size a single site link occupies — and caught something real: four of the carrier's gateway addresses, in ranges I'll call 192.168.4.x and 192.168.15.x, fell inside two subnets the firewall actually routed.

It traced the path cleanly. A static route on the primary virtual router handed off to a transit router, which forwarded out the DIA (dedicated internet access) interface. It even flagged that the interface was labeled for one carrier while the address space it carried matched another. Useful.

I asked it to double-check the routing. It re-extracted all 57 routes on the primary router, did real subnet-containment math instead of eyeballing octets, and was refreshingly honest about noise: most apparent "matches" were just the default route catching everything, which tells you nothing. Final answer: two routes, four gateway IPs, nothing else anywhere in the config.

That looked exhaustive. Two routes, four IPs, done. And if I had trusted it, I would have sent the carrier a report that was accurate and incomplete at the same time — with no idea the second half was missing.

Round two: asking a broader question

Before writing anything up, I ran one more pass with a deliberately wider instruction. Stop asking whether the spreadsheet's values exist in the config. Start asking every way the carrier's address space could touch this firewall at all, including ranges broader than, adjacent to, or historically related to the literal numbers in the sheet.

I want to be straight about what I changed, because it's the part most write-ups quietly skip. I moved two variables at once. The question got wider, and I also switched models, because Claude Fable 5 had just become available and I ran this pass on it. On top of that, it was a third pass over ground two earlier passes had already mapped. Three things changed together, so I can't honestly credit the result to the model — I never re-ran the older model with this same broad instruction to see whether it would have found the same things. What I can tell you is what the broader question surfaced. It was a lot.

The first thing it did was catch an entire category the earlier passes had skipped. The spreadsheet listed IPv6 addresses for the carrier's terminating equipment — six of them — and no prior pass had checked IPv6 at all. That single omission is enough to quietly invalidate any claim of an "exhaustive" search. The broad pass swept both configs for that range, confirmed neither contained any of it, and closed the gap instead of leaving it silently open.

Then it widened the surface itself. Rather than only checking routes and raw IP strings, it went through every section type in the firewall config — address objects, address groups, NAT rules (network address translation, which rewrites addresses as traffic crosses the firewall), security policy, VPN definitions, DHCP, DNS, logging targets, the lot. And it added the technique that actually broke the case open: supernet overlap.

Here is why that matters, because it is the whole point. A spreadsheet lists narrow ranges — a /24, 256 addresses. But a firewall can hold a much broader object, say a /16, which contains 256 of those /24 blocks. That broad object can completely contain a carrier's range while never matching it character for character. Exact-match search, and even same-size /24 comparison, sails right past it. You have to deliberately ask "is anything in here bigger than what I'm looking for, and does it swallow my target?" Almost nobody asks that by default.

The exhaustive-sounding answer was a function of the narrow question, not of the configs being clean.

The supernet sweep turned up four traces every prior pass had missed:

1. Two address objects defining a 192.168.0.0/16 block, labeled for a separate internal network, that quietly contained all six of the carrier's gateway addresses — including two that had no specific route at all.
2. That /16 object was a member of an address group referenced by three active security rules. The firewall's policy still permits this carrier's address space today, even where routing does not actually deliver traffic to it.
3. An address object literally named after the carrier's gateway subnet, prefixed "Remove_", with its value deliberately changed to a bogus range. Someone, years ago, neutered the object instead of deleting it — direct archaeological evidence that the subnet was once live.
4. A group description that read "From [previous firewall vendor]: (Interface was [other carrier]-DIA)", which explained the carrier-label mismatch in a single line. The objects had been migrated wholesale from the prior firewall, labels and all.

An address object that exists is trivia. An address object referenced by three active allow rules is an attack surface and an audit finding.

It also surfaced something with real operational weight: two of those six gateways, including the one for our highest-bandwidth circuit, were permitted by policy but had no route. Traffic toward them would follow the default route out to the internet and die. Either those circuits are supposed to work and our routing is broken, or they aren't supposed to work and we may be paying for dead links. That ambiguity is exactly why we ended up requesting a utilization report from the carrier.

For thoroughness, the pass also ran every non-IP identifier — hostnames, location codes, port identifiers, equipment model numbers, raw service IDs, billing account numbers — as plain strings against both configs, and even chased down a few false hits. A "match" on the string "ASE" turned out to be a fragment of crypto profile names, not a real reference. Then it produced the formal report.

The scoreboard

Laying the two phases side by side:

Exact-match passes Broad-spectrum pass Traces identified 2 (the static routes) 6 (routes, supernet objects, active rules, deprecated object, migration note) IPv6 checked No Yes, gap identified and closed Supernet overlap analysis No Yes, this is what found the policy exposure Security rule usage traced No Yes, three active rules identified Root cause of label mismatch Flagged as unknown Explained (firewall migration artifact) Unrouted-but-permitted gateways Not detected Two found, including the 4 Gbps circuit

One environment, one case, so weight it accordingly. But within this investigation the broad pass found three times the traces, caught a verification gap the earlier work had left open, and converted a flat "nothing else is here" into something actionable: which circuits our gear can actually reach, which it merely tolerates in policy, and which seem to exist only on the invoice.

Why the difference

Watching the same problem worked two ways, the lesson wasn't about horsepower. It was about scope.

The exact-match pass answered the question I asked. I asked whether the spreadsheet's values appeared in the config, and that is exactly what it checked — carefully, honestly, with sound subnet math.

The broad pass answered the question I actually needed answered. "Does this exist" became "what is every possible way this address space could touch this firewall, including broader, adjacent, and historical forms." Supernet containment, tracing objects into live policy, reading deprecated objects as evidence — these are all moves an experienced firewall engineer would make on a third pass. The wider instruction is what put them on the table. Same data, same configs, different question.

The IPv6 miss is the part that stays with me. An entire address family went unexamined across two passes, and the only thing that surfaced it was widening the question from "do these specific values appear" to "what haven't we looked at yet."

What to take into your own config archaeology

A few things I'll carry into the next one of these.

Exact-match searching proves almost nothing here. A carrier inventory describes the carrier's side of the demarcation point — the demarc, where the carrier's responsibility ends and yours begins — and your config describes yours. The two rarely share a literal string, because the overlap lives in subnet math, not in matching text. Which is why supernet overlap, checking for objects broader than your target rather than only equal to or smaller than it, isn't optional. The genuinely dangerous object is the wide one somebody created fifteen years ago, and it will never string-match a narrow range from a spreadsheet.

Existence is also the wrong bar for an object. An address object that merely exists is trivia; an address object referenced by three active allow rules is an attack surface and an audit finding, and the only way to know which you are holding is to trace it into policy. Deprecated objects deserve the same attention. That "Remove_" object told us more about this circuit's history than any document we still possess.

And the meta-lesson, the one that nearly bit me: re-run with a broader question before you trust the word "nothing." I would have shipped a confident, incomplete report if I had stopped after the exact-match pass. The fix wasn't a smarter tool. It was a wider question on a fresh pass, and that pass cost a few minutes against four findings and a carrier dispute now backed by real evidence. A newer model on that pass doesn't hurt — but the load-bearing move was changing the question, not the tooling.

We closed by sending the carrier a formal request for per-circuit utilization data, because a config can only ever tell you what should flow, never what does. But we walked into that conversation knowing precisely which circuits our equipment can reach, which it merely tolerates, and which appear to live only on an invoice. That is a far stronger place to negotiate from than "we couldn't find anything" — which is exactly where the week started.

All names, addresses, and identifiers in this article have been altered. The methodology is real; the data is not.

Originally published at blog.vertexops.org.