ASM: A Self-Hosted Attack Surface Management Platform and a Postmortem on the Bug That Took Four…

This post has two parts. The first explains what ASM is and why it exists, for anyone evaluating recon tooling or learning how an ASM pipeline is actually built. The second is a detailed postmortem on debugging one module of it written for anyone who's dealt with intermittent failures in a multi-process scanning pipeline and knows how easy it is to chase the wrong root cause with total confidence.

Part 1: What ASM Does

Most attack surface management tooling falls into one of two categories: an expensive SaaS subscription, or a pile of scripts held together with cron. ASM is a self-hosted middle ground one platform that runs a full recon-to-report pipeline, keeps point-in-time history of every scan, and scores risk against something more grounded than a raw CVSS number.

Point it at a domain. It runs, in sequence:

• Subdomain enumeration — Subfinder + Amass
• DNS resolution and WHOIS/ASN lookup — registrar, expiration, name servers, network ownership
• Port scanning — Nmap
• HTTP probing — httpx, following redirects, capturing status/title/headers
• Technology fingerprinting — WhatWeb, merged with httpx's own detection
• Directory/content discovery — feroxbuster (the subject of this postmortem)
• Vulnerability scanning — Nuclei templates
• TLS/SSL auditing — sslyze: deprecated protocols, weak ciphers, expired certs, Heartbleed
• Screenshot capture — EyeWitness

Every scan diffs against the previous one assets get flagged new, changed, or disappeared based on a content hash of ports, tech stack, and HTTP state. That diff drives alerts and optional webhook delivery. Findings roll into a client-ready PDF: executive summary, infrastructure section, vulnerability table with CVE links, and severity-tiered remediation SLAs.

The detail that matters most to anyone doing this professionally: risk scoring cross-references every matched CVE against CISA's Known Exploited Vulnerabilities catalog. A finding scoring 9.8 on paper and a finding that's actively being exploited in the wild are not the same thing, and most tooling treats them identically. A Critical in ASM means confirmed active exploitation, not just a high number.

Architecture, for anyone building something similar: React frontend, FastAPI backend, Postgres for state, Redis + Celery for the scan queue, Celery Beat for scheduled recurring scans, six services total under Docker Compose. Every scan runs as one Celery task, updating stage-by-stage progress in Postgres so the frontend can show live status instead of a spinner.

If you're a student building your own version of this: the hard part is never the individual scanner wrappers subprocess calls to Nmap or httpx are a Tuesday afternoon. The hard part is everything below, and it's the actual subject of this post.

Part 2: The Postmortem

Module under investigation: directory/content discovery (feroxbuster) — brute-forces hidden paths and endpoints per live host, surfacing admin panels, backup files, and config artifacts none of the earlier stages can find by design.

Session length: ~6 hours. Root causes found: one real concurrency bug, one real race condition, two self-inflicted regressions, zero target-side or infrastructure problems despite a solid chunk of the session spent convinced otherwise.

If you've run a scanning pipeline against a live target for hours at a stretch, you already know how this goes: something starts behaving inconsistently, and the target itself becomes the easiest thing to blame, because blaming your own last commit requires actually reading it again.

Phase 0 — Audit before debugging

Before touching the actual failure, I audited the module against a standard every stage in this pipeline has to clear: would this survive being questioned in a real engagement report? It failed on four counts none of them the eventual bug, but real technical debt worth naming on its own:

GapWhy it mattersFixNo raw tool output retainedA finding can't be defended without the source data behind itRaw JSON per host written to disk, referenced by path deliberately kept out of Postgres, since raw output at scale bloats row size and slows backups for zero query-time benefitFailures silently absorbed into "success"A scan failing on 3 of 10 hosts looked identical to one succeeding on all 10Distinct module_status value for partial failureNo file-extension brute-forcingBackup files, exposed source, config dumps a whole finding class invisible by defaultAdded, though this needed a second pass see Phase 1Path parsing via naive string replacementBreaks silently if the base URL recurs inside a discovered pathReplaced with urllib.parse

Phase 1 — Two regressions I created in the same sitting

Regression A: extension multiplication. Adding eight file extensions sounds minor. It isn't — it multiplies request volume by roughly the extension count plus one:

~2,500-word list × 9 variants (bare word + 8 extensions)
≈ 22,500 requests per host
at rate-limit 10 req/s:
≈ 37.5 minutes per host
process timeout budget: 5 minutes
→ every scan times out, on every host, immediately

~2,500-word list × 9 variants (bare word + 8 extensions)
≈ 22,500 requests per host
at rate-limit 10 req/s:
≈ 37.5 minutes per host
process timeout budget: 5 minutes
→ every scan times out, on every host, immediately

For anyone who's tuned ffuf, gobuster, or feroxbuster against a rate-limited target: this is an easy trap. Extension flags feel like a coverage improvement, not a budget decision until you do the multiplication.

Fix: extensions gated to the wordlist tier already labeled "thorough, slower" in the UI. The tier labeled "fast" stays fast.

Regression B: unbounded link extraction. --extract-links follows discovered links recursively independent of --depth, which only bounds directory recursion, not link-following. Same failure class as Regression A, different mechanism, shipped in the same round of changes. Removed.

Both trace to the same root habit: adding a capability without recalculating what it costs against a budget that already existed. Diagnosing each took longer than writing the fix the actual lesson is in that asymmetry.

Phase 2 — A theory worth being wrong about

With both regressions fixed, timing still didn't match a clean earlier baseline. The target sat behind Cloudflare, and by this point had absorbed hours of repeated automated traffic across curl, Nmap, httpx, and multiple feroxbuster runs. WAF-side soft-throttling was the obvious next theory plausible, consistent with every symptom, and psychologically comfortable, because it doesn't implicate anything you wrote.

I tested it instead of trusting it: ran the identical command against an internal lab VM — no WAF, no CDN, no external throttling possible.

It hung too. Theory dead in under a minute.

If you're newer to this: this is the actual skill that separates fast debugging from slow debugging. Not knowing more theories testing the one you already have against a control before building the next layer of reasoning on top of it.

Phase 3 — The real root cause

Two processes were running concurrently against the same two hosts, splitting the same rate-limit budget and CPU. Confirmed directly: ps aux inside the containers showed two separate Nuclei processes, started roughly 17 minutes apart, both hammering identical targets.

The mechanism, for anyone running a similar Celery + Redis setup:

T+0     scan A acquires scan_lock:{target_id} in Redis, begins running
T+12m   `docker compose down` run to apply a code change
        → stops every container, including Redis
        → Redis has no persistence configured — in-memory lock state is gone
        → scan A's Celery task itself survives (the worker container isn't
          what got killed; only Redis was wiped)
T+17m   scan B triggered — acquires scan_lock:{target_id} because Redis
        now sees no lock at all, even though scan A never stopped running
T+17m→  both scans execute against the same hosts simultaneously

T+0     scan A acquires scan_lock:{target_id} in Redis, begins running
T+12m   `docker compose down` run to apply a code change
        → stops every container, including Redis
        → Redis has no persistence configured — in-memory lock state is gone
        → scan A's Celery task itself survives (the worker container isn't
          what got killed; only Redis was wiped)
T+17m   scan B triggered — acquires scan_lock:{target_id} because Redis
        now sees no lock at all, even though scan A never stopped running
T+17m→  both scans execute against the same hosts simultaneously

Nothing in the scanner code was broken. Nothing on the target's infrastructure was throttling anything. The debugging process itself — restarting containers to apply fixes without accounting for what that does to a coordination layer with no persistence manufactured the exact symptom under investigation.

For anyone building a similar lock: a Redis lock without persistence is only as durable as your assumption that Redis never restarts mid-task. In a long debugging session, that assumption breaks constantly.

Phase 4 — The bug that was real from the start

Not every lead this session was a false one. A separate intermittent failure empty results from the HTTP probing (httpx) stage was a genuine race condition, not session noise. The exact same httpx command that returned clean JSON when run directly in a shell came back with returncode=0, empty stdout, empty stderr, when invoked from inside a Celery prefork worker via a helper that calls os.setsid() to create a new process group.

That helper exists for a real reason: Amass spawns a long-lived "engine" child process that survives a plain subprocess.run(timeout=...) kill, and process-group signaling is the correct way to reap the whole tree. httpx-toolkit is a single well-behaved process with no children — it never needed that helper, and reusing it for convenience introduced a fork-interaction flake the tool doesn't have on its own.

Current fix: a one-time retry on empty output. I'm labeling that precisely it's a mitigation, not a resolution. The real fix is splitting "tools that need process-tree cleanup" from "tools that don't" at the call site, tracked as follow-up rather than quietly marked done.

What shipped

• Extension brute-forcing gated to the slower tier only
• Unbounded link-extraction flag removed
• Raw per-host tool output retained on disk, referenced by path, kept out of Postgres by design
• Partial-failure state surfaced distinctly instead of collapsing into "success"
• Path parsing hardened with urllib.parse
• One-time retry guard on the httpx race condition (explicitly labeled as a mitigation)
• Per-scan directory-discovery on/off toggle wired end-to-end schema, API, Celery task signature, conditional stage execution

What's still open

• The httpx/Celery fork interaction needs a real fix, not just the retry guard
• The Redis lock has no persistence a DB-backed lock, or Redis persistence enabled, would survive a broker restart instead of depending on nothing cycling mid-scan
• TLS/SSL findings aren't deduplicated on rescan (pre-existing, unrelated to this session)

The takeaway, for whoever's reading this

If you're experienced: none of these individual bugs are exotic. A rate-limit budget blown by a new flag, a lock with no persistence, a fork/subprocess interaction that only shows up under a specific worker model you've seen versions of all three. The value here isn't novelty.

It's the order: two regressions I caused, one theory I correctly killed instead of building on, and only then the actual bug and being honest in the writeup about which was which instead of presenting it as a straight line from problem to fix.

If you're a student: this is what real debugging looks like, and it's not a straight line. You will introduce your own bugs while fixing other bugs. You will have a theory that fits the evidence perfectly and is still wrong. The skill isn't avoiding that it's building the habit of testing a theory against a control before trusting it, and being precise about the difference between "I fixed this" and "I mitigated this," even when nobody's checking.

Code at github.com/TurlaFSB/ASM-.