July 30, 2026

My 3-Step Bug Bounty Recon Blueprint: The Ultimate 2026 Web Crawling Cheatsheet

If you are just throwing standard tools at a target and walking away, you are leaving thousands of dollars on the table. Modern bug bounty…

By Nitin yadav

3 min read

If you are just throwing standard tools at a target and walking away, you are leaving thousands of dollars on the table. Modern bug bounty targets are built on complex JavaScript frameworks, hidden APIs, and forgotten staging servers.

When you are going up against massive corporate networks, you have to operate like a one-man army. You can't just rely on passive scraping. You have to dig deeper into the infrastructure to find the unlinked parameters and hidden endpoints where the critical XSS and IDOR vulnerabilities actually live.

If you are coming here from my latest YouTube video, welcome. Fire up your Ubuntu terminal — here is my complete, unfiltered 2026 Web Crawling Pipeline. Copy, paste, and start hunting.

Part 1: The Core Crawlers

Before chaining tools together, you need to master the single-tool executions. These are the commands I use to establish a baseline on a single target.

Katana Deep Crawl

Standard deep crawl grabbing all known endpoints while silently outputting to a text file.

katana -u https://target.com -d 10 -jc -kf all -aff -silent | anew crawl.txt

Gospider Full Crawl

Excellent for standard crawling while blacklisting useless static files to keep your output clean.

gospider -s https://target.com -c 20 -d 5 --blacklist ".(jpg|jpeg|gif|css|tif|tiff|png|ttf|woff|woff2|ico)" | anew

Hakrawler with Scope

Quick and dirty scoped crawling.

echo https://target.com | hakrawler -d 5 -subs -u | anew hakrawler.txt

ParamSpider Discovery

Isolating parameters is how you find injection points. This strips out the noise.

paramspider -d target.com --exclude woff,css,js,png,svg,jpg -o params.txt

Waymore Historical URLs

Live crawling only shows you what's there today. Waymore pulls the historical data developers forgot to delete.

waymore -i target.com -mode U -oU urls.txt

Katana Headless Browser Crawl

Bypass basic bot protections and render the DOM just like a real user.

katana -u https://target.com -headless -d 5 -jc -silent | anew headless_crawl.txt

Extract Forms Only

Forms equal input. Input equals vulnerabilities.

katana -u https://target.com -f qurl -silent | grep "?" | anew forms.txt

Part 2: The "God-Mode" Pipelines

This is where the magic happens. By piping the output of one tool directly into another, we create automated discovery systems that find assets the crowd completely misses.

☠️ Katana Multi-Target Deep Crawl + JS Parsing

When you have a massive list of alive subdomains, feed them all into Katana to extract endpoints from JavaScript files across the entire infrastructure.

cat alive.txt | katana -d 8 -jc -kf all -aff -ef woff,css,png,svg,jpg,woff2,jpeg,gif,ico -c 50 -p 20 -silent -o katana_multi.txt

☠️ Gospider Recursive + Sitemap + Robots

The ultimate wide-net crawl. Parses sitemaps, robots.txt, and JS files, then cleans and sorts the output.

gospider -S alive.txt -c 30 -d 5 -t 20 --sitemap --robots --js -a -w --blacklist ".(jpg|jpeg|gif|css|tif|tiff|png|ttf|woff|woff2|ico|svg)" -o gospider_output && cat gospider_output/* | grep -oE 'https?://[^"]+' | sort -u | anew gospider_urls.txt

☠️ Triple Source: Hakrawler + Wayback + GAU

Merge live crawling with historical archives (Wayback Machine and Global AlienVault), sort for unique URLs, and probe them to see what is still alive today.

echo target.com | hakrawler -d 5 -subs -u > hakrawler.txt && waybackurls target.com > wayback.txt && gau target.com > gau.txt && cat hakrawler.txt wayback.txt gau.txt | sort -u | httpx -silent | anew all_crawled.txt

☠️ Katana Headless + Form Autofill + Screenshot

A heavy-duty headless crawl that actually interacts with forms and captures background XHR requests. This is how you find hidden APIs.

katana -u https://target.com -headless -d 6 -jc -aff -xhr -form -timeout 15 -silent -nc -c 20 | anew headless_interactive.txt

☠️ Cariddi Full Crawl + Secret Extraction

Crawl the target and simultaneously run regex to rip out API keys, auth tokens, and secrets hidden in the source code.

cariddi -u https://target.com -d 5 -s -e -ext 1 -plain -t 50 -c 20 | tee cariddi_results.txt && grep -E "(api|secret|key|token|pass|auth)" cariddi_results.txt | anew secrets_found.txt

☠️ Mass Parallel Domain Crawler

Got a massive scope? Run Katana concurrently across your target list using parallel and clean the output with uro.

cat domains.txt | parallel -j 10 "katana -u https://{} -d 5 -jc -silent" | uro | anew parallel_crawl.txt

☠️ JS Endpoint Extraction Pipeline (Katana + Gospider)

Hunt specifically for JavaScript files, pull them, and extract every hidden directory and endpoint buried inside the code.

katana -u https://target.com -d 5 -jc -silent | grep "\.js$" | httpx -silent | xargs -I@ bash -c 'curl -s @ | grep -oE "(\/[a-zA-Z0-9_\-\/]+)" | sort -u' | anew js_endpoints.txt && gospider -s https://target.com -d 5 -c 10 --js -q | grep -oE 'https?://[^"]+' | anew combined_crawl.txt

☠️ Recursive Crawl + Nuclei Auto-Scan

My personal favorite. Hunt down dynamic files (PHP, ASP, JSP), grab anything taking a parameter, and pipe those exact, highly-vulnerable targets directly into Nuclei.

katana -u https://target.com -d 6 -jc -kf all -aff -silent | tee crawl_output.txt | grep -E "\.(php|asp|aspx|jsp|do|action)(\?|$)" | nuclei -t /root/nuclei-templates/ -severity high,critical -silent -o crawl_vulns.txt

☠️ Historical + Live Merge Coverage

Get the best of both worlds. Pull historical URLs, run a live crawl, merge the lists, deduplicate, and check exactly which ones return interesting status codes (200, 301, 302, 403).

waymore -i target.com -mode U -oU waymore_urls.txt && katana -u https://target.com -d 5 -jc -aff -silent -o katana_live.txt && cat waymore_urls.txt katana_live.txt | uro | httpx -silent -mc 200,301,302,403 | anew merged_crawl.txt

☠️ The Ultimate Parameter Extractor

Run all three major crawlers simultaneously, merge the results, isolate the keys/parameters, and rank the top 100 most common parameters for targeted injection testing.

(gospider -s https://target.com -d 3 -c 10 -q; hakrawler -url https://target.com -d 3; katana -u https://target.com -d 3 -jc -silent) | sort -u | unfurl -u keys | sort | uniq -c | sort -rn | head -100 | anew top_params.txt

The Next Step

Recon isn't about just running tools; it's about building a system. Take these command pipelines, tweak the flags to fit your specific methodology, and you will start finding the hidden attack surfaces the rest of the crowd is completely blind to.

If this blueprint helped you level up your recon game, let's connect:

Happy hacking.