Do We Still Need Burp Suite for Web Security Testing?

Burp was first released in 2003, and ZAP was built around 2010. Caido is much more recent, built in 2022, but still before the coding-agent era. Both were designed for humans, while the web security landscape has shifted dramatically toward AI agents in recent years.

Today, an AI agent in a terminal can use command-line tools to browse an application, inspect HTTP traffic, write scripts to replay requests, research relevant vulnerabilities, and choose its next test based on the results, without opening any GUI-based systems like Burp.

Of course, Burp remains useful for automated scanning, interactive investigation, and reviewing an agent's work. It also recently relreased MCP-based support for AI agents. Whether to use it depends on the tests you need to run and how you want to inspect the results.

Agents can combine CLI tools to run the entire worlflow of web security tests

An agent can use Nmap for port and service discovery, rustbuster for web content discovery, curl for HTTP requests, ffuf for fuzzing, and Nuclei for existing vulnerability checks. mitmdump captures and modifies traffic, Playwright scripts automate full browsers, and h5i combines headless browsing with HTTP inspection, replay, and modification.

Disclosure: I've been an active user of Burp for several years, and also am the maintainer of h5i (600+ GitHub stars), an open-source security-first headless browser with built-in HTTP traffic capturing and modification. I admit that my preference for Burp Suite, along with my optimism regarding human-AI collaboration, might introduce some bias in this article.

AI agents tend to be very good at connecting these tools with bash scripts, such as repeating a request under different sessions, comparing responses, and saving a working reproduction. I really consider Burp Suite one of the best tools for web security testing, but agents can already complete many of these tasks using CLI tools alone, without Burp or its MCP integration.

Agents can solve most challenges in these benchmarks using only CLI tools

Thesedays, many researches have shown that AI agents can solve suprisingly massive areas of security tasks with only CLI tools.

"LLM Agents can Autonomously Hack Websites" gaves agents a Playwright headless browser, basic CLI tools like curl, and a Python interpreter. The agents use them to test sandboxed websites and execute multistep attacks, including database schema extraction and SQL injection.

Their follow-up, LLM Agents can Autonomously Exploit One-day Vulnerabilities, gave agents browser operations, a terminal, web search results, file editing, and a code interpreter. In one case, the agent navigated a checkout flow, collected the required fields, wrote a Python script to exploit a race condition, and executed it through the terminal. GPT-4 exploited 87% of the 15 vulnerabilities when supplied with CVE descriptions, compared with 7% without them.

Both studies used terminal commands and programmatic browser tools to execute attacks. They support testing without a human operating a security GUI, although neither was a CLI-only evaluation.

I also confirmed the agents' security-related problem solving power with only CLI tools. In my own h5i-benchmark experiment, a general-purpose agent like the normal Opus 5.0 model solved 100 of 104 XBOW validation challenges and 54 of 60 Argus validation challenges through h5i, python script, and other basic CLIs like curl. There are also some teams who have already shown that stronger agents or domain-specific agents can solve more challenges.

Caution: other teams had already published writeups of XBOW and Argus before our experiment. While the agent in my experiment did not directly access or observe those solutions during the run, there is a possibility that those writeups might have been included in its training data. These results only demonstrate that h5i supported the interactions needed to solve most challenges in the two corpora, and they do not show the agents'U performance on unseen vulnerabilities.

Burp still helps testers review agent findings

A GUI-based workspace of Burp is quite helpful for humanto inspect traffic and review uncertain findings. This process is still necessary when we want to check whether AI-generated report is valid or not. There are also relatively modern alternatives such as Caido or ZAP.

These tools can also work with agents. For example, both of Burp and Caido supports MCP integration with AI agents, so that human testers can automate the workflow with agents.

I think Burp, Caido, or ZKP are still necessary for exploratory testing and manual follow-up, while agents using CLI tools would defenitely make the entire workflow more efficient.

Model providers can block authorized testing

An agent may be capable of a test that its provider blocks. Anthropic's cyber safeguards documentation describes default blocks on certain high-risk dual-use activities, including activities with legitimate defensive applications. It offers a verification process for eligible users and acknowledges that approved users can still encounter blocks on legitimate work.

These restrictions aim to prevent abuse, but they can also interrupt an authorized assessment. Permission from the application owner does not guarantee access to the model capabilities needed for a test.

Save reviewed reproduction scripts, captured traffic, and explicit assertions so the team can verify findings without asking the model to repeat the investigation. A reproduction script can also become a regression test after the vulnerability is fixed.

We human still need the skills to verify agent findings

I would keep practicing manual testing and bug discovery even if AI agents can handle most workflows. Reviewing theiir output requires understanding HTTP, sessions, access controls, and the application's intended behavior. Model providers like OpenAI and Anthropic may also downgrade the security-related capabilities of agents (which they already do).

Agents can make the bug discovery and report writing process dramatically more efficient, but human testers still need to inspect the evidence, reproduce findings, and continue the assessment when the model fails or refuses a task. Burp and other interactive tools remain useful for that work.