Understanding URLs: Breaking Down Every Part of a Website Address

Introduction:

Today i learned so much about URLs which i was using without knowing about it. Each of the components in the URL stands a different story like why it is used. I am learning ethical hacking so this is one of the crucial topic in the path of learning ethical hacking because it is used for many purposes while pentesting. Since, attackers use URLs to manipulate the response from the server.

So lets dive deep into the URL:

What is an URL?

URL stands for Uniform Resource Locator. It is used to access web page and media like photos, videos, blogs, etc throughout the internet. It guides you to fetch perfect output you need from the website.

HTTP Messages:

There two types of HTTP messages:

HTTP Request: Sent by user to web server to request pages.
HTTP Response: Received from web server as a response to the user's requests.

HTTP Request: Request Line and Methods

Request Line:

GET /user/login.html HTTP/1.1

GET: GET is the method which is used to perform operations on the website. Specifically, GET is used to retrieve data from the server.

/user/login.html: It is the path used to fetch the desired page from the server and by the user.

HTTP/1.1: This represents HTTP version 1.1.

HTTP Request Headers

Request headers provide additional information to the web server about the client's request. They help the server understand how to process the request and respond appropriately.

Some commonly used headers include:

Host: Specifies the target website or server.
User-Agent: Identifies the browser, operating system, or client making the request.
Referer: Indicates the webpage from which the request originated.
Cookie: Sends stored session or user information back to the server.
Content-Type: Specifies the format of the data being sent (e.g., JSON or form data).

HTTP Request Body

The request body contains the data sent to the server, typically with POST, PUT, or PATCH requests. The format of this data is defined by the Content-Type header.

Common request body formats include:

application/x-www-form-urlencoded – Sends data as key=value pairs, commonly used in HTML forms.
multipart/form-data – Used for submitting forms that include file uploads such as images or documents.
application/json – Sends data in JSON format, widely used by modern web applications and APIs.
application/xml – Sends data using XML tags, mainly found in legacy systems and some enterprise applications.

HTTP Response Status Codes

After receiving an HTTP request, the server sends back an HTTP response indicating whether the request was successful or if an error occurred. The first line of the response, called the Status Line, contains:

HTTP Version — The version of HTTP being used.
Status Code — A three-digit number indicating the result of the request.
Reason Phrase — A short description of the status code.

HTTP status codes are grouped into five categories:

1xx (Informational): The request has been received, and the server is waiting for the remaining data.
2xx (Success): The request was processed successfully.
3xx (Redirection): The requested resource has moved to a different location.
4xx (Client Error): The request contains an error, such as an incorrect URL or missing authentication.
5xx (Server Error): The server encountered an issue while processing the request.

Some commonly used status codes include:

100 Continue — The server is ready to receive the rest of the request.
200 OK — The request was successful.
301 Moved Permanently — The requested resource has been permanently moved to a new URL.
404 Not Found — The requested resource could not be found.
500 Internal Server Error — The server encountered an unexpected error while processing the request.

Common HTTP Response Headers

Response headers provide additional information about the server's response and tell the client how to handle it.

Some commonly used response headers include:

Date: Shows when the server generated the response.
Content-Type: Specifies the type of content being returned, such as text/html or application/json.
Server: Indicates the web server software (e.g., Apache or Nginx). Many organizations hide this information to reduce unnecessary exposure.
Set-Cookie: Sends cookies from the server to the client's browser, often used for session management.
Cache-Control: Defines how long the response can be cached or whether it should be cached at all.
Location: Used with redirect responses (3xx) to tell the browser where to navigate next.

HTTP Response Body

The response body contains the actual data returned by the server, such as an HTML page, JSON data, images, or files. Since it may include user-generated content, developers should properly validate and sanitize data before sending it to prevent vulnerabilities like Cross-Site Scripting (XSS).

HTTP Security Headers

HTTP Security Headers add an extra layer of protection to web applications by helping prevent attacks such as Cross-Site Scripting (XSS), clickjacking, and MIME type sniffing. Some of the most important security headers are:

Content-Security-Policy (CSP): Controls which sources (scripts, styles, images, etc.) the browser is allowed to load, reducing the risk of XSS attacks.
Strict-Transport-Security (HSTS): Forces browsers to communicate with the website only over HTTPS, protecting users from downgrade and man-in-the-middle attacks.
X-Content-Type-Options: When set to nosniff, it prevents browsers from guessing a file's MIME type and ensures they use the value specified in the Content-Type header.
Referrer-Policy: Controls how much referrer information is shared when users navigate from one website to another, helping protect user privacy.

Properly configuring these headers significantly improves a web application's security posture by reducing common attack vectors and enforcing secure browser behavior.

Conclusion

Understanding how HTTP works is a fundamental step in learning ethical hacking. From URLs and request methods to headers, status codes, and security headers, every component plays a role in how browsers and web servers communicate. Mastering these concepts will make it much easier to analyze web traffic, use tools like Burp Suite, and understand how web vulnerabilities arise. This knowledge forms the foundation for exploring advanced topics such as authentication, session management, and web application security.

Contents