Splunk: The Basics: Forwarders, Indexers, Search Heads, and Ingesting VPN Logs

Introduction

Welcome to my walkthrough of Splunk: The Basics! This room is part of the SOC Level 1 path, inside the Core SOC Solutions module. In my previous article, I covered Introduction to SIEM, where we learned why scattered logs create blind spots and how a SIEM collects, normalizes, and correlates them into actionable alerts.

This room takes that theory and puts a real SIEM in our hands. Splunk is one of the leading SIEM solutions on the market, and here we learn how its components fit together, find our way around its interface, and ingest and search our first set of logs.

Catch up on my previous article, Introduction to SIEM, by clicking the banner below.

What we will cover

• Splunk's three main components: the Forwarder, the Indexer, and the Search Head
• How to navigate the Splunk home screen, from the Splunk Bar to the Home Dashboard
• How Splunk ingests data and the five steps of the upload wizard
• Uploading a set of VPN logs into a custom index and querying them with SPL
• Answers to every question in the room

Room Information

Before we dive into the tasks, here is a quick overview of the room details.

• Room Name: Splunk: The Basics
• Path: SOC Level 1
• Module: Core SOC Solutions
• Topic: Splunk Components, Navigation, Log Ingestion, and SPL Searches
• Difficulty: Easy
• Room Link: TryHackMe — Splunk: The Basics

Task 1: Introduction

Splunk is one of the leading SIEM solutions on the market, letting analysts collect, analyze, and correlate network and machine logs in real time. This room explores the basics of Splunk and how it gives better visibility into network activity and helps speed up detection.

The learning objectives are to understand Splunk's components, explore some of the options available in its interface, understand how log ingestion works, and practice ingesting and analyzing logs. If SIEM is new to you, the room recommends completing Introduction to SIEM first.

Task 2: Connect with the Lab

Before moving on, start the attached lab machine by clicking the Start Lab Machine button. The machine can take 3 to 5 minutes to boot, after which the Splunk instance is reachable directly from your web browser. We will use this instance in the following tasks, so it is worth starting it now and letting it load while you read through the theory.

Task 3: Splunk Components

Splunk is built around three main components that work together to collect, process, and search data. The Forwarder is a lightweight agent installed on each endpoint you want to monitor. It uses very few resources, so it does not affect the endpoint's performance, and its only job is to collect data and send it on. Typical sources include web servers generating web traffic, Windows machines producing Event Logs, PowerShell, and Sysmon data, Linux hosts producing host centric logs, and databases logging connection requests, responses, and errors.

The Indexer receives the data from the forwarders and does the heavy processing. It parses and normalizes the data into field value pairs, categorizes it, and stores the results as events that are easy to search and analyze.

The Search Head lives inside the Search & Reporting app and is where analysts actually query the indexed logs using SPL (Search Processing Language). Each search is sent to the indexer, which returns the matching events as field value pairs, and the Search Head can then turn those results into tables and visualizations like pie, bar, and column charts.

Questions and Answers

Which component is used to collect and send data over the Splunk instance?

Answer:

Forwarder

Forwarder

Task 4: Navigating Splunk

The Splunk home screen is split into four sections. At the top, the Splunk Bar gives access to Messages (system notifications), Settings (instance configuration), Activity (progress of search jobs and processes), Help (tutorials and documentation), and Find (searching across the app), and it also lets you switch between installed apps. Below it, the Apps Panel lists the installed apps, with Search & Reporting as the default on every installation.

Next comes Explore Splunk, a panel of quick links for adding data, installing new apps, and opening the documentation. Finally, the Home Dashboard is empty by default, but you can pick from the dashboards already available in your instance or create your own, which then appear under the Yours tab.

Guided Walkthrough: Exploring the Add Data Page

With the lab machine running, open the Splunk instance in your browser. The home screen shows the Apps panel on the left and the Explore Splunk panel along the top, with the Splunk Bar above everything. In the Explore Splunk panel, click Add Data.

The page that opens asks what data you want to send to the Splunk platform. The top half offers onboarding guides grouped by category, but the part we care about is the bottom section, which lists three methods for getting data in, each with a short description underneath. The first one handles files uploaded from your own computer and the last one receives data from a Splunk forwarder. The method in the middle is the one described as collecting from files and ports on the Splunk instance itself, and its name is the answer to the question.

Questions and Answers

In the Add Data tab, which option is used to collect data from files and ports?

Answer:

Monitor

Monitor

Task 5: Adding Data

Splunk can ingest practically any data, whether event logs, website logs, or firewall logs. When data is added, Splunk processes it and transforms it into a series of individual events, and the documentation groups data sources into categories to make them easier to manage.

Uploading data through the wizard always follows five steps. Select Source picks the log file and data source, Select Source Type defines what kind of logs are being ingested (such as JSON or syslog), Input Settings chooses the index where the logs are stored and the hostname to associate with them, Review lets you double check everything, and Done completes the upload so the data is ready for analysis.

Guided Walkthrough: Uploading and Searching VPN Logs

Start by downloading the VPN_logs file from the Download Task Files button. If you are using the AttackBox, the file is already available in the /root/Rooms/SplunkBasic/ directory. The file is newline delimited JSON, so each line becomes one event once Splunk ingests it.

To import it, open Add Data, choose Upload, and select the VPN_logs file. Keep the JSON source type that Splunk detects automatically, create a new index on the Input Settings step, then click through Review and Done. I named my index vpn_logs, so that is the name used in every search below. If you named your index something else, replace vpn_logs in each query with your own index name, otherwise the searches will return no results.

Once the upload finishes, open Search & Reporting and set the time picker to All time, otherwise older events are left out of every count. For the first question, search the whole index and count its events with index=vpn_logs | stats count. The single value in the count column of the Statistics tab is the total number of events in the log file.

For the remaining questions, add | spath after the base search so Splunk parses the JSON fields from each event, then filter on the field you need. To count the events captured by a specific user, run index=vpn_logs | spath | search UserName="Maleena" | stats count and read the value in the count column.

To find which user is tied to a given IP address, run index=vpn_logs | spath | search Source_ip="107.14.182.38" | stats values(UserName) as UserName count. The values() function lists every unique username seen for that IP, so the UserName column shows who it belongs to, alongside the number of matching events.

The != operator excludes a value, so index=vpn_logs | spath | search Source_Country!="France" | stats count counts every event that did not originate from France.

Finally, index=vpn_logs | spath | search Source_ip="107.3.206.58" | stats count returns the number of VPN events associated with that IP address.

Questions and Answers

Upload the data attached to this task and create an index "VPN_Logs". How many events are present in the log file?

Answer:

2862

2862

How many log events are captured by the user Maleena?

Answer:

60

60

What is the username associated with IP 107.14.182.38?

Answer:

Smith

Smith

What is the number of events that originated from all countries except France?

Answer:

2814

2814

How many VPN events were associated with the IP 107.3.206.58?

Answer:

14

14

Summary & Key Takeaways

That wraps up Splunk: The Basics, where we moved from SIEM theory to hands on work with one of the most widely used SIEM platforms. With the components, the interface, and the ingestion process covered, you now have the foundation needed for the deeper Splunk investigation rooms, such as Splunk: Exploring SPL and Incident Handling with Splunk.

Key lessons:

• Splunk is built on three components: Forwarders collect data, the Indexer normalizes and stores it, and the Search Head lets analysts query it
• Forwarders are lightweight agents that run on endpoints without affecting their performance
• The home screen is organized into the Splunk Bar, the Apps Panel, Explore Splunk, and the Home Dashboard
• Uploading data follows five steps: Select Source, Select Source Type, Input Settings, Review, and Done
• SPL searches start from an index, and commands like spath, search, and stats let you parse, filter, and count events

Next up is Elastic Stack: The Basics, where we explore another popular SIEM solution and learn how to search and visualize logs with it. Click the banner below to check it out!