Introduction
My end-of-first-year project aims to automate vulnerability analysis by gathering service versions from different virtual environments (Virtual Machines) and generating reports in a PDF with a design I created, along with a Markdown file. I designed, managed, and completed this project independently. I developed the web app, created the database, set up Docker containers and VMs, and conducted pentests using Nmap specifically. All of this took about 6 weeks:
Week 1 involved initiating penetration tests, mastering Docker and Docker Compose, setting up PostgreSQL with schema creation, and exploring CVE search REST API integration. In Week 2 and 3, efforts were focused on developing the first version of the tool, establishing the PDF template, and configuring Apache for Flask deployment. Moving into Week 4 and 5, tasks included crafting frontend HTML and Python scripts for each section of the web app, alongside implementing Apache-based login and refining the Pandoc report template. By Week 6, achievements included enabling report downloads from the webpage (ZIP file), integrating real pentest findings, and finalizing comprehensive documentation updated through July 7th.
If this project were to be used by anyone other than myself (which is not its intended purpose, given how complicated it is to set up the entire architecture), it is designed for security analysts and IT professionals seeking to streamline vulnerability assessment.
My primary focus was on developing the backend architecture which includes: the Apache web server with Flask and PostgreSQL, the generation of reports in Python.
This project has allowed me to merge my passion for programming with my strong desire to delve into vulnerability discovery and cybersecurity in general. You can never truly get ahead, but the risk of falling behind is real — you must continually keep pace, and that is one of the aspects of cybersecurity and information security that draws me to it.
Architecture Overview
The architecture of my project comprises three main types of environments:
VMs:
- One VM hosts Kali Linux, providing essential penetration testing tools.
- Several other VMs were used for vulnerability testing with Nmap.
Docker Containers:
- Three containers are dedicated to CVE search, enabling local queries to a known vulnerability database (NVDs).
- Two containers serve as hosts for the application and database: + Apache serves as the web server. + PostgreSQL is utilized for the database.
Windows and WSL:
- Those environments are used for web browsing and managing Docker containers with docker compose.
Features (TL;DR style)
- Apache securisation with password authentication, password that is encrypted and stored in database.
- The reports are provided in commonly used and portable formats: Markdown and PDF.
- Easily generate the reports from a user-friendly web app interface.
Technologies Used
Aside from Python Flask and SQLAlchemy, all the technologies I used were new to me, and each came with a steep learning curve.
Python Flask and SQLAlchemy frameworks were chosen for developing the web application due to their suitability for building scalable web applications with Python, learned during my studies at Holberton.
Behind the design of the pages is HTML and CSS, as well as GRID.js for the display of the data in a dynamic grid that also has a filter.
Docker and Docker Compose were utilized to containerize both the application and database, simplifying the management of multi-container Docker applications and docker compose specifically simplifies greatly the syntax of Docker.
Apache was selected as the web server for its compatibility with Flask applications (using WSGI), while PostgreSQL was chosen as the database due to the fact that the Metasploitable console already provides a PostgreSQL database.
For formatting data into markdown I used mdutils while Pandoc and LuaLaTeX were employed for their capabilities in generating well-formatted PDF reports quickly.
Lastly, integrating the CVE Search API with Kali-Linux facilitated local searches in vulnerability databases (NVDs), complementing the active penetration testing capabilities of the Kali Linux VM and enabling comprehensive vulnerability assessments and penetration tests using one of the over 600+ tools Kali offers (Nmap with db_nmap).
+ Technical Challenge (the biggest one)
I was responsible for producing an automatically well-formatted report that included images and links. Initially, the project faced several significant issues: images were not displaying as required, page breaks were poorly placed, and links were not functioning correctly.
To tackle these problems, I began by conducting extensive personal research. This involved watching YouTube tutorials, reading blog posts, and consulting official documentation. Through this process, I tested numerous methods until I found effective solutions for each issue.
For the image formatting problem, I switched from using the \includegraphics{} tag to the \backgroundsetup tag in LuaLaTeX. By doing so, I could place content over the images if needed. This approach, although not part of the original plan, prevented the images from interfering with the rest of the content and stopped them from moving around unexpectedly.
The issue with the links turned out to be a specificity of LuaLaTeX. By defining the target of the links in lowercase, I was able to get them to function correctly. This solution was discovered through careful troubleshooting rather than direct research.
Ultimately, these changes resolved all the initial issues. The report was visually appealing and functionally sound, with correctly positioned images, appropriate page breaks, and fully operational links. The final product met all my design and content expectations.
+ Technical Takeaways
I Learned how to use Docker with docker-compose and performed penetration testing commands with Nmap. Moreover, I expanded my database management skills, particularly with database views, which were new to me; I also deepened my proficiency in Python and Flask.
I learned that I could manage learning new technologies while making sure to meet the deadline. My wish to work in cybersecurity in the future was also reinforced. This project was a way for me to get into cybersecurity during this first year, and I'm glad I took this route.
I got the confirmation that Docker is a very powerful tool to provision and manage environments, and I wish to delve deeper into it now that I'm not rushed by a deadline.
To end this article, let me introduce myself (I should have done that first, but oh well). I'm Chloe, I'm from the C#22 cohort, and Alvohive is a project that I'm very proud of and will definitely keep working on in the future.
Useful links: