Hope you're doing well! Nowadays, for many of us it is probably hard to imagine our everyday life without AI or LLMs. We use these tools at day to day jobs, create new content, applications, build startups, learn new skills and much more. More and more people become AI/LLM users, connect MCP servers, build agents and do some cool stuff using this technology.

While AI is cool, it also brings new unique vulnerabilities that require attention and should be mitigated in order to protect end users from malicious actors and keep sensitive data secure. But, as we all know, to create a robust protection mechanism we should know how the attacker exploits different types of issues.

Before we start

As a pentester, I always try to keep up to date with the latest changes across industry and that's why passing AI/ML pentesting certification was on my roadmap probably since last year. One of the certifications that attracted my attention was Certified AI/ML Pentester (C-AI/MLPen) from SecOps Group. It became my fifth certification from this vendor and I am happy to announce that I have successfully passed it!

None

In this article I would like to share my thoughts on this certification and provide learning resources that anyone can use to prepare for the exam. It is important to mention that The SecOps Group's unique characteristic is that they do not provide training for their certifications unlike many other vendors. Instead, they suggest some resources that can be useful to prepare for the exam and list topics that will be covered, so exam takers can use it during their learning journey.

About the exam

As it is mentioned on the official website (https://pentestingexams.com/product/certified-ai-ml-pentester/) , the exam lasts 4 hours and is delivered online via The SecOps Group certification platform. It consists of 8 unique challenges that you should take and collect at least 6 out of 8 flags in order to pass the assessment. Covered topics include:

  • Prompt Injection - Direct Prompt Injections - Indirect Prompt Injections - Insecure Output Handling - Training Data Poisoning
  • Supply Chain Vulnerabilities - Traditional third-party package vulnerabilities, including outdated or deprecated components. - Using a vulnerable pre-trained model for fine-tuning. - Using outdated or deprecated models that are no longer maintained leads to security issues. - Use of poisoned crowd-sourced data for training.
  • Sensitive Information Disclosure - Incomplete or improper filtering of sensitive information in the LLM responses. - Overfitting or memorization of sensitive data in the LLM training process. - Unintended disclosure of confidential information due to LLM misinterpretation, lack of data scrubbing methods or errors.
  • Insecure Plugin Design
  • Excessive Agency - Excessive Functionality. - Excessive Permissions. - Excessive Autonomy.
  • Overreliance
  • Model Theft
  • System Prompt Leakage

As you can see from the above list, it seems to cover a lot. And I can agree that the exam covers many of them.

My feedback

The exam itself is a good challenge that will help you to test your AI/ML pentesting skills, creativity, out of the box thinking and self discipline. I would also mention here web/API pentesting skills because they can be helpful during some challenges.

Another useful skill or experience that can be helpful for you during the actual exam is experience in conducting simulated social engineering campaigns in scope of pentests. It is a highly valuable skill because at the end of the day in some cases you should convince an AI Chatbot to reveal a secret message to you and this type of skill can help a lot.

If we are talking about the exam duration, I would say that you most probably use at least more than half of the allocated time slot. In my case I used all the time and managed to secure my certification only during the last hour because some of the challenges were tricky and contained rabbit holes that can easily confuse you. When you experience a similar situation just step ahead and probably take a short break or switch to another challenge. This trick will help you to avoid being stuck and successfully pass the exam.

Additional valuable experience for me was participation in different CTF competitions from Wiz and Hack The Box where I had opportunities to learn AI pentesting and practice new skills. Especially some techniques learned during work on HTB challenges helped me to overcome exam tasks. But, in general, I think that if you invest a certain amount of time in preparation, you will be able to pass this exam for sure.

Learning resources that I have used

Most of the resources that can help you to prepare for this certification are already mentioned on the exam page, so I am not going to list them all. Instead, I would like to highlight material that I have personally used during my learning journey. So, here they are:

Series of writeups from HTB CTF competitions

Final thoughts

That's all for today. I hope the listed materials and feedback will help you to earn this certification. And stay tuned for the discounts from The SecOps Group, they often provide up to 90% discount for their certification that makes them affordable for community members. Good luck with your next exam and let's make AI applications more secure!