The Weirdest Cyber Security Incident That I Have Ever Handled

What pops into your Mind when I mention "Cyber Security Incident"? Most of you are probably thinking about a catastrophic piece of Malware that ravages Systems and causes a total Shutdown. You can see the Employees panic as they run around screaming for help while their Computer Screens flicker before a menacing Ransomware Message appears, threatening to take Action if Money is not paid. That is quite a vivid Imagination. However, what if I told you that the weirdest Incident that I faced was not even a Cyber Attack per se? You do not believe Me, do you? Well then, grab your hot Mug of Tea/Coffee for this is the Story of the weirdest Cyber Security Incident that a Team I once worked with and I have ever handled…

Disclaimer

Personally Identifiable Information [PII] like Names, Addresses, and other Sensitive Information have been redacted for Confidentiality. The Terms "Company X", "Company Y", and "Document Z" have been used to fulfil the same Purpose. I once worked with Company X, and Company Y was one of our Clients. No Legal Action was taken as the Incident was safely contained and remediated. Therefore, this Case Study is solely meant to raise Awareness and educate whoever it applies to…

What Happened?

On a random Morning, Company Y sent a Complaint to us citing a Data Privacy Breach. At first, I was obviously surprised to hear that since I had not received any Alerts from our Monitoring & Visibility Systems. However, I later learnt that the Breach was caused by an AI Engineer from one of our AI Teams who had downloaded Document Z from The Internet for Internal Testing without the Owner's Consent. How did Company Y find out, and why were they agitated by this? We had built an AI Product, and Company Y was our Client, meaning that they hosted this Product on their Infrastructure [GCP] for Testing. As a Result, Document Z was flagged by Company Y as soon as it was detected in their Infrastructure. Company Y demanded a formal Apology over the next 7 Days, accompanied by Remediations and other Measures taken to contain the Incident…

Coincidentally, my Team and I had just completed drafting and integrating the Playbook and Runbooks into our Incident Response Plan as part of Compliance. This was the perfect Moment to test how robust our Incident Response Plan was. Spoiler Alert. It was not…

As I was leading The Cyber Security Team, my first Course of Action was to investigate the Incident and gather as many Details as required. Through a Series of Phone Calls with The CTO and Emails with everyone involved, I now fully understood what had happened and proceeded to brief my Team as quickly as possible…

Close to Noon, I had already drafted and sent a Report to The CTO detailing what Steps we were going to take to handle this Situation. Were they perfect? Of course not. Our Incident Response Plan covered everything from Ransomware to Phishing to even DDoS Attacks [remember those?]. This was something else…

The Plan was good enough, and we received a Greenlight from The CTO to execute our Incident Response Plan. He also offered to help by granting us Access to the Servers and Databases. Why? Part of our Plan was Eradication, and this meant that we had to manually scrape Document Z and similar Entities from our Systems. This took Hours and by 19:00, we were done. Well, not done done. We still had to do what every Cyber Security Engineer dreads doing. Documentation…

Thankfully, my Team had done a marvellous Job of documenting the Findings as we handled the Incident and through those Findings and other Forensic Material, I put together a Report that was sent to the FinTech along with the formal Apology. This is the part where I thank Claude since how do you even start reporting such an Incident?

Through this quick Incident Response Plan, we were able to avoid Millions in Fines and, even worse, Legal Charges such as Tax Fraud and Identity Theft. I consider Myself too young to go to Jail anyway, not that I intend to at all…

ROOT CAUSES

Of course, my Instinct was to start blaming someone for this Mess. It was the AI Engineer. After all, if he had not used Document Z, none of this would have happened. Maybe it was The CTO. If he had not taken Disciplinary Action against any AI Malpractice made before, then maybe The AI Engineer would have never made that Mistake…

This was all in my Mind. However, I quickly reminded Myself whose Fault it truly was. He was the only one who had the Ability to prevent this Occurence, had he been proactive enough. That is who I want to focus on. So, what were the Root Causes of this Incident?

1] Inadequate Security Training

While I had done some Introductory Sessions, I did not kickstart the required Cyber Security Trainings that addressed Data Privacy and Best Practices. Had I conducted these Trainings, then every Staff Member, including The AI Engineer would have probably understood the Consequences of Data Privacy Breaches and used the right Resources for Internal Testing…

2] Unrefined Incident Response Plan

I highly commend my Team for their rigorous Efforts in establishing the Policies that now exist in Company X, given that they were not there in the first Place. The Cyber Security Team itself did not exist, by the way. Anyway, looking back, I realise that I should have done a better Job in anticipating Human-made Incidents like this. Our Incident Response Plan covered the common Cyber Attacks, but Real Life is anything but common…

3] Lack of Proactivity in AI Security

Delegation was instrumental in Me building and growing The Cyber Security Team. It is what helped us move fast and achieve major Milestones within a Fraction of the Time that everyone envisioned. What I overlooked, though, was being more involved. Since AI Security was not my immediate Domain, I paid little Attention to it. Instead, I was more involved in Application Security, including interacting with different Engineers to help them push more secure Code since that itself was quite an uphill Mission. Had I been as proactive with AI Security as I was with Application Security, then this Incident may never have happened…

IMPACT

Technical Damages are only one Part of the Equation, as Cyber Security Engineering has taught Me over the Years. What usually matters more is how it translates to Business Goals such as Revenue and Costs. From what happened, here was the Impact that was experienced:

• Reputational Risk: Probably the biggest, this Incident attracted Legal Action, which would have soiled Company X's Reputation not just with existing and Potential Clients, but with FinTech and other Industries. Translated to Business Language, this Incident could have cost both Money and Branding, with the Latter being permanent…
• Financial Risk: Do I need to explain how costly this Incident would have been should Legal Charges have been pressed? Today, it is clear how costly Cyber Security Incidents can be. Put simply, a Business wants to make Money. Cyber Security Incidents cost Money. Get it?
• Governance Risk: It turned out that Document Z had been used for Months without anyone from both The Cyber Security Team and The AI Team knowing. How was this even possible? Three Words: Governance, Risk, and Compliance [GRC]. It was evident that Governance was poorly implemented, and this meant that we had just discovered a major Vulnerability…

LESSONS LEARNT

You are probably here for this. What did I learn from all of this? Sure, it was quite bizarre and had its fair share of Chaos. However, through my Experiences, I know better than to let it pass. I understand that Incidents serve one Purpose above all others: Teach…

So, what did this Incident teach Me?

• Awareness is Key: If there is anything that this Incident taught Me the most, it is that most Cyber Attacks thrive on one Weakness, and that is Ignorance. You might have the most sophisticated Cyber Security Tools and Systems, but all it takes is for one Employee to click a Phishing Link not because they had malicious Intent [there are those too…] but simply because they were unaware of the Risks. Therefore, you can imagine how frequently I implemented Cyber Security Trainings ever since…
• Data is The New Currency: As obvious as it sounds, this Concept seems to be forgotten when it comes to AI. While AI has probably been the most useful Tool to be invented so far in terms of Technology, we keep forgetting that the same Cyber Security Risks that preceded it still apply. Whether it is using Data for Training Purposes or simply doing a quick Search, Data Privacy is still something, and as you have already seen, all it can take is just one Click and you are behind Bars. That would be unfortunate, would it not?
• You can never be too prepared: As I advocate for taking Responsibility, I must remind Myself that we are all Human. Well, most of us, at least, wink wink. AI is still a rapidly evolving Technology and as a Result, we are bound to constantly learn from it. Sure, we could have done everything possible to counter this Incident. Maybe we could have prevented it, maybe not. Heck, maybe another Incident would have occurred altogether. However, what I believe is the most important Takeaway from this is what we can learn from this. I am fully convinced that if this Incident had not occurred, then we would probably not have survived any other…

RECOMMENDATIONS

You now know what happened, why it happened, and what my Team and I learnt from it. Now what?

• Learn AI: Seriously, though. Learn AI. I am not only referring to Fundamental Concepts like Prompt Engineering. Start there. Do not end it there, though. Learn AI Security. Learn how AI Systems work. Learn what Risks they pose. Learn what AI-related Cyber Attacks have been executed. In this case, learn the Human Risks too. After all, you now know that AI will only replace those who fail to learn how to use it…
• Teach: Whether you are leading your Cyber Security Team or even a part of it, understand that it is your Responsibility to educate others in your Organisation and Circles about Cyber Security. It starts with you. It may not be your entire Job to keep them safe, but you are the Leader, so lead…
• Document: I get it. Most of us cannot stand Governance, Risk, and Compliance [GRC]. However, there is a Reason that it exists. Refer to the Compliance Policies and, as recommended above, teach others how to adhere to them. Document every Incident that happens, whether big or small. You never know when you will need to look back at your Notes during a high-stakes Incident…

CONCLUSION

As we come to the end of this Journey that you and I have been on, I want to remind you that Cyber Security is itself a Journey, and not a Destination. Yes, Hacking is fun, but it is not the full Picture. Most of Cyber Security actually has to do with People. Do they understand the Importance of Secure Best Practices? Do they practise those Best Practices? Do they understand the Risks posed by ignoring Best Practices? These and many more Questions will always be here to remind us to not only focus on our Systems but the People manning them too. I hope that my Experience has taught you that and much more. Stay tuned for more, will ya?

PS: I am not sure what my Signature Outro is yet, so hopefully this Conclusion was, well, conclusive…