Hackers Did Not Need To Be Clever In October, Because We Kept Leaving Our Keys Lying Around

For most of the last two weeks, my feed has been a steady stream of security incidents. By October 22, people had started calling the period Black October.

I spent a couple of days reading the advisories and incident reports behind these incidents, mostly because I wanted to know which ones affected the kind of systems I build every day.

What I found was a little different from what I expected. I was less worried about clever hackers finding complicated vulnerabilities and more worried about how casually our industry handles secrets.

Some of these incidents involved serious software bugs that needed patching. But many others came down to something much simpler. A key that was too easy to guess. A password sitting in plain text. An API token stored on a platform that someone managed to break into.

After nearly eleven years of building backend systems, I recognize these habits. Most of us have probably made similar decisions at some point because we wanted to ship something quickly and move on to the next task.

And that is what makes these incidents uncomfortable.

The Private Key That Was Literally 4

The incident that made me laugh before it made me uncomfortable involved eMudhra, an Indian certificate authority.

If you have never worked with certificates, here is the short version.

A certificate authority is an organization that vouches for the identity of websites. When your browser establishes a secure connection, it relies on a chain of trust that can lead back to a trusted certificate authority.

The private keys used to sign certificates are a big part of that trust. They need to be generated using strong cryptographic randomness, making them practically impossible to guess.

Now, according to Mozilla's investigation, a faulty script run by eMudhra in April 2025 generated eight intermediate certificates with private keys smaller than 256.

Think about that for a moment. These were not keys that required years of computing power to crack. Someone could recover them by trying a maximum of 256 possible values.

As security researcher Scott Helme pointed out, one of those keys was literally the number 4.

Yes, the number 4.

Anyone who recovered those private keys could potentially issue trusted certificates for websites across the internet, including banking websites. That is a serious problem when browsers and operating systems are supposed to trust the certificates issued by your organization.

A researcher reported the issue to browser makers on October 17. Chrome and Firefox blocked the affected certificates that same day. Apple and Microsoft followed within three days. So far, there is no confirmed evidence that anyone abused them.

But the part I find difficult to understand is how these certificates remained in use for roughly 18 months before anyone caught the problem.

This was not some random website running on a forgotten server. This was a certificate authority, a company whose entire business depends on people trusting its security practices.

And one of its private keys was 4.

Your Production Secrets Are Only as Safe as the Platform Storing Them

The eMudhra incident was bad enough. But the incidents involving deployment platforms were the ones that really got my attention.

These are the platforms developers use because they make our lives easier. Connect a GitHub repository, configure a few environment variables, and your application is running without having to spend an afternoon configuring infrastructure.

I like that convenience. Most developers do. The problem is that we rarely think about what happens to our secrets after we hand them over to these platforms.

On October 21, Railway disclosed a security breach. An attacker had gained access to systems storing customer environment variables and decrypted some of them.

In layman's terms, that meant attackers could potentially get their hands on the keys to our applications, passwords to our databases, and credentials used to access software repositories.

Then there was Zeabur, a similar deployment platform that suffered two breaches in two months. Its preliminary report reportedly acknowledged that long-lived administrator credentials had been stored in plain text and could be accessed over the public internet.

The attackers accessed approximately 3.2 million environment variable records. Zeabur has since announced plans to shut down on December 31.

Embedded content

A few days earlier, OpenCode users also faced exposed AI provider keys saved in its Console product. Some users were hit with more than $1,700 in unauthorized charges overnight.

Embedded content

Three different incidents. Different platforms. Different attack paths.

But the underlying problem is familiar to anyone who has worked with production systems. We paste secrets into a dashboard, click Save, and forget about them.

The interface usually hides the value after we save it. That makes it feel as though the secret is safely locked away somewhere. But hiding a value on the screen does not mean it is protected inside the platform.

The platform still needs to store that value and make it available to the application. If its internal systems are compromised, those secrets may become accessible to an attacker.

And if the platform itself has poor security practices, there is only so much protection its customers can expect. This is why I think we need to stop treating secrets as ordinary configuration.

A database password that never expires, copied into three different platforms and a CI/CD pipeline, becomes a liability every month it remains active. If one copy leaks, every other system using the same password may be at risk.

Short-lived credentials are a much better approach. A service gets the access it needs, uses it, and lets the credential expire automatically.

Major cloud providers already support workload identity federation, which allows many workloads to access cloud resources without storing long-lived service account keys.

Of course, setting this up takes more effort than pasting an API key into a text box. But after watching what happened this month, I am not sure why we are still treating that extra effort as an inconvenience.

AI Found the Worst Bug of the Month

Not every incident in Black October involved leaked credentials or compromised software releases. Some were good old-fashioned software vulnerabilities.

PHP shipped emergency fixes on October 21 for two serious flaws associated with PHP2Shell that could allow unauthenticated attackers to execute code on affected servers.

CISA issued an alert advising organizations that could not patch to take affected PHP systems offline.

GitLab patched a vulnerability with a maximum severity score of 10 that affected self-managed releases dating back to 2014. Node.js fixed ten vulnerabilities across versions 22, 24, and 26. Meanwhile, attackers were exploiting a chain of Plex Media Server vulnerabilities to install cryptominers shortly after the details became public.

But the PHP vulnerabilities stands out for another reason. They were reportedly discovered with the help of AI.

Ryan Austin, a researcher at security firm Calif, used OpenAI's GPT-5.6 Sol to identify the flaws in August. Researchers at Wiz independently found the same vulnerabilities using AI models. The PHP maintainers then worked through a coordinated disclosure process before the fixes were released roughly two months later.

Embedded content

When I wrote in July about Google losing four of its top AI researchers in six days, part of the discussion was about the growing concern that advanced AI models could find and exploit software vulnerabilities at scale.

That concern had already influenced decisions about which frontier models would be made publicly available.

Now we have a more concrete example of what these capabilities can do.

AI helped researchers identify serious flaws in a widely used language runtime, giving maintainers a chance to fix them before broader exploitation.

That is good news. But I also think it makes things more complicated for everyone responsible for keeping software secure.

For years, developers could reasonably hope that obscure bugs buried in old code would remain undiscovered for a long time. Finding them required specialized knowledge, considerable effort, and sometimes a lot of luck.

AI is changing that equation. If a model can help a researcher find a serious vulnerability in PHP, similar capabilities could help an attacker find the next one. And attackers do not need to follow coordinated disclosure practices or wait for maintainers to prepare a patch.

I find that more concerning than any individual vulnerability announced this month. The same technology that helps defenders find bugs can also help attackers discover them. The difference is what happens after the discovery.

Defenders need to verify the finding, coordinate a fix, ship the patch, and convince organizations to install it. Attackers need only one vulnerable system that has not been updated.

And we all know how long some organizations take to install security patches.

Closing Thoughts

Black October was a reminder that security is not just about finding clever ways to break into systems.

We cannot prevent every breach or patch every vulnerability before someone discovers it.

But we can make sure that a leaked key does not unlock everything, that one compromised platform does not expose our entire infrastructure, and that fixing a security issue does not depend on someone remembering to do it.

That is the kind of engineering work that rarely gets noticed when things go right. Until they don't.

If you enjoyed reading this, you might also find the below articles worth your time.

AI Gave Junior Engineers an Intern Before They Had the Chance to Become One Eleven years in, and the part of the job that made me an engineer is being optimised out of existence.

AI Is A Better Excuse Than It Is A Replacement AI Is A Better Excuse Than It Is A Replacement Oracle needs the money, Microsoft needs the paperwork, and PayPal needed…