Managing a Million Security Profiles at the Edge

A global network can't keep every site's defenses loaded everywhere at once. The trick is to predict two things at the same time: which domains are dangerous, and which one a user is about to open.

By Jose Lejin P J · Principal Engineer & Inventor · Cybersecurity • Applied AI / GenAI • Cloud Platform Engineering · 50+ U.S. patent filings

A request lands on an edge server somewhere near the user, for a website that particular server has not handled in a while. In the split second before it responds, the server has to answer a quiet but consequential question: do I have the right protections loaded for this site, or am I about to defend it with a generic rulebook while its real, tailored defenses sit somewhere else?

That question sounds small. At the scale of a modern content delivery network, with edge nodes spread across the world and millions of domains flowing through them, it is anything but. It is the problem behind one of my granted patents, US 12,476,976 B2, "Managing security profiles in content delivery networks," which I hold as the sole inventor. And the way out of it turns a dumb memory cache into something that genuinely thinks ahead.

The problem: you cannot keep every defense everywhere

A content delivery network, or CDN, is not one machine. It is many edge instances distributed across regions, each sitting in front of applications and filtering traffic. Each instance typically runs a web application firewall, and that firewall applies a security profile, a tailored set of rules that says what to allow, what to block, and how to respond, on a per-domain basis. One site's rules are not another's.

Here is the bind. Any edge instance can receive traffic for any domain, so in an ideal world every instance would hold the security profile for every domain. But you cannot keep millions of profiles loaded on every node. There is not enough room, and the churn would be enormous.

The obvious workarounds all fail in their own way:

• Hold everything everywhere. Too expensive, and impossible at scale.
• Use one global profile for all domains. You lose per-domain customization, and a single rulebook inevitably breaks some applications while under-protecting others.
• Keep a limited set and evict the least recently used. This is the classic caching move, and for security it is quietly dangerous. Least-recently-used eviction will happily drop the tailored defenses for a domain that is high risk, or one that is about to be visited, simply because it has been quiet for a few minutes. You end up evicting exactly the protection you were about to need.

Recency, it turns out, is the wrong thing to optimize for when what you are caching is protection.

The idea: keep a limited set, but be smart about what you keep

The approach in the patent lets each edge instance hold only a limited set of security profiles, while being deliberate about which ones to keep and which to let go. Two structures make this work.

Each instance maintains a domain map, a simple mapping from each domain it currently holds to that domain's security profile. Alongside it sits an auto-adjusting list: the same domains, but ordered by eviction priority. The domain at the tail is the first to go; the domain at the head is the most protected from removal.

The runtime behavior is careful about latency. When traffic arrives for a domain the instance already holds, it applies that domain's profile and moves on. When traffic arrives for a domain it does not hold, it does not stall. It immediately applies a default profile, so the request gets at least baseline protection with no delay, then it evicts the domain at the tail of the list, fetches the new domain's real profile, and updates both structures. Nothing blocks on the hot path.

All of that is sensible caching. The part that makes it think ahead is how that list gets ordered.

Two predictions at once: the threat and the traffic

Off to the side, running asynchronously so it never slows the edge, a component the patent calls a domain security scaler continuously re-sorts each instance's auto-adjusting list. It does so using two very different signals, and holding both at once is the heart of the invention.

The first signal is risk. A risk processor ingests security events detected by the firewalls across every edge instance in the network, and from them computes a risk score for each domain: how much threat that domain is attracting or posing right now. High-risk domains are pushed toward the head of the list, the protected end. This is the crucial break from least-recently-used thinking: a domain under attack keeps its tailored defenses loaded even if it has been quiet on this particular node, because the network as a whole is seeing it get hammered. An attacker cannot get a target's strong protections evicted simply by pausing.

The second signal is anticipation. A domain predictor tries to figure out which domains a given edge instance is about to serve. It does this in a genuinely clever way: it takes a web application that was recently accessed through that instance, renders it, and analyzes the links inside it, the hyperlinks a user is likely to click next, following them more than one level deep if configured to. Those reachable domains are the possible next level domains, and their profiles get pulled toward the protected end of the list before the traffic for them ever arrives. It is prefetching, but for protection, guided by where people are actually about to go.

Put the two together and the auto-adjusting list is continuously reshuffled so that domains that are high risk or likely to be visited soon stay loaded, and domains that are low risk and unlikely to be needed drift to the tail and get evicted first. Because all of this happens off the hot path, the edge pays almost nothing for the intelligence.

Why this is really an AI problem in disguise

Step back and look at what those two signals are. One is a model of threat, learned from live security telemetry across a whole network. The other is a model of traffic, learned from how users actually navigate from page to page. The system is making two predictions at once, about danger and about demand, and allocating a scarce resource, loaded protection, to wherever those predictions say it will matter most.

That is fertile ground for machine learning, and it is where this design points. Risk scoring is a natural fit for models that learn the signatures of emerging attacks from patterns across millions of events, spotting a domain heating up before a human would. Next-domain prediction is a natural fit for models that learn navigation and traffic patterns, going well beyond static link-following to anticipate where demand will surge. The architecture is, in effect, a template for self-tuning security at the edge: a system that continuously re-prioritizes its own defenses from live signals, getting sharper as it learns.

There is a broader principle here that keeps proving out in security work. When a resource is scarce, stop allocating it by luck and start allocating it by consequence. Least-recently-used is a bet on habit. Risk-and-anticipation is a bet on what is about to matter. For caching a video, habit is fine. For caching protection, only consequence will do.

Practical takeaways

• Do not cache protection the way you cache content. Recency-based eviction can drop exactly the defenses you are about to need. Security caching needs different rules.
• Score by risk, not just by use. Feed security telemetry from across your fleet into a per-domain risk score, so high-threat targets keep their defenses loaded even when a given node has been quiet.
• Prefetch protection along the path of intent. Predict where users are about to go, by following the links they are likely to click, and warm the defenses for those destinations before the traffic arrives.
• Keep the intelligence off the hot path. Do the scoring and prediction asynchronously, apply a safe default instantly, and never make a user's request wait on the thinking.
• Treat it as a learning system. Risk scoring and next-step prediction both improve with models trained on real event and navigation data. Design for that from the start.

Closing

The edge server in that opening moment still has to decide, in a split second, whether it is ready to defend a site properly. The difference is that it no longer decides by accident. It decides based on what the whole network has learned about which sites are dangerous, and what the traffic is about to do next. It keeps the right defenses warm and lets the rest go, and it does so quietly, continuously, without ever making a request wait.

That is what it means for infrastructure to think ahead. Not a dramatic act of intelligence, but a steady, predictive allocation of scarce protection to exactly the places it is about to be needed. In security, being ready a moment early is most of the battle. The edge that anticipates beats the edge that merely remembers.

This article describes my own granted U.S. patent, US 12,476,976 B2, of which I am the sole inventor. The full record is publicly searchable through the USPTO and Google Patents.

Written by Jose Lejin P J. I spend my days building cybersecurity and applied-AI systems at internet scale, and I have spent the last several years turning the small frictions I run into along the way into 50+ U.S. patent filings. I write about cybersecurity, applied AI and GenAI, cloud-platform engineering, and the craft of innovation, the everyday business of turning friction into real inventions.