The Blast Radius of Zero: Scoped API Keys for LLM Traffic

Every shared provider API key is a bet. The bet is that nobody leaks it, nobody logs it, and nobody abuses it. Infrastructure teams lose that bet quietly and often: the same key gets copied from the app to staging, from staging to CI, from CI to the agent prototype someone spun up over a weekend. By the time it's in a dozen places, rotating it means coordinating every team at once — so nobody rotates it, and the blast radius keeps growing.

The fix isn't guarding the master key harder. It's making sure the key you distribute is never the key that matters.

Virtual keys work on a simple principle: provider credentials stay sealed in the vault — Azure Key Vault or AWS Secrets Manager — and downstream, everything receives a scoped sk-nrouter-* key minted for one purpose. One key for the production app, one for staging, one for the CI pipeline, one for the agent. Each carries only the permissions its purpose requires.

The scoping is where the security lives. An IP allowlist binds the key to trusted CIDR blocks; anything else is refused with HTTP 403 in under a millisecond. A route ACL pins it to the endpoints it was created for, like /v1/chat/completions. Daily and monthly spend ceilings turn budget overruns into clean HTTP 402 refusals before any provider call is dispatched. Model allowlists decide which models the key may touch, so a staging key can never reach the expensive reasoning tier. RPM and TPM slots with Retry-After headers contain the rogue loop and the noisy neighbor alike.

Key hygiene follows the same strictness. You're shown the plaintext exactly once, at creation. After that, the database holds only a SHA-256 hash, verified against an in-memory index in the Rust gateway in sub-millisecond time. When a key turns up where it shouldn't — a repo, a log file, a chat transcript — revocation takes effect in under 10ms with a single click. No master-key rotation, no cross-team coordination, no incident.

Then there's the question nobody answers until finance asks it: who spent what? Every inference call is audit-logged against its virtual-key ID and client IP, and custom metadata tags like environment:production or cost-center:fintech attach to spend rows. Chargeback reporting becomes a query instead of an investigation.