I built a free CLI that catches the security holes your AI leaves behind

npx kavaca — a free, local, open-source pre-flight check that finds exposed secrets and open Supabase databases in your AI-built app, in about 10 seconds. Nothing leaves your machine.

The uneasy feeling

I build a lot of small apps with AI tools now — Claude Code, Cursor, the occasional Lovable or Bolt prototype. They're fast. Sometimes too fast: you go from idea to deployed app in an afternoon, and somewhere in that afternoon you stop reading every line the model writes.

And every time I got close to shipping one, I'd get this uneasy feeling:

• Did it actually turn on row-level security for that Supabase table?
• Is my Stripe key sitting in an env var, or did it get committed?
• Would I even notice if a secret leaked into the client bundle?

I'm reasonably technical and I still couldn't answer those quickly. So I started checking by hand, every time. Then I got tired of checking by hand, and did what we all do — I scratched my own itch and wrote a script.

That script became Kavaca.

Why AI-built apps share the same holes

This isn't a "the AI is dumb" post. The models are great at writing code that works. The problem is that working code and safe code aren't the same thing, and the gap is exactly the stuff a senior engineer does on instinct — and never says out loud, so the model never learns to do it unprompted.

The result is a handful of mistakes that show up again and again:

• service_role keys or sk_live_… secrets committed into the repo or, worse, shipped to the browser.
• Supabase tables created with no RLS — which means anyone with your public anon key can read (or delete) rows.
• Secrets behind a public env prefix (NEXT_PUBLIC_, VITE_) that get inlined into the client bundle.

None of these are exotic. They're behind a huge share of the "my vibe-coded app got hacked" threads. They're also boringly detectable — which is the whole idea.

The tool

One command, no install, no account:

npx kavaca

npx kavaca

It scans the current directory and prints a result in seconds. Point it somewhere specific or get machine-readable output:

npx kavaca ./my-app
npx kavaca --json        # for CI

npx kavaca ./my-app
npx kavaca --json        # for CI

Exit code is 0 when clean and 1 when it finds something, so it drops straight into a pipeline.

What it actually checks

Three deterministic, local detectors:

1. Exposed secrets. Committed live credentials — Stripe (sk_live_…, rk_live_…), OpenAI / Anthropic keys, AWS access key IDs, database URLs with embedded passwords, and Supabase service_role JWTs. The JWT check isn't just a pattern match — it base64-decodes the payload and only flags if "role": "service_role" is actually in there, which kills a lot of false positives.

2. Frontend exposure. Sensitive values sitting behind a public env prefix (NEXT_PUBLIC_*, VITE_*, REACT_APP_*, EXPO_PUBLIC_*), and service_role keys referenced from client-side code.

3. Open Supabase databases. When it detects Supabase, it parses your SQL migrations and flags every CREATE TABLE that never gets a matching ENABLE ROW LEVEL SECURITY.

It's deliberately quiet about false positives — .example files, process.env. references, and placeholders like your_key_here are ignored. A security tool that cries wolf gets uninstalled on the first run, so it biases hard toward precision.

Here's what a run looks like:

◎ Kavaca — pre-flight security check
  Scanning ./my-app  (local only — nothing leaves your machine)
  ✔ Secrets in code .............. 1 issue
  ✔ Frontend exposure ............ clear
  ✔ Supabase database ............ 1 issue
  ──────────────────────────────────────────────
  ⚠  2 issues found · score 50/100
  HIGH  Live Stripe key (lib/payments.ts:14)
        sk_live_••••••••1234 — visible to anyone with the repo
  HIGH  Supabase table "orders" has no RLS policy
        Anyone with your anon key may be able to read it

◎ Kavaca — pre-flight security check
  Scanning ./my-app  (local only — nothing leaves your machine)
  ✔ Secrets in code .............. 1 issue
  ✔ Frontend exposure ............ clear
  ✔ Supabase database ............ 1 issue
  ──────────────────────────────────────────────
  ⚠  2 issues found · score 50/100
  HIGH  Live Stripe key (lib/payments.ts:14)
        sk_live_••••••••1234 — visible to anyone with the repo
  HIGH  Supabase table "orders" has no RLS policy
        Anyone with your anon key may be able to read it

Evidence is always masked — it never prints (or uploads) a full secret.

Local-first, and I mean it

Here's the part that matters most for a security tool: the default run makes zero network requests. Your code, your file paths, your secrets — none of it leaves your machine. There's an optional --share-stats flag that sends anonymous aggregate counts only ({ toolVersion, issueCountsByCheck, totalFiles }), it's off by default, and the entire network surface is a single, clearly-gated function you can read in src/telemetry.ts.

That's also why it's open source (MIT). "Trust me, I'm a security tool" is a tough sell. "Read the code" is a better one.

Drop it into CI

name: Kavaca security check
on: [push, pull_request]
jobs:
  kavaca:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npx --yes kavaca . --json

name: Kavaca security check
on: [push, pull_request]
jobs:
  kavaca:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npx --yes kavaca . --json

Now every push gets a 10-second pre-flight check.

Being straight with you

This CLI checks 3 of 8 risk areas. It's the smoke alarm, not the full inspection. There's a hosted version at kavaca.io that adds auth, API, and dependency checks with plain-English fixes and paste-ready prompts — that's the thing I'm actually building a business around. But the CLI is genuinely free and standalone, and it'll catch the stuff that ends launches. Use just the CLI forever if you want; that's a completely valid choice.

I'd love your help

The detectors live in src/detectors/ and every regex is documented in src/patterns.ts. If you know a secret format I'm not catching, or you've hit a false positive, that's exactly the feedback I want — open an issue or send a PR with a fixture.

• ⭐ Repo: github.com/eabhvee/kavaca-cli
• Try it: npx kavaca

If you've shipped something with AI that has real users, do me a favor and run it before you read the next post in your feed. Worst case, it takes ten seconds and tells you you're clean. Best case, it catches the thing you really didn't want a stranger to find first.

What patterns should it check next? Let me know in the comments 👇