Depwire SLM v0.2 — A Model That Reasons Over Graph Facts, Not Source Code

Every major code model — GPT-4 Codex, Qwen-Coder, DeepSeek-Coder, CodeLlama — was trained on raw source files scraped from GitHub.

They learned what code looks like. Syntax. Patterns. Tokens.

None of them learned what code means.

Depwire SLM v0.2 is built on a different premise. It was trained on dependency graphs — the actual structural relationships between symbols, files, and modules in real codebases. And every answer it produces is verified deterministically against the same graph engine that computed the facts.

Here's how it works and what the numbers actually mean.

The problem every developer knows

You ask your AI assistant what breaks if you change a function. It gives you five paragraphs of hedging. "It depends on how the function is used… here are some things to consider…"

This is not a capability problem. It is an information problem.

The model has no map. It reads your files linearly, makes educated guesses about what connects to what, and produces confident answers based on incomplete context.

Meanwhile, depwire-cli already knows the answer. It has the full dependency graph. It knows that authenticate() has 23 callers, that 7 of them have no test coverage, that 3 are at depth 1 and 4 are at depth 2. It computed this deterministically from the actual code structure.

The question is: how do you get a model to reason over that graph data accurately — without hallucinating numbers that aren't there?

The architecture: facts-in / reasoning-out

The answer is to never ask the model to produce graph facts from its weights.

Instead:

depwire-cli (ground truth)
        │
        ▼
Facts-IN prompt ──► SLM reasons ──► Typed JSON answer
        │                                    │
        └──── oracle_eval.py verifies ───────┘

depwire-cli (ground truth)
        │
        ▼
Facts-IN prompt ──► SLM reasons ──► Typed JSON answer
        │                                    │
        └──── oracle_eval.py verifies ───────┘

The model receives real graph facts in the prompt — computed by depwire-cli, not guessed. Its job is to reason over them: which files to review first, what the blast radius means for this team's workflow, what the safe change sequence is.

Every factual claim the model makes is then checked against the same depwire-cli output that produced the facts. If the model mentions a file that isn't in the graph — automatic fail. If it claims a severity level that doesn't match what depwire found — automatic fail.

The model cannot hallucinate facts it was never asked to produce.

The training pipeline

Building this required a different approach to training data.

Rather than generating pairs from templates, we used teacher distillation with oracle filtering:

Run depwire-cli on a real repository — get ground truth graph facts
Build a facts-in prompt with those exact facts
Send to DeepSeek as the teacher model — generate structured reasoning
Run oracle_eval.py on the teacher's answer — verify every claim against the real graph
If the teacher contradicts the graph in any way — discard the example
Only oracle-passing examples enter the training set

This means the training set is, by construction, 100% grounded. Every completion was verified against real graph output before being included.

Total: 5,234 oracle-passing pairs from 1,532 repositories across three task types.

Impact analysis  ·  2,860 pairs  ·  54.6%
Dead code        ·  1,439 pairs  ·  27.5%
Security         ·    935 pairs  ·  17.9%
──────────────────────────────────────────
Total            ·  5,234 pairs

Impact analysis  ·  2,860 pairs  ·  54.6%
Dead code        ·  1,439 pairs  ·  27.5%
Security         ·    935 pairs  ·  17.9%
──────────────────────────────────────────
Total            ·  5,234 pairs

Training cost: $0.68. Runtime: approximately 90 minutes on an M1 Ultra 128 GB with no cloud GPU.

The oracle harness

The evaluation methodology matters as much as the training data.

oracle_eval.py is a deterministic harness that:

Takes a held-out repository — never seen during training
Runs depwire-cli live to get real graph facts
Builds a facts-in prompt with those facts
Runs the adapter
Extracts the structured JSON answer
Compares every claim field against live depwire output

The checks are exact. For impact: claimed files must be a subset of the real affected files. For security: claimed severity must match the exact enum depwire found. For dead code: claimed confidence must match after case normalization.

Any contradiction is an automatic fail. No fuzzy matching. No LLM as judge. depwire-cli is the judge.

This harness is reusable and open source under Apache 2.0. Any model — Claude, GPT-4, anything — can be run through it. If you want to know whether your AI coding assistant hallucinates graph facts about your actual codebase, the harness gives you a deterministic answer.

The results

131 evaluation cases across 40 repositories strictly held out from training. Every case: real depwire facts in the prompt, model answer verified against live depwire output.

Impact analysis  ·  98.4%  (62/63 cases)
Security         ·  90.0%  (27/30 cases)
Dead code        ·  94.7%  (36/38 cases)
─────────────────────────────────────────
Overall          ·  95.4%  (125/131 cases)

Impact analysis  ·  98.4%  (62/63 cases)
Security         ·  90.0%  (27/30 cases)
Dead code        ·  94.7%  (36/38 cases)
─────────────────────────────────────────
Overall          ·  95.4%  (125/131 cases)

Critical hallucinations: 0 Phantom vulnerabilities: 0 Oracle errors: 0

The 7 remaining failures are all format-level — unparseable JSON on unusually complex repos that exceed the token limit, and minor enum casing issues. None are reasoning failures where the model invented graph facts or contradicted the oracle on substance.

Why this is different from other code models

The differentiation is not the benchmark score. It is the architecture.

Other code models are scored by LLM-as-judge (expensive, fuzzy, inconsistent) or keyword matching (gameable, not meaningful). Depwire SLM is scored by depwire-cli — a deterministic program that either agrees or disagrees with the model's claims about a real codebase.

You cannot build this eval harness without a graph engine. That is the moat. Competitors can fine-tune larger models on more data. They cannot replicate a deterministic ground-truth verifier for code architecture claims without building the graph engine first.

The eval harness is also the product. Once it exists, any model can be benchmarked on your specific codebase. That becomes a standard. That is what v0.3 is building toward.

What's next

The immediate priority is MCP integration.

When this is complete: you ask about impact analysis in Claude Desktop, Goose, or any MCP client — depwire runs automatically, facts go into the SLM prompt, the model reasons, the oracle verifies. The full pipeline runs end-to-end without you building the prompt manually.

That is when this becomes useful to developers who are not building it.

Beyond that: scale to 10,000 oracle-filtered pairs, add the change-safety task (is this specific code change safe to merge?), push grounding above 97%, and publish the benchmark as a standard others can run against their models.

Try it

The adapter is live on HuggingFace. The full pipeline is open source.

Adapter: huggingface.co/atefataya/depwire-slm Pipeline: github.com/atef-ataya/depwire-slm Graph engine: depwire.dev

To run it locally you need depwire-cli installed, an M1 Mac with 64 GB RAM or more, and the base model downloaded. The README has the exact commands.

If you find a case where the model hallucinates a graph fact — I want to know. That is a real failure and the oracle harness will catch it.

Atef Ataya is a Lead Software Architect and the creator of Depwire. He publishes technical content at youtube.com/@atefataya.