Post cover image
What each agent sends to a model, and what stays in your region.

October 20, 2026

03 — Running the SDLC factory

Know where data goes, prove what happened, pick models by measurement, test behaviour, ship every change through CI/CD.

By Oleksandr Derechei

8 min read

The SDLC factory, a series in four parts. 01 — SDLC factory · 02 — Scaling the SDLC factory · 03 — Running the SDLC factory · 04 — Adopting the SDLC factory

Part 1 showed the line: Signals, then Discover, Validate and Shape, then Approve ticket, then Design, Build, Document and Verify, then Release, To production, Operate and Learn. One agent per step, a router in code, gates without a model, a typed handoff. Part 2 designed it to scale: a consistent run state, a core behind ports, stateless workers, bounded parallel work, memory, and a feedback loop that turns every session into a tested change to the agents.

That loop means the agents change often: a prompt this week, a model next month. This part is about what makes that safe, and about the day a security reviewer, a lawyer or a finance lead asks plain questions. Where does our code go? Who did what? Why this model? How do you know it still works? How does a change reach production?

None of these is new. Every production system answers them. The answers are old practices applied to a new kind of worker.

Where the data lives

When an agent runs, what leaves your network? Each agent sends a model the input of its step:

  • Signals to Shape: ticket text, comments, user feedback. Sometimes customer names, because people paste them.
  • Design to Verify: source code, diffs, test output, the approved acceptance criteria.
  • Release to Learn: build logs, deploy output, alerts. Logs are the riskiest input, because they hold things nobody meant to write down.
None

Four things decide if that is acceptable. Vendor terms change, so check each fact against the linked source before you rely on it.

Where the model runs. The access path decides.

  • On the Anthropic API you choose global or us per request or workspace, on Claude 4.6 and later. US-only costs 1.1 times the price. Workspace data at rest is US only. There is no EU option on the first-party API yet.
  • On Amazon Bedrock, content stays in the Region you call, but cross-region inference can move a request. Geographic profiles stay inside a geography. Global profiles route anywhere, for about ten percent less. When a rule names a geography, choose the geographic profile.
  • Vertex AI and Microsoft Foundry offer regional options. EU options on both are unconfirmed: ask before you plan on them.

How long the provider keeps it.

  • Default: inputs and outputs are deleted within 30 days, except content flagged for policy violations (up to 2 years). Commercial data is not used for training without express permission.
  • Zero data retention (ZDR): nothing stored at rest. You get it per organisation, through sales. It covers the Messages and Token Counting APIs and Claude Code with commercial API keys. It excludes the Console, batch jobs, files, code execution, Managed Agents, and the Fable and Mythos models (30-day retention unless authorized).
  • Claude Code: it sends every file it reads and keeps local transcripts 30 days. Telemetry and error reporting are off by default on Bedrock, Vertex and Foundry. Claude Code is not covered by HIPAA readiness.

On Bedrock, retention modes are set per Region. The Fable models need AWS review retention of up to 30 days, and a Service Control Policy can force none.

One Bedrock trap: invocation logging is off by default. When on, it stores full prompts and outputs in your own storage. Those logs are as sensitive as the code.

What you keep yourself. Two things, kept apart. The run record is one line per agent run: tokens, time, turns, outcome, and no text. The event log is the full transcript of every session, which the feedback loop needs to replay a dialog. Treat it as sensitive as the code: it stays in your own region, secrets are scrubbed before it is written, access is logged, and it is deleted when its retention ends. Store only what the loop uses, because every line you keep is a line you must protect.

What the vendor can show. Security teams ask for reports. Anthropic's Trust Center lists SOC 2, ISO 27001, ISO 42001, a HIPAA agreement and a data processing agreement. Bedrock lists SOC, ISO 27001 and others, HIPAA eligibility and FedRAMP Moderate. Request the reports; do not quote a logo.

Try it: pick one agent and list every field it sends to a model. Mark each one that could hold a customer name or a secret.

Compliance and security: prove what happened

An audit trail per run. For every action record who started it, which agent and version, which tool, which target, and the outcome. The run record carries the agent as name@version, the role, the model, the turns and the status. Agent commits carry the ticket key. When someone asks what changed a file and why, the answer is a ticket, a run line and a commit, and the event log holds the full session behind them.

None

Separation of duties. No agent approves its own work or releases to production. Every pull request needs one review by a second Developer before a person merges it; when the work is risky, that review becomes the Approve PR checkpoint and the run waits for it. Branch protection and the pipeline approval live outside the agent, so no agent can switch them off. Staging is automatic. To production is always a person.

Secrets stay out of the context. Deny reads of environment files and keys. A check scans new content for secrets. Tools get short-lived, scoped tokens held by a proxy, so a secret never enters the model's view and a tricked agent cannot leak it. The same scrubbing runs before anything is written to the event log.

Scrub the eval data. Test cases come from real tickets and incidents. Remove names, keys and customer text before they enter the repository.

Now the hard part. No prompt is a security boundary. Prompt injection is not solved. The best published result for a top model in browser use is about one percent attack success against an adaptive attacker. Lower, not zero.

The risks with the agents: private data, untrusted content, and a way to send data out. With all three in one session, data can be stolen. The agents can read tickets and pull request comments, touch private repositories, and can push and call the network. For risky operations agent must have human approval and supervision.

The wall is what the agent cannot do: least privilege per step, branch protection, typed handoffs. Each agent also runs in a sandbox with a network allowlist and its own identity, set by managed settings nobody can loosen. Model-based checks add depth, never the only barrier.

Picking the model per agent

  • Design and Verify: Opus. Hard reasoning where a wrong answer is costly to find later.
  • Signals and Release: Haiku. Small, bounded, frequent work: sorting the incoming signals, preparing a release.
  • All other agents: Sonnet. The default.
None

As in Part 1, Verify runs on a different model than Build, because a model leans towards approving its own work. The table is a starting point; your evals decide.

Never pick from leaderboards. They measure other tasks, and public test sets leak into training data. Your cases are the only test that matters.

A fallback is a model change. A fallback that has not passed the same evals is a silent drop in quality.

Then the boring parts. Pin full model IDs, not aliases, so a change is one you made. Models are retired with notice, so track your pins and run the model-change evals early.

Last, cost per finished task, not per call: a cheap run that fails more often is not a saving. The cache helps. Ten calls on the same large prefix cost about a fifth of the uncached price.

Evaluating agents

None

Start with error analysis. Pull fifty to a hundred real sessions from the event log, failures first, and read each one. Write down the first thing that went wrong in each, group the notes into failure types, and turn each type into cases. A score says that something failed. Only reading says why.

Code graders first. Grade the end state: did the handoff validate, did the test pass. Use a judge model for prose only, after checking it against people. Ask for pass, fail or unknown, never a one-to-five scale. Pin the judge's model, prompt and examples.

Small samples are noisy. Treat a first drop as a warning, rerun those cases, and block only if it holds.

Read the path. On a failure, find the first wrong step and label its cause: tool, arguments, reasoning or a bad tool result. Later errors usually follow from it.

CI/CD makes the difference

With agents there are two pipelines.

The product pipeline checks the agents' code. Build writes the change, the checks run, Verify reads the result. A red check sends the work back to Build, at most three rounds, then a person takes over. One rule is new: an agent must not be able to change the checks it has to pass. As in Part 1, an agent asked to make a test pass sometimes edits the test.

The agent pipeline checks the agents. A prompt, a skill, a tool or a model is code, so it changes by pull request. A check without a model runs first. Then evals run the new version and the old one on the same cases. A person reads the result and merges. Every merge is a new version, and the run record shows which version ran.

Failures connect the two. A pull request a person rejected, a run that used all three rounds, an incident: each one becomes a case in the agent pipeline. The next version of the agent must pass it.

None

Two rules:

  • One change per pull request: a prompt, a tool or a model. Change all three and you cannot tell which one broke it.
  • Run a small set on a schedule. A hosted model can change with no commit from you.

Must have with agents

  1. Write down what each agent sends to a model, get a legal decision per repository, and check the vendor facts on regions, retention and ZDR.
  2. Keep a run record per agent run, and an event log per session. The record has no text. The log has the transcript: in your region, scrubbed of secrets, with logged access and a retention limit.
  3. Keep approvals outside the agent: branch protection, and a pipeline approval for production.
  4. Keep secrets out of the context, and give each session at most two legs of the trifecta.
  5. Pick a model per agent from your own cases. Verify on a different model than Build. Pin full IDs. Make every downgrade and fallback clear the same bar.
  6. Ship every change through CI/CD, change one thing at a time, and turn every incident into a case.

That is the factory for one project, what if you hae many projects? Part 4 is about adoption of the same factory on in different ways.

Previous: 02 — Scaling the SDLC factory · Next: 04 — Adopting the SDLC factory · Start of the series: 01 — SDLC factory

Sources