Executive summary

Over the past year, AI moved from "assistant" to "operator" in secure software delivery: model systems now routinely retrieve repository context, call tools, and propose or implement fixes via PRs — while leaving an auditable trail (traces, check runs, review comments) for humans to approve. This is enabled by agent-focused platforms (tool calling, remote MCP connectors), deterministic output controls (JSON Schema / structured outputs, confidence thresholds), and operational safeguards (prompt-attack detection, groundedness checks).

Key leadership takeaways:

  • Treat AI as a workflow accelerator for triage and remediation, not a replacement for SAST/SCA/DAST. Modern products are converging on "find → explain → propose fix → human approve" in PR-native flows (e.g., Copilot Autofix, Snyk Agent Fix, Semgrep Assistant).
  • Your "supply chain" now includes agent connectors and model artifacts. MCP servers and plugins are new dependency edges; AI models need provenance controls (signing) and inventory (ML-BOM/AIBOM).
  • Evaluation and observability are now first-class: agent traces can be graded; vendor tools expose usage/cost dashboards; mature pilots measure MTTR, false-positive reduction, and time-to-review.

What changed in AI models and platform capabilities

Tool use and agents become the default integration pattern

OpenAI's Responses API (introduced as an "agents" primitive) emphasizes built-in tools (web/file search, computer use), persistence for multi-step work, and support for remote MCP servers — turning an LLM into an orchestrator that can fetch context, run analysis, and produce structured outputs. Microsoft Security Copilot also formalized "agents" as extensibility units, including building agents via YAML and MCP tooling. Google's Gemini Code Assist agent mode similarly supports configuring MCP servers to extend abilities, with explicit caution that changes outside the IDE may not be undoable — an operational risk AppSec should gate carefully.

AppSec implication: You must threat-model the tool boundary (what an agent can touch) as carefully as the model. In practice, "least privilege for tools" becomes your new baseline control, aligned with OWASP's focus on agentic risks and MCP-specific vulnerabilities.

Long context and multimodal reduce "context gaps" in code security analysis

Gemini's long-context documentation highlights 1M+ token windows, enabling repo-scale reasoning without aggressive chunking, which can reduce the common AppSec failure mode where security reviewers (human or AI) lack cross-file or cross-module context. Long context does not eliminate the need for retrieval/grounding, but it changes the architecture trade-offs: many teams can shift from "RAG for everything" to "RAG for authoritative references and decisions."

Retrieval, embeddings, and "engineerable grounding"

OpenAI positions file search and retrieval as hosted capabilities built on vector stores, where files are chunked, embedded, and indexed — explicitly supporting semantic plus keyword search. Web search can return sourced citations, which is valuable for "verify before act" flows (e.g., detecting whether a dependency CVE is real/current). Microsoft supports groundedness detection in Azure AI Content Safety to assess whether an LLM response is supported by provided sources, reducing fabricated conclusions (a common issue in vulnerability explanations).

Fine-tuning and instruction tuning: more selective, more governance-heavy

OpenAI documents supervised fine-tuning as training on example inputs/outputs for reliability and efficiency; reinforcement fine-tuning adds programmable graders for expert tasks. In contrast, Google's Gemini API tuning documentation notes that API/AI Studio fine-tuning support became unavailable after a May 2025 model deprecation and is instead supported in Vertex AI, with plans to bring it back — useful context for vendor selection and rollout planning.

AppSec recommendation: for most AppSec workflows, exhaust RAG + structured outputs + evaluation before fine-tuning, because fine-tuning increases data handling/compliance burden and can "bake in" bad guidance if training data is noisy.

AI-driven AppSec and supply-chain tooling advances

OpenAI platform primitives that map well to AppSec

OpenAI's Responses API and Agents SDK emphasize building agentic systems with tool use and a "full trace of what happened." Trace grading formalizes scoring across an agent's end-to-end trace (tool calls + decisions), which is directly applicable to AppSec pilots (e.g., "did the agent correctly classify exploitability and propose a safe fix?"). Structured outputs enforce JSON Schema adherence, reducing "hallucinated fields" and enabling deterministic downstream automation (ticket creation, policy checks). OpenAI's "your data" guidance reiterates that API data is not used for training by default (unless you opt in), which is foundational for vendor risk reviews.

Google: agent mode + security platform integration

Google documents agent mode for Gemini Code Assist, including MCP server configuration and an explicit warning about the lack of undo for changes to external resources — important for tooling governance in engineering orgs. On the security platform side, Gemini in Security Command Center provides AI-generated case summaries and recommended next steps, which can be adapted to vulnerability and misconfiguration triage (even if your AppSec stack is not SCC-first).

Microsoft: agent ecosystem and prompt attack defenses

Microsoft Security Copilot centers on building and publishing agents, including an agent store and MCP tooling. It is provisioned via Security Compute Units (SCUs) with tracked consumption, which matters when leadership asks for a predictable pilot budget. For prompt injection and adversarial inputs, Prompt Shields (Azure AI Content Safety) is designed to detect and block user prompt attacks and document-embedded attacks before generation.

GitHub: PR-native "fix loops" and AI secret detection

GitHub Code Security and Secret Protection were split from GHAS starting April 1, 2025, making "pilot scoping" easier (e.g., start with secrets + push protection, then expand to code scanning + autofix). Copilot Autofix is documented as generating targeted fix recommendations for code scanning alerts, with a responsible-use guide for limitations. GitHub also documents AI-powered generic secret detection ("Copilot secret scanning") as an expansion of secret scanning to find unstructured secrets such as passwords. Security campaigns with Copilot Autofix reached GA (April 2025), positioning AI not just as a fix generator but as a backlog-reduction mechanism at scale.

Snyk and Semgrep: AI in the PR comment loop

Snyk's "Agent Fix in PRs" supports generating fix suggestions via @snyk /fix and applying them as a commit via @snyk /apply, making remediation "PR-native" and reducing developer context-switching. Semgrep Assistant provides AI triage and optional autofix suggestions, with configurable confidence thresholds for when to show generated fixes—an enterprise-friendly control for noise and trust calibration.

Claude Code Review and Claude Cowork

Claude Code Review (managed PR review service) is explicitly designed to run on GitHub PRs via a Claude GitHub App, posting inline comments but not approving/blocking PRs (so existing governance remains intact). It requires read/write permissions (contents, issues, pull requests) and supports per-repo triggers (once per PR, every push, or manual via @claude review). Cost is billed separately based on token usage; Anthropic documents average review cost of $15–$25, scaling with PR size/complexity and verification workload, with spend caps and analytics dashboards. It also supports repo-scoped policy via CLAUDE.md and review-only rules in REVIEW.md, which is a practical "policy-as-text" method for AppSec teams to encode mandatory checks (e.g., "new API endpoints must have integration tests," "don't leak internal details in error messages").

AppSec and supply-chain impact: Claude Code Review effectively adds a new automated security reviewer with repo write-level surface area (via GitHub App permissions). This increases the importance of least privilege, review trigger discipline ("every push" can be expensive and noisy), and auditability of AI comments as part of your secure SDLC evidence.

Claude Cowork (desktop agent) brings agentic capabilities to the Claude desktop app, explicitly enabling access to local folders and connectors, running in an isolated VM with network allowlisting. However, Anthropic states that for Team/Enterprise plans, Cowork activity is not captured in audit logs, compliance API, or data exports, and that it is not suitable for HIPAA, FedRAMP, or FSI regulated workloads.

AppSec and supply-chain impact: Cowork is a powerful "local-file agent" that can touch sensitive artifacts (keys, SBOMs, incident notes). The lack of centralized auditing in its current form means security leadership should treat Cowork as high-risk for regulated scopes and require explicit local-data handling policies. Anthropic's open-source "Claude Code Security Reviewer" GitHub Action explicitly warns it is not hardened against prompt injection and recommends requiring approval for external contributors so workflows run only after maintainer review — this is directly applicable to any agentic reviewer on untrusted PRs.

Where AI is improving AppSec and supply-chain tools

AI value is highest when it compresses human effort in repetitive cognition, but still relies on deterministic scanners and controls for truth.

SAST/Code scanning augmentation: AI shines at explaining complex findings and drafting safe fix diffs (Copilot Autofix) while the scanner still provides deterministic detection.

SCA, dependency review, and SBOM operations: Traditional dependency review remains deterministic (e.g., GitHub Dependency Review Action shows PR dependency diffs and can add enforcement). AI adds value by summarizing dependency change risk ("what changed, why it matters, exploitability assumptions") and generating remediation PRs, but you should gate merges on the scanner/policy results.

Secret detection: GitHub's AI-based generic secret detection addresses unstructured secrets in source code, expanding beyond pattern-only detection.

Fuzzing and robustness: OSS-Fuzz research shows LLM-assisted fuzz target generation where the harnesses are validated via compilation/execution and measured by coverage/crashes — an ideal pattern for safe AI use because the "truth signal" is objective.

AI supply chain security: OpenSSF Model Signing v1.0 addresses authenticity/integrity for ML models; CycloneDX ML-BOM and OWASP AIBOM initiatives address transparency and inventory for AI systems.

Practical workflows for AppSec teams

The most robust pattern is "AI as a structured, tool-bounded analyst": it can explain and propose actions, but it must output machine-readable plans and operate under least privilege.

Adoption checklist for AppSec teams

  • Define automation levels: summarize-only, suggest-fix, create-PR, and never-auto-merge; enforce with branch protections and required reviews.
  • Require structured outputs for every automated step (finding_id, evidence_refs, risk_reasoning, proposed_fix, tests_added, confidence).
  • Ground decisions with retrieval over internal standards and past decisions (RAG/file search), not model memory.
  • Create a "review policy file" pattern: use repo-local instruction files (e.g., REVIEW.md) to encode security review expectations so agent reviewers flag the right things consistently.
  • Add "threat model deltas" as a deliverable: for each security PR, generate a small, reviewable model update (new trust boundaries, data flows, authz assumptions) and store it with the change.

Risks and mitigations for AI-augmented AppSec and AI supply chains

MCP and agentic systems expand the attack surface. OWASP's LLM Top Ten categories map directly to enterprise AppSec risk: sensitive information disclosure, supply chain risks, data/model poisoning, and excessive agency (excess functionality/permissions/autonomy). OWASP's MCP Top Ten adds connector-specific risks (token mismanagement, tool poisoning) once agents can access SCM, CI, and secrets.

For broader program structure, CoSAI's Workstream on software supply chain security for AI systems publishes a v1.0 risks-and-controls document that can be mapped to AppSec governance (data provenance, model integration, and infrastructure controls).

Mitigation checklist aligned to AppSec controls

  • Data leakage: Adopt vendor data controls and "no training by default" assumptions explicitly in contracts and configurations; avoid sending secrets or proprietary code unless your policy allows it.
  • Prompt injection / tool poisoning: Use Prompt Shields (or similar) to detect prompt/document attacks before generation; treat retrieved text as attacker-controlled; isolate tools by scope.
  • Untrusted PR execution: Require maintainer approval for workflows on external contributors (agentic reviewers should not run automatically on hostile inputs).
  • Explainability and audit: Prefer platforms that provide trace logs and evaluable artifacts (agent traces, check runs, inline comments, confidence scores).
  • AI supply chain integrity: Sign model artifacts (OpenSSF Model Signing v1.0), and inventory model/data/tool dependencies via ML-BOM/AIBOM for incident response and compliance.
  • Cowork governance: Because Cowork activity is not captured in audit logs/compliance export and is not suitable for certain regulated workloads, treat it as opt-in/disabled-by-default for regulated teams or sensitive data environments until auditing matures.

Treat AI adoption as an engineering experiment with stop conditions; pricing is usually metered (per token, per seat/premium request, or per compute unit). OpenAI publishes per-token API pricing; Microsoft Security Copilot is provisioned in Security Compute Units (SCUs); GitHub Copilot is per-seat with premium request overages.