Post cover image

May 11, 2026

Building AI Agents Part 3C: Why Your Framework Choice Will Make or Break Your Production System

Why the framework that worked in your prototype will stall your production system

By Raj kumar

10 min read

The fintech team I mentioned in Part 3B (Testing and Evaluation Strategies for Production AI Agents) had built something impressive. Their agent worked. Testing passed. Stakeholders were happy. Then they tried to extend it.

Adding a new compliance check took three days. Debugging a state management issue required four engineers across two time zones. Onboarding a new developer meant two weeks of documentation reading before they could touch the codebase. The agent had not broken. But the system had become brittle in a way that only reveals itself under the weight of real development velocity.

The root cause was not the model, the prompts, or the tooling. It was the framework. They had chosen based on what was trending at the time of their prototype, not what their team could sustain in production.

Framework selection is the decision that most teams make too quickly and feel too slowly.

This is part of the series Building Production AI Agents: A Complete Architecture Guide, where we walk through an 9-step framework to take agents from concept to deployment, with practical patterns and examples across banking, healthcare, retail, manufacturing, and beyond.

In this part, I want to change how you think about frameworks — away from feature lists and toward operational consequences.

The Four Questions That Actually Matter

Before opening a single documentation page, answer these four questions about your system.

  • Who owns the production system in six months? If your answer is a small ML team with strong Python skills, your framework options differ significantly from a cross-functional team where non-engineers will manage workflows. The best framework is the one your actual team can debug at 2am.
  • What is the cost of a wrong decision? A fraud detection agent in banking carries materially different failure consequences than a content generation agent for marketing. Higher stakes demand frameworks with explicit state management, audit trails, and human-in-the-loop support built in — not bolted on.
  • Where does your complexity actually live? Some systems are complex at the retrieval layer. Others at the orchestration layer. Others at the integration layer. Many teams choose a powerful orchestration framework when their real problem is a RAG quality issue. Identify where your system is genuinely hard before selecting a framework to solve it.
  • What does "scale" mean for your use case? Scale in AI agents rarely means handling more concurrent users. It usually means handling more complex workflows, more data sources, more edge cases, and more regulatory requirements. A framework that scales elegantly along your actual growth dimension is worth far more than one optimised for raw throughput you will never hit.

With those questions answered, you can evaluate the four categories of frameworks with sharper criteria.

Category One: Consumer AI Agents

ChatGPT, Claude, and Perplexity occupy this space. They offer pre-built capability, polished interfaces, and near-zero setup time. Teams reach for them instinctively because they are immediately useful.

The hard limit of consumer agents is the boundary between your data and their infrastructure. You cannot integrate them with proprietary systems in any meaningful production sense. You cannot customise the decision logic. You cannot audit what happened when something goes wrong.

Where they genuinely belong: individual productivity, general research, content brainstorming, and exploration of a problem space before you build a solution. A compliance analyst using Claude to summarise regulatory documents is appropriate. A bank routing actual compliance decisions through Claude's consumer API is not.

The most common mistake I see is teams prototyping successfully with a consumer agent and then attempting to productionise that same setup. The prototype worked because the constraints of production — data privacy, audit requirements, integration depth, volume — were not in play. Consumer agents are excellent at revealing what your agent should do. They are poor foundations for systems that actually do it.

Category Two: Agentic Coding Tools

Cursor, Windsurf, and Claude Code represent a different category entirely. These are tools for building agents, not agents themselves. They belong in your development workflow, not your production architecture.

The distinction matters because teams sometimes conflate the two. The coding tool helps you write the system. The framework is the system.

Where agentic coding tools provide genuine leverage: when you are implementing custom agent logic, building bespoke integrations with internal systems, or maintaining a codebase that has grown beyond what standard IDE tooling handles cleanly. A fintech team building a custom fraud detection pipeline should be using Cursor or Windsurf to write that pipeline, not using them as the pipeline itself.

Windsurf's cascade model is particularly strong for large codebases where changes in one module have non-obvious implications in another. Claude Code fits naturally into terminal-native workflows and DevOps automation. Cursor's multi-file composer handles the cross-cutting changes that dominate most agent implementation work.

The choice between them is almost entirely a function of your team's existing workflow, not the technical superiority of any one tool.

Category Three: No-Code and Low-Code Builders

Lindy, Relay, and n8n serve a legitimate and often underserved purpose: connecting systems that already exist, without writing the glue code yourself.

The architectural insight here is that many "AI agent" use cases are actually integration use cases with an LLM added. If your workflow is fundamentally about routing information between systems — emails triggering CRM updates, approvals triggering notifications, sensor data triggering maintenance tickets — a no-code builder may be the correct answer even for a technically sophisticated team.

n8n is worth particular attention because it supports self-hosting, which resolves the data privacy objection that disqualifies most no-code tools for regulated industries. A manufacturing team routing IoT sensor data through quality control and into their ERP can do that on self-hosted n8n without their production data touching any third-party cloud.

The failure mode of no-code tools is almost always the same: teams use them for workflows that contain genuine complexity, then find themselves fighting the abstraction when they need to handle an edge case the platform did not anticipate. The rule I apply is straightforward. If you can describe the entire workflow in a linear sentence — "when X happens, do Y, then notify Z" — a no-code builder is appropriate. If your description requires "unless" or "depending on" more than twice, you need a development framework.

Category Four: Development Frameworks

LangGraph, CrewAI, and LlamaIndex are where production agent systems for complex use cases are actually built. Each solves a different problem, and understanding that distinction is essential before choosing.

LangGraph: When Control Flow Is Your Problem

LangGraph is fundamentally a state management framework with a graph-based workflow engine on top. It excels when your agent needs to handle conditional branching, maintain state across multiple steps, support human approval gates, and produce auditable decision trails.

In banking and financial services, these requirements appear constantly. A transaction investigation workflow is not a linear sequence. It branches based on risk score. It pauses for human review. It persists state across what might be hours of real-world clock time while waiting for a compliance officer. It needs to log every decision node for regulatory examination.

LangGraph handles all of this natively. The tradeoff is that it requires your team to think in graph abstractions, which has a real learning curve, particularly for engineers who are comfortable with sequential code but less familiar with explicit state machines.

from langgraph.graph import StateGraph, END
from typing import TypedDict

class ComplianceInvestigationState(TypedDict):
    transaction_id: str
    risk_score: float
    regulatory_flags: list[str]
    decision: str
    audit_log: list[dict]
def assess_risk(state: ComplianceInvestigationState) -> ComplianceInvestigationState:
    # Risk scoring against transaction patterns and counterparty data
    score = calculate_transaction_risk(state["transaction_id"])
    state["risk_score"] = score
    state["audit_log"].append({"step": "risk_assessment", "score": score})
    return state
def check_regulatory_flags(state: ComplianceInvestigationState) -> ComplianceInvestigationState:
    # AML, sanctions screening, PEP checks
    flags = run_regulatory_checks(state["transaction_id"])
    state["regulatory_flags"] = flags
    state["audit_log"].append({"step": "regulatory_check", "flags": flags})
    return state
def route_decision(state: ComplianceInvestigationState) -> str:
    # Conditional routing based on risk and regulatory outcome
    if state["risk_score"] > 0.85 or state["regulatory_flags"]:
        return "escalate_to_human"
    elif state["risk_score"] > 0.6:
        return "enhanced_review"
    else:
        return "approve"
workflow = StateGraph(ComplianceInvestigationState)
workflow.add_node("assess_risk", assess_risk)
workflow.add_node("check_regulatory_flags", check_regulatory_flags)
workflow.add_node("escalate_to_human", human_review_node)
workflow.add_node("enhanced_review", enhanced_review_node)
workflow.add_node("approve", approve_node)
workflow.add_edge("assess_risk", "check_regulatory_flags")
workflow.add_conditional_edges("check_regulatory_flags", route_decision)
workflow.set_entry_point("assess_risk")
agent = workflow.compile(checkpointer=SqliteSaver.from_conn_string("compliance.db"))

Notice what this code actually does beyond the obvious. The checkpointer means state persists to a database, so the investigation survives a system restart or network partition. The conditional routing makes the decision logic inspectable — a compliance officer can read the route_decision function and understand exactly why a transaction was escalated. This is not incidental. In regulated environments, explainability of routing logic is often a regulatory requirement, not a nice-to-have.

LlamaIndex: When Knowledge Retrieval Is Your Problem

LlamaIndex is frequently described as a RAG framework, which undersells it. It is more accurately a framework for building knowledge-intensive applications where the quality and structure of retrieval directly determines answer quality.

In banking, this covers a wide territory: regulatory document question answering, policy lookup, historical case analysis, technical documentation for internal tools. The pattern is common enough that most financial institutions should have at least one LlamaIndex-powered system even if they have no other agentic workloads.

from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.llms import Anthropic
from llama_index.embeddings import HuggingFaceEmbedding

# Load regulatory corpus: Basel III, MiFID II, local prudential standards
documents = SimpleDirectoryReader(
    'regulatory_documents',
    required_exts=['.pdf', '.txt']
).load_data()
service_context = ServiceContext.from_defaults(
    llm=Anthropic(model="claude-3-5-sonnet-20241022"),
    embed_model=HuggingFaceEmbedding(model_name="BAAI/bge-large-en-v1.5"),
    chunk_size=512,
    chunk_overlap=64
)
index = VectorStoreIndex.from_documents(
    documents,
    service_context=service_context
)
query_engine = index.as_query_engine(
    similarity_top_k=6,
    response_mode="tree_summarize",
    node_postprocessors=[SentenceTransformerRerank(top_n=3)]
)
response = query_engine.query(
    "What are the capital adequacy requirements for exposures to central counterparties "
    "under Basel III, and how do they interact with our current tier-1 capital ratio?"
)
# Response includes source citations - critical for compliance validation
print(response.source_nodes)

The reranking step here is not decorative. In regulatory RAG, the difference between returning the most semantically similar chunk and the most relevant chunk can be the difference between correct guidance and dangerously incomplete guidance. LlamaIndex makes that distinction tractable.

CrewAI: When Specialisation and Collaboration Are Your Problem

CrewAI addresses a different architectural pattern: workflows where a single agent with broad capability is less effective than multiple agents with narrow, deep specialisation working in coordination.

This pattern appears when you need multiple perspectives on the same problem, when different parts of your workflow require meaningfully different expertise, or when the cognitive load of a single-agent approach causes quality degradation on complex tasks.

A practical example from aviation maintenance: a system reviewing a fleet maintenance decision might benefit from one agent with deep access to maintenance records and regulatory requirements, another with knowledge of parts availability and lead times, and a third focused on operational impact and scheduling. A single agent context-switching between these concerns will do each of them less well than three focused agents whose outputs are synthesised.

from crewai import Agent, Task, Crew, Process

maintenance_engineer = Agent(
    role="Aircraft Maintenance Engineer",
    goal="Assess technical feasibility and regulatory compliance of the maintenance action",
    backstory="20 years of line maintenance experience, deep familiarity with EASA Part-145",
    tools=[maintenance_manual_search, airworthiness_directive_checker, defect_history_lookup],
    llm="claude-3-5-sonnet-20241022",
    max_iter=3
)
logistics_coordinator = Agent(
    role="Parts and Logistics Coordinator",
    goal="Determine parts availability, sourcing options, and realistic lead times",
    backstory="Specialist in AOG situations and alternative supplier qualification",
    tools=[parts_inventory_search, supplier_database, lead_time_estimator],
    llm="claude-3-5-sonnet-20241022",
    max_iter=3
)
operations_planner = Agent(
    role="Operations Control Manager",
    goal="Evaluate operational impact and recommend schedule recovery options",
    backstory="Experienced in fleet management and disruption recovery planning",
    tools=[schedule_impact_calculator, aircraft_availability_checker],
    llm="claude-3-5-sonnet-20241022",
    max_iter=3
)
maintenance_assessment = Task(
    description="Assess technical requirements and regulatory compliance for {defect_description} on {aircraft_registration}",
    agent=maintenance_engineer,
    expected_output="Technical assessment with regulatory references and recommended rectification approach"
)
logistics_assessment = Task(
    description="Determine parts and tooling requirements based on the technical assessment",
    agent=logistics_coordinator,
    context=[maintenance_assessment],
    expected_output="Parts availability, sourcing options, and confirmed lead times"
)
operational_impact = Task(
    description="Calculate schedule impact and recommend recovery options given technical and logistics constraints",
    agent=operations_planner,
    context=[maintenance_assessment, logistics_assessment],
    expected_output="Go/no-go recommendation with schedule recovery plan"
)
maintenance_crew = Crew(
    agents=[maintenance_engineer, logistics_coordinator, operations_planner],
    tasks=[maintenance_assessment, logistics_assessment, operational_impact],
    process=Process.sequential,
    verbose=True
)
result = maintenance_crew.kickoff(inputs={
    "defect_description": "hydraulic leak on main gear actuator",
    "aircraft_registration": "VT-AXC"
})

The context parameter in the later tasks is doing significant work here. The logistics coordinator does not need to re-derive the technical requirements — they receive the maintenance engineer's structured output directly. This mirrors how expert human teams actually operate, and it reduces both token cost and error propagation.

The Multi-Framework Principle

Real production systems rarely map cleanly onto a single framework. The useful mental model is to identify the dominant problem in each layer of your architecture and select the framework that solves that specific problem most cleanly.

A banking compliance system might reasonably combine: LangGraph for the investigation workflow that requires conditional routing and state persistence; LlamaIndex for regulatory document retrieval that feeds into that workflow; custom Python for the quantitative risk scoring models where you need full control over the mathematical logic; and n8n for the downstream integration layer that updates case management systems and sends notifications.

Each framework is doing what it is actually good at. None is being stretched beyond its design intent.

The failure mode to avoid is framework sprawl for its own sake. Every additional framework in your architecture is an additional cognitive load for your team, an additional failure surface to monitor, and an additional abstraction boundary where debugging becomes harder. Add frameworks only when the benefit is concrete and the alternative would require building something non-trivial yourself.

A Decision Framework, Not a Decision Matrix

I want to offer a way of thinking about this rather than a table that pretends the decision is simpler than it is.

Start with your team's operational capability. A framework that requires graph-theory intuition from your entire engineering team is only as useful as your team's ability to maintain it under pressure. Be honest about this.

Then match the framework's architectural strength to your system's dominant complexity. LangGraph for workflow complexity. LlamaIndex for knowledge complexity. CrewAI for collaboration complexity. No-code builders for integration complexity that lacks custom logic.

Then consider the failure mode. Every framework fails in a predictable way. LangGraph state bugs are subtle and hard to reproduce. LlamaIndex retrieval failures are quality issues that surface gradually rather than catastrophically. CrewAI multi-agent coordination can produce inconsistent outputs when agent roles overlap. Know your framework's failure mode before you are in production experiencing it.

Finally, validate against your compliance and security requirements before writing any production code. In banking and aviation, framework selection is not purely a technical decision. Data residency requirements, audit logging capabilities, and on-premise deployment support can eliminate large parts of the framework landscape before you have evaluated a single feature.

Closing: Frameworks Serve Architecture, Not the Other Way Around

The fintech team eventually rebuilt their system. They kept the model selection, the prompt architecture, and the tool integrations. They replaced the orchestration layer with LangGraph, which gave them the explicit state management and conditional routing they needed but had been trying to approximate with complex Python control flow.

Development velocity recovered. Debugging became tractable. New team members could read the graph definition and understand the system's decision logic without having to reverse-engineer it from code.

The lesson was not that LangGraph is universally correct. It was that they finally matched the framework to the problem they actually had, rather than the problem they thought they had when they started.

Choose frameworks that serve your system design and your team's capability. Those two constraints, taken together, will narrow the decision considerably. What remains is engineering judgment, and that is something no framework can substitute for.

Interested in building production-ready AI agents? Explore the Agentic AI Engineering: The Complete Production Systems Collection, a continuously updated resource covering AI agents, LangGraph, LangChain, CrewAI, RAG architectures, memory systems, MCP, orchestration patterns, observability, deployment strategies, and enterprise AI engineering best practices. Whether you're designing your first agent or scaling autonomous systems in production, this collection brings together practical guides, real-world architectures, and implementation-focused insights in one place.

If this was useful, I would genuinely appreciate a clap or a comment sharing which framework decisions you have struggled with in your own systems. And follow along for the next parts in the series.