Sketching Agentic AI: A Coffee Table Discussion

I recently found myself discussing Agentic AI with a few friends over coffee. What started as a casual conversation quickly moved beyond the buzzwords and into the engineering principles behind the technology. As the discussion evolved, I realized many of us were asking the same questions: Why do we need Agentic AI? What problem is it actually solving? Is it simply another buzzword, or does it represent a genuine shift in software architecture?

This article is my attempt to capture those thoughts — not as the definitive answer, but as a way of thinking about Agentic AI from first principles.

On LinkedIn, Medium, and technology blogs, Agentic AI has become one of the most talked-about — and often misunderstood — topics in modern software engineering.

The underlying technology is real and genuinely transformative. However, the surrounding narrative often overstates what today's (and tomorrow's ) systems can realistically achieve. It is easy to come away with the impression that AI agents are autonomous digital employees capable of replacing complex business workflows with little or no human oversight.

Production systems tell a different story:

Most enterprise AI applications are built around well-defined workflows where an LLM is just one component of a larger deterministic system. These systems combine business rules, APIs, retrieval mechanisms, validation layers, observability, security controls, and, when necessary, human approval. Their objective is not unlimited autonomy, but reliable task completion within clearly defined operational boundaries.

This leads to an important realization: Agentic AI is not a product — it is an architectural pattern.

Agentic AI is not a product — it is an architectural pattern:

It is a way of designing intelligent software systems by combining the strengths of deterministic software with the reasoning capabilities of Large Language Models (LLMs). The goal is not simply to automate tasks, but to solve problems that traditional software struggles to solve effectively.

Traditional software is remarkably good at deterministic problems. If the rules are known, the inputs are well-defined, and the expected outcome is predictable, conventional programming remains the most reliable, efficient, and cost-effective solution.

However, many real-world problems do not fit this model.

Understanding natural language, interpreting documents, reasoning over incomplete information, prioritizing actions, generating explanations, or deciding the next best step are inherently non-deterministic. There is often no single algorithm that guarantees the correct answer every time because the problem itself contains ambiguity.

Historically, software engineers tried to address these challenges by building increasingly sophisticated rule engines and decision trees. While these approaches work well within carefully defined scenarios, they eventually become difficult to maintain, brittle in the face of change, and often produce mediocre results when confronted with the complexity and variability of the real world.

This is where Agentic AI introduces a fundamentally different architectural approach.

Rather than forcing deterministic software to solve non-deterministic problems, it delegates those responsibilities to reasoning-capable LLMs. The LLM becomes responsible for interpreting information, reasoning over context, planning, or selecting the next action. Deterministic software continues to orchestrate workflows, enforce business rules, execute tools, validate outputs, maintain security, and ensure overall system reliability.

In other words, each component is used for what it does best.

The objective is not to eliminate uncertainty — no AI system can do that. Instead, the objective is to manage uncertainty within acceptable error boundaries. By combining reasoning, retrieval, tool use, validation, feedback loops, and human oversight where appropriate, Agentic AI systems can produce solutions that are both practical and dependable.

Perhaps the most important takeaway is this:

Agentic AI is not about replacing software with AI. It is about placing AI where deterministic software reaches its limits.

Once you view Agentic AI through this architectural lens, the conversation shifts away from frameworks, hype, and marketing terminology. Instead, it becomes a discussion about system design — carefully partitioning responsibilities between deterministic engineering and probabilistic reasoning so that each can contribute where it is most effective.

Frameworks will evolve. Models will improve. New orchestration libraries will emerge.

But this architectural principle is likely to remain the foundation on which production-grade Agentic AI systems are built.

#AgenticAI #AIEngineering #SoftwareArchitecture #LLM #ArtificialIntelligence #SystemDesign #GenerativeAI #SoftwareEngineering