Post cover image

October 2, 2025

Building an AI Agent for Codebase Analysis and Understanding

Several months ago, I inherited a 50,000-line Ruby codebase with zero documentation. The previous team had vanished, leaving behind a maze…

By Zogoo

8 min read

Several months ago, I inherited a 50,000-line Ruby codebase with zero documentation. The previous team had vanished, leaving behind a maze of interconnected services, custom gems, and business logic scattered across dozens of files. Sound familiar?

Instead of spending weeks deciphering the code manually, I decided to build a prototype: an AI agent that could read, understand, and explain codebases. Not a simple ChatGPT wrapper, but a system that grasps code structure, relationships, and context.

The prototype worked better than expected. It significantly reduced the time needed to understand the codebase structure and locate relevant functionality.

Here's what I built and the lessons learned that you can apply to your own projects.

The Real Problem With Code Understanding

We've all been there. You're staring at a function that calls three other functions, which import from four different modules, which depend on two external services. Your brain starts building a mental map, but it's fragile. One interruption and you're back to square one.

Traditional code search helps with "where is this function defined?" but fails at "how does this system actually work?" We need something that understands intent, not just syntax.

What if you could ask:

  • "How does user authentication flow through the system?"

  • "What happens when an API request fails?"

  • "Where would I add logging to track user behavior?"

This isn't about replacing developers. It's about augmenting our ability to understand complex systems quickly.

How It Actually Works

Think of it like building a smart librarian for your code:

  1. The Scanner reads your entire codebase and breaks it into meaningful chunks (not random text blocks, but actual functions, classes, and modules)

  2. The Embedding Engine converts each chunk into a mathematical representation that captures its meaning

  3. The Vector Database stores these representations so we can find similar code efficiently

  4. The Query Interface takes your question, finds the most relevant code, and asks GPT to explain it

The key technique is semantic search. Instead of looking for exact keyword matches, we find code that's conceptually related to your question. Ask about "user login" and it will find authentication middleware, session handling, and password validation — even if those exact words never appear.

What I tried to build

  • Ingest any codebase (I'll use Ruby, but the concepts work for any language)

  • Answer natural language questions about your code

  • Run entirely locally (no code leaves your machine)

  • Handle codebases up to 100k+ lines efficiently

Step 1: Smart Code Chunking (The Foundation)

Here's where most attempts fail: they treat code like regular text and split it arbitrarily. But code has structure. A function belongs together. A class definition needs its methods. Context matters.

The key insight: use Abstract Syntax Trees (ASTs) to split code at meaningful boundaries.

class CodebaseScanner
  def initialize(root_dir)
    @root_dir = Pathname.new(root_dir)
    @results = []
  end

  def scan
    scan_directory(@root_dir)
    @results
  end

  private

  def process_file(file)
    content = File.read(file, encoding: "UTF-8")

    if content.size <= 1000
      # Small files: keep entire content
      chunk = create_chunk(file.relative_path_from(@root_dir), content)
      @results << chunk
    else
      # Large files: split by class/method boundaries
      chunks = split_by_ast_nodes(file, content)
      @results.concat(chunks)
    end
  end
end

This approach solves three critical problems:

  1. Semantic boundaries: Functions stay together, classes keep their methods, modules maintain their structure

  2. Context preservation: Each chunk knows what file it came from, what type of code it is (controller, model, config), and its role in the system

  3. Size optimization: Small files (under 1000 chars) stay whole, large files split intelligently at natural boundaries

The metadata is very important. When someone asks about "database queries," we can prioritize chunks tagged as "model" or "migration" over generic utility functions.

Why ASTs matter: Traditional text splitting might break a function in half or separate a class from its methods. AST parsing ensures each chunk is a complete, meaningful unit of code.

Here's how the recursive parsing works in practice, using the actual create_chunk method:

def create_chunk(file_path, content)
  # Estimate tokens (rough approximation: 1 token ≈ 4 chars)
  estimated_tokens = content.length / 4

  if estimated_tokens > 8000
    # Parse the content and split by blocks/methods
    ast = Prism.parse(content)
    return [fallback_chunk] unless ast

    chunks = []
    current_chunk = []
    current_tokens = 0

    # Recursive function to extract meaningful nodes
    def extract_nodes(node)
      nodes = []
      case node
      when Prism::DefNode, Prism::ClassNode, Prism::ModuleNode, Prism::CallNode
        nodes << node
      when Prism::StatementsNode
        # Recurse into container nodes
        node.body.each { |n| nodes.concat(extract_nodes(n)) }
      end
      nodes
    end

    # Extract all relevant nodes first
    nodes = extract_nodes(ast.value.statements)

    # Process nodes into chunks, respecting token limits
    nodes.each do |node|
      source = node.location.slice  # Get the actual source code
      tokens = source.length / 4

      if tokens > 8000
        # If a single block is too large, truncate it
        source = source[0..(8000 * 4)]
      end

      if current_tokens + tokens > 8000
        # Current chunk is full, start a new one
        chunks << create_chunk(file_path, current_chunk.join("\n")) if current_chunk.any?
        current_chunk = [source]
        current_tokens = tokens
      else
        # Add to current chunk
        current_chunk << source
        current_tokens += tokens
      end
    end

    # Don't forget the last chunk
    chunks << create_chunk(file_path, current_chunk.join("\n")) if current_chunk.any?
    return chunks
  end

  # For small files, return as single chunk with metadata
  {
    id: generate_id(file_path),
    context: content,
    path: file_path,
    metadata: {
      type: determine_file_type(file_path),
      gem: determine_gem(file_path),
      layer: determine_layer(file_path)
    }
  }
end

This approach handles the real-world complexity:

  1. Token-aware chunking: Keeps chunks under 8000 tokens (GPT's context window)

  2. Recursive extraction: Walks through nested structures automatically

  3. Smart batching: Groups related code together until hitting size limits

  4. Metadata preservation: Each chunk knows its file type, gem, and architectural layer

  5. Fallback handling: Gracefully handles unparseable code

When processing a large Rails controller, this might extract:

  • Class definition as one chunk

  • Each method as separate chunks

  • Complex methods split at logical boundaries

  • All tagged with metadata like "controller", "api layer", "business_logic gem"

The result: instead of arbitrary text blocks, you get semantically meaningful code units that the AI can actually understand and explain.

Step 2: Turning Code Into Math (Embeddings Explained)

The next step is converting code into numerical vectors that capture semantic meaning.

Think of embeddings like this: similar code gets similar numbers. A login function and an authentication middleware will have vectors that are mathematically close, even if they use completely different variable names.

I built two approaches — one for speed and convenience, one for privacy and cost control:

Option 1: OpenAI Embeddings (Cloud)

def generate_embeddings(chunks)
  client = OpenAI::Client.new(access_token: ENV['OPENAI_API_KEY'])

  chunks.each do |chunk|
    response = client.embeddings(
      parameters: {
        model: 'text-embedding-3-small',
        input: chunk['context']
      }
    )
    chunk['embedding'] = response.dig('data', 0, 'embedding')
  end
end

Option 2: Local E5 Embeddings (Offline)

from sentence_transformers import SentenceTransformer

def generate_local_embeddings(chunks):
    model = SentenceTransformer('intfloat/e5-large-v2')

    for chunk in chunks:
        # E5 models expect "query:" or "passage:" prefixes
        text = f"passage: {chunk['context']}"
        embedding = model.encode(text, normalize_embeddings=True)
        chunk['embedding'] = embedding.tolist()

The tradeoff: OpenAI embeddings are higher quality and easier to set up, but cost money and send your code to their servers. E5 embeddings run entirely on your machine — no API keys, no data leaving your system, no ongoing costs.

For most projects, the quality difference is negligible. I actually prefer the local approach for anything containing proprietary code.

Step 3: Fast Similarity Search (Enter Vector Databases)

Now we have thousands of code chunks, each represented as a vector of 1,536 numbers. When someone asks a question, we need to find the most similar vectors in milliseconds, not minutes.

Vector databases are designed for this use case. I chose Qdrant because it's performant, reliable, and runs in a single Docker container.

def store_embeddings(chunks, collection_name)
  client = Qdrant::Client.new(url: "http://localhost:6333")

  # Create collection
  client.collections.create(
    collection_name: collection_name,
    vectors: {
      size: 1536, # OpenAI embedding size
      distance: "Cosine"
    }
  )

  # Batch upload for efficiency
  chunks.each_slice(100) do |batch|
    points = batch.map do |chunk|
      {
        id: generate_id(chunk["path"]),
        vector: chunk["embedding"],
        payload: {
          context: chunk["context"],
          path: chunk["path"],
          type: chunk["metadata"]["type"]
        }
      }
    end

    client.points.upsert(
      collection_name: collection_name,
      points: points
    )
  end
end

Qdrant handles the vector similarity calculations efficiently. You can query for the 5 most similar chunks and get results in under 50ms, even with 10,000+ code chunks.

Pro tip: Batch uploads are essential. Uploading chunks one by one takes forever. Batching 100 at a time reduces upload time from hours to minutes.

Step 4: Bringing It All Together (The Query Interface)

Here's where the system becomes truly useful. A user asks a natural language question, and we return a comprehensive answer with relevant code examples.

def answer_question(query)
  # Generate embedding for the question
  query_embedding = generate_query_embedding(query)

  # Search for similar code
  results = qdrant_client.points.search(
    collection_name: "code_chunks",
    vector: query_embedding,
    limit: 5,
    with_payload: true
  )

  # Build context from search results
  context = results["result"].map do |result|
    "## File: #{result["payload"]["path"]}\n#{result["payload"]["context"]}"
  end.join("\n\n")

  # Ask GPT to explain
  response = openai_client.chat(
    parameters: {
      model: "gpt-4-turbo",
      messages: [
        {
          role: "system",
          content: "You are a code analysis assistant. Explain the following code based on the user's question."
        },
        {
          role: "user",
          content: "Question: #{query}\n\nRelevant Code:\n#{context}"
        }
      ]
    }
  )

  response.dig("choices", 0, "message", "content")
end

The process follows these steps:

  1. Question → Vector: Convert the user's question into the same embedding space as our code

  2. Vector → Code: Find the most semantically similar code chunks

  3. Code → Context: Bundle the relevant code with metadata and file paths

  4. Context → Answer: Let GPT analyze the code and provide a human explanation

The key insight: we're not asking GPT to understand the entire codebase. We're giving it just the relevant pieces and asking it to explain those specific chunks.

Prototype Results

I tested this prototype on the 20,000-line Ruby codebase that motivated this project. Here are the results:

Query: "How does user authentication work?"

  • Found: Authentication middleware, session handling, JWT token validation, and password hashing logic across 4 different files

  • Response time: 2.1 seconds

  • Accuracy: Found all relevant components, including edge cases I had forgotten about.

Query: "What happens when an API request fails?"

  • Found: Error handling in service layer, retry logic, notification system, and database rollback procedures

  • Response time: 1.8 seconds

  • Accurac*: Comprehensive coverage of the entire failure recovery flow.

Query: "Where would I add logging for user actions?"

  • Found: Existing logging patterns, audit trail implementation, and suggested integration points

  • Response time: 1.6 seconds

  • Accuracy: Not just where to add logging, but how to follow existing patterns.

What I Learned Building This

Key Insights

AST-based chunking was critical. I initially tried simple text splitting and got poor results. Functions were split in half, and classes were separated from their methods. AST parsing ensures every chunk is semantically complete.

Local embeddings performed better than expected. While OpenAI embeddings have higher quality, E5 performed nearly as well for code understanding. The privacy and cost benefits make it worth considering for most use cases.

Metadata is crucial. Just having the code isn't enough. Knowing whether a chunk is from a controller, model, or config file helps prioritize results dramatically.

Challenges and Limitations

Cross-file relationships are difficult to capture. The prototype understands individual files well but struggles with how they connect. If a controller calls a service, it doesn't automatically understand that relationship.

Context windows matter. GPT-4 can handle a lot of code, but there's still a limit. For complex queries spanning many files, the prototype required careful context summarization and prioritization.

Error handling needs improvement. When the prototype fails, it often fails silently. Users ask a question, get no results, and assume the system is broken. This area needs significant work for a production system.

Implementation Guide

Here's how to implement a similar prototype:

Prerequisites

  • Ruby 3+ (for the scanner)

  • Python 3.8+ (for local embeddings)

  • Docker (for Qdrant)

  • OpenAI API key (optional, for cloud embeddings)

Quick Start

  1. Start Qdrant:
docker run -p 6333:6333 qdrant/qdrant
  1. Scan your codebase:
# Implement the scanner based on the examples above
./scan_codebase.rb /path/to/your/code
  1. Generate embeddings:
# Local approach (no API key needed)
./embed_by_e5.py
# OR cloud approach (requires OPENAI_API_KEY)
./embed.rb
  1. Store in Qdrant:
./store.rb
  1. Start asking questions:
./ask.rb "How does authentication work?"

Cost Breakdown

  • Local setup: $0 (just your compute)

  • Cloud embeddings: ~$5–10 for a 50k line codebase

  • Ongoing queries: ~$0.01–0.05 per question

The entire prototype runs on a laptop. No cloud infrastructure required.

Future Directions

This prototype demonstrates the potential for AI-assisted code understanding. Several areas warrant further exploration for a production system:

Immediate Extensions

  • Code review assistance: "What could go wrong with this change?"

  • Architecture documentation: "Generate a system overview for new team members"

  • Refactoring guidance: "How would you improve this code?"

  • Bug hunting: "Find potential security issues in the request handling flow"

Long-term Possibilities

Future development environments could support:

  • Natural language explanations for any function or module

  • Architectural guidance for feature placement

  • Legacy code analysis and documentation generation

  • Accelerated onboarding for new team members

Why This Matters

We're moving from code search to code understanding. Instead of hunting for where something is defined, we can ask what it does, why it exists, and how it fits into the bigger picture.

The tools we build today will determine how the next generation of developers learns and works with code.

What's Your Take?

Building this prototype was a weekend experiment that turned into something genuinely useful. The combination of AST parsing, embeddings, and vector search works surprisingly well for code understanding, but I'm sure there are better approaches I haven't considered.

If you've worked on similar problems, I'd love to hear about your experiences:

  • Have you tried other ways to chunk code for AI analysis?

  • What embedding models work best for your programming language?

  • How do you handle cross-file relationships and dependencies?

  • What would make a tool like this actually useful in your daily workflow?

The hardest part wasn't the technical implementation — it was figuring out what questions developers actually want to ask about their codebases. What would you want to ask yours?

Code examples are simplified for clarity and demonstrate the core concepts.