RAG vs GraphRAG vs ADK/A2A/MCP: Architecture Choices for Agentic Systems

Your LLM is a brilliant liar and your users can tell

You deployed RAG. Documents went into Pinecone. Queries hit the vector store, retrieved relevant chunks, fed them to the model. It worked beautifully for the first month.

Then someone asked a question that required connecting information across three documents. The model retrieved one document, missed the other two, and confidently answered with partial information. Your support team started getting complaints: “Your AI said X, but our docs clearly say Y.”

The problem isn’t the model. The problem is the retrieval strategy doesn’t match the reasoning your data requires.

Three architectures, three different problems they solve

1. RAG (Retrieval-Augmented Generation): The baseline

Problem it solves: Model doesn’t know your specific data.

How it works:

1. Chunk your documents (512 tokens each, typically)
2. Embed chunks into vectors
3. At query time, search for similar vectors
4. Return top-K chunks to the model as context
5. Model answers with chunks in context

Strengths:

• Simple, proven, works at scale
• Fast (vector search is O(log N))
• Requires minimal infrastructure
• Easy to update (add new documents, re-embed, done)

Weaknesses:

• Loses structure (document chunks are tree structures, relationships, hierarchies)
• Vector search finds “similar” not “relevant”
• Struggles with reasoning across multiple documents
• Hard to handle multi-hop questions (“What’s the ROI of product X for customers in Y region?”)

When to use: Document QA, FAQ automation, knowledge base search

2. GraphRAG: Structure matters

Problem it solves: Your data has relationships that matter for reasoning.

How it works:

1. Extract entities and relationships from documents (product → customer, feature → bug)
2. Build a knowledge graph (nodes = entities, edges = relationships)
3. At query time, search the graph for relevant subgraphs
4. Pass the subgraph to the model as structured context
5. Model reasons over the structure

Why graphs are better for certain problems:

• Preserves relationships (not just “similar chunks”)
• Explicit reasoning (graph paths show why something is true)
• Multi-hop queries work (follow paths: product → customer → region)
• Updates are cheaper (change a relationship, don’t re-embed everything)

Example:

Traditional RAG query: "What customers have churn risk?"
Result: [Chunk about customer X, chunk about churn metrics, ...]
Problem: Chunks don't explicitly show which customers

GraphRAG query: "What customers have churn risk?"
Result: Graph showing [Customer A] --(risk_score: 0.8)--[ChurnRisk]
        [Customer B] --(risk_score: 0.6)--[ChurnRisk]
        [Customer A] --(product)--[PremiumTier]
Benefit: Model sees explicit relationships, can reason about patterns

Strengths:

• Captures domain structure
• Handles multi-hop reasoning
• Transparent (you can trace why the model said something)
• Scales to billions of relationships (Neo4j, etc.)

Weaknesses:

• More complex to set up (need entity extraction, relationship parsing)
• Graph extraction can be lossy (some information doesn’t fit the schema)
• Still requires an LLM to interpret the graph (not all questions are graph queries)
• Requires domain expertise to design the schema

When to use: Complex domains with known relationships (finance, healthcare, supply chains)

3. Agents with Tool Protocols (A2A, MCP, ADK): Reasoning and action

Problem it solves: LLM alone can’t answer; needs to search, compute, or call APIs.

How it works:

1. Model decides what tool to use (database, calculator, email, Slack)
2. Model provides tool parameters
3. System executes the tool
4. Model sees the result, decides next action
5. Repeat until answer is complete

Example workflow:

User: "Which customers have been inactive for 6 months?"
Model thinks: I need to query the database
Model calls: tool_query_crm(inactive_days=180)
System executes: Returns list of customers
Model thinks: I need to enrich this with product info
Model calls: tool_get_customer_products(customer_ids=[...])
System executes: Returns product data
Model thinks: I can answer now
Model returns: "5 customers have been inactive..."

Protocols:

• A2A (Agent-to-Agent): Agents coordinate with each other (one handles customer data, one handles billing)
• MCP (Model Context Protocol): Standardized protocol for tools (Claude, tools via MCP)
• ADK (Agentic Development Kit): Framework for building agent behaviors

Strengths:

• Handles reasoning + action (not just retrieval)
• Can compute on the fly (database queries are fresh)
• Transparent action logs (see exactly what it did)
• Works with existing systems (APIs, databases)

Weaknesses:

• Unpredictable execution (model might call wrong tool)
• Slow (multiple tool calls = multiple round trips)
• Expensive (each tool call is an LLM API call)
• Requires tool implementations

When to use: Automating complex workflows, cross-system coordination

How to choose

Real-world example: E-commerce support

Customer asks: “I bought product A on March 1st. When does my warranty expire?”

RAG approach:

1. Search for “warranty” + “product A”
2. Get chunk: “Warranty is 1 year from purchase”
3. Answer: “Your warranty expires in March 2027”
4. Problem: Didn’t verify it’s actually the customer’s product

GraphRAG approach:

1. Query graph: [Customer] –(purchased)–[ProductA] –(warranty_length)–[12_months]
2. Extract purchase date from graph
3. Answer: “Your warranty expires in March 2027”
4. Benefit: Verified it’s their actual product

Agent approach:

1. Model: “I need to look up the customer’s orders”
2. Calls: get_customer_orders(customer_id)
3. Model: “I found product A purchased 2026-03-01, warranty is 12 months”
4. Calls: calculate_expiry(2026-03-01, 12_months)
5. Answer: “Your warranty expires March 1, 2027”
6. Benefit: Works with real-time data, auditable

Combining architectures

The best systems use all three:

1. Agent decides what to do (“I need warranty info”)
2. Graph finds related entities (warranty policies, product info)
3. RAG retrieves the actual policy text from documents
4. Model combines all context into an answer

This is called “Retrieval-Augmented Generation with Agentic Reasoning” and it’s becoming the standard for complex domains.

The bridge post

GraphRAG is the bridge between this post and the next ones in the series. Here’s why:

• Post 3 (local stack): You can build a small knowledge graph locally with Ollama
• Post 4 (this one): You understand why knowledge graphs are better than vectors for reasoning
• Post 5 (enterprise): You’re ready to deploy this at scale in production systems

The takeaway

RAG is the foundation. GraphRAG adds structure. Agents add reasoning and action.

For your next system, start with RAG (fastest to implement). Then ask: “Does my data have relationships that matter?” If yes, layer in graphs. Then ask: “Does my system need to take action?” If yes, add agents.

Most teams get this wrong by assuming RAG is enough and wondering why their LLM gets confused. It’s not confused; it’s just not seeing the relationships that matter.

Next in this series: GenAI in the Enterprise SaaS Ecosystem: A Staff Engineer’s Integration Playbook