EN
FrançaisEnglishDeutschPortuguêsEspañolTiếng Việt한국어简体中文日本語
Home / AI & Machine Learning / MCP

MCP vs RAG vs AI Agent: Detailed Explanation with Examples

Cris The Dev · 10/05/2026 · 9 min read · 145
This article has a short video:YouTubeTikTok

1. Overview

Three names MCP, RAG, AI Agent appear everywhere when talking about LLM applications, and it’s easy to mistake them as “three competing options”. In reality they solve three different problems: RAG helps LLM answer correctly based on private data, AI Agent helps LLM plan and act independently, while MCP is a communication standard so LLM apps connect to tools/data in a unified way. After reading this post, you’ll distinguish the three concepts clearly, know when to use which, and have several runnable Java/Spring AI code snippets to grasp the reality.

2. What is RAG

RAG (Retrieval-Augmented Generation) is a technique to “embed” external data into the prompt before calling LLM, so the model answers based on real data instead of just relying on knowledge learned during training.

2.1. RAG Workflow

  1. Convert source text (policy, documents, FAQ) into embeddings (numeric vectors) and store in a vector database.
  2. When user asks, convert the question into embedding, find the closest text snippets (similarity search).
  3. Put found snippets into the prompt, then send to LLM to generate the answer.

Example: OrderService needs to answer “what is the return policy for online orders?” based on company policy documents, not the model’s general knowledge.

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-vector-store-pgvector</artifactId>
</dependency>

2.2. Simple RAG Code Example

@Service
public class ReturnPolicyAssistant {

    private final VectorStore vectorStore;
    private final ChatClient chatClient;

    public ReturnPolicyAssistant(VectorStore vectorStore, ChatClient.Builder builder) {
        this.vectorStore = vectorStore;
        this.chatClient = builder.build();
    }

    public String answer(String question) {
        List<Document> relevant = vectorStore.similaritySearch(
                SearchRequest.query(question).withTopK(3));

        String context = relevant.stream()
                .map(Document::getContent)
                .collect(Collectors.joining("\n---\n"));

        String prompt = """
                Based on the policy documents below, answer the customer's question.
                If there is no information, say "I'm not sure".

                Documents:
                %s

                Question: %s
                """.formatted(context, question);

        return chatClient.prompt(prompt).call().content();
    }
}
  • vectorStore.similaritySearch returns the 3 document snippets most semantically similar to the question.
  • The prompt is “augmented” with context retrieved from the vector store before sending to LLM.
  • If documents don’t contain relevant information, the model is instructed to answer “I’m not sure” instead of hallucinating (making things up).
Input:  "I bought a shirt online 5 days ago, can I exchange it?"
Output: "According to our policy, you can exchange within 7 days of receiving your order,
         as long as the product still has tags. Your shirt is still within the exchange period."

Note: RAG only helps the model “know more data”. It doesn’t help the model take action (call APIs, update databases). If you need the model to do something instead of just answer, you need an Agent.

3. What is AI Agent

AI Agent is a system using LLM as the “brain” to plan, choose appropriate tools, call them, observe results, then repeat until reaching the goal — different from RAG which only stops at answering one question.

3.1. The Reason → Act → Observe Loop

The most common pattern is ReAct: the model thinks (reasoning), chooses a tool to call (action), receives the result (observation), then thinks again based on that result.

public interface AgentTool {
    String name();
    String description();
    String execute(String input);
}

@Component
public class CheckOrderStatusTool implements AgentTool {
    private final OrderRepository orderRepository;

    public CheckOrderStatusTool(OrderRepository orderRepository) {
        this.orderRepository = orderRepository;
    }

    public String name() { return "checkOrderStatus"; }
    public String description() { return "Check order status by orderId"; }

    public String execute(String orderId) {
        return orderRepository.findById(orderId)
                .map(o -> "Order %s is %s".formatted(orderId, o.getStatus()))
                .orElse("Order not found " + orderId);
    }
}

3.2. Agent Tool Selection and Invocation

@Service
public class SupportAgent {

    private final ChatClient chatClient;
    private final List<AgentTool> tools;

    public SupportAgent(ChatClient.Builder builder, List<AgentTool> tools) {
        this.chatClient = builder.build();
        this.tools = tools;
    }

    public String handle(String userMessage) {
        return chatClient.prompt()
                .user(userMessage)
                .tools(tools.toArray()) // Spring AI automatically maps execute() method to function-calling
                .call()
                .content();
    }
}
  • tools(tools.toArray()) tells the model it can call these tools; Spring AI automatically generates schema for function-calling.
  • The model will decide on its own: if the question is about order status, it calls checkOrderStatus with orderId inferred from the question, receives the result, then explains it back in natural language.
User: "What's the status of my order ORD-2024-891?"
Agent: (calls checkOrderStatus("ORD-2024-891"))
       → "Order ORD-2024-891 is being delivered, expected to arrive in 2 more days."

3.3. Testing Tool Selection Behavior

@Test
void shouldCallCheckOrderStatusWhenAskingAboutOrder() {
    AgentTool tool = new CheckOrderStatusTool(orderRepository);
    when(orderRepository.findById("ORD-1")).thenReturn(Optional.of(shippedOrder));

    String result = tool.execute("ORD-1");

    assertThat(result)
        .contains("ORD-1")
        .contains("SHIPPED");
}

This test doesn’t call real LLM, just verifies the tool logic works correctly — the part “which tool the model chooses” should be tested separately with an evaluation set, not through rigid unit tests because LLM output is non-deterministic.

Note: Agent is powerful but higher risk than RAG — if a tool has write permissions (refund, update), one wrong reasoning from the model could have real consequences. Always add a confirmation step or limit permissions for sensitive tools.

4. What is MCP

MCP (Model Context Protocol) is an open standard defining how an LLM application (called host) connects to external data sources and tools (called server) through a unified protocol, instead of each integration writing its own code.

4.1. Problem MCP Solves

Before MCP, if you had 5 LLM apps and wanted each to use 10 data sources/tools (database, CRM, file system…), you had to write up to 50 separate integrations — the familiar N×M problem. MCP turns it into N+M: each server is written once, every host following the MCP standard can call it.

4.2. MCP Architecture

  • Host: application containing LLM (chatbot, IDE assistant…).
  • Client: component within host, maintains 1-1 connection with one MCP server.
  • Server: process exposing 3 types of “capabilities”:
    • Resources: readable data (like GET endpoint).
    • Tools: callable actions (like POST/action).
    • Prompts: reusable prompt templates.

4.3. Example: Writing an MCP Server in Java

<dependency>
    <groupId>io.modelcontextprotocol.sdk</groupId>
    <artifactId>mcp</artifactId>
    <version>0.8.0</version>
</dependency>
public class OrderMcpServer {

    public static void main(String[] args) {
        McpSyncServer server = McpServer.sync(StdioServerTransport.builder().build())
                .serverInfo("order-service", "1.0.0")
                .tool(orderLookupTool())
                .build();

        server.start();
    }

    private static McpServerFeatures.SyncToolSpecification orderLookupTool() {
        var schema = """
                {
                  "type": "object",
                  "properties": { "orderId": { "type": "string" } },
                  "required": ["orderId"]
                }
                """;

        return new McpServerFeatures.SyncToolSpecification(
                new Tool("getOrderById", "Look up order by orderId", schema),
                (exchange, args) -> {
                    String orderId = (String) args.get("orderId");
                    String status = OrderLookup.findStatus(orderId);
                    return new CallToolResult(List.of(new TextContent(status)), false);
                });
    }
}
  • McpServer.sync(...) builds a server using stdio transport (host runs server as subprocess and communicates via stdin/stdout).
  • tool(orderLookupTool()) registers a tool getOrderById with JSON schema describing input, so any host following MCP standard can read “what fields does this tool need”.
  • When client calls getOrderById, server returns CallToolResult containing text content.

4.4. Calling MCP Server from Spring AI Client

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-mcp-client</artifactId>
</dependency>
spring:
  ai:
    mcp:
      client:
        stdio:
          connections:
            order-service:
              command: java
              args: ["-jar", "order-mcp-server.jar"]
@Service
public class McpAgentBridge {

    private final SyncMcpToolCallbackProvider toolCallbackProvider;
    private final ChatClient chatClient;

    public McpAgentBridge(SyncMcpToolCallbackProvider toolCallbackProvider,
                           ChatClient.Builder builder) {
        this.toolCallbackProvider = toolCallbackProvider;
        this.chatClient = builder.build();
    }

    public String ask(String question) {
        return chatClient.prompt()
                .user(question)
                .tools(toolCallbackProvider.getToolCallbacks())
                .call()
                .content();
    }
}
  • Spring AI automatically starts MCP server (order-mcp-server.jar) as subprocess, maintains connection via stdio.
  • toolCallbackProvider.getToolCallbacks() retrieves all tools the server exposes (here: getOrderById) and feeds them to model as function-calling, exactly like manually registering AgentTool in section 3 — just the difference is this tool is provided by an independent process following a common standard.

Note: MCP doesn’t replace function-calling in LLM. It’s a transport/standardization layer below function-calling, helping tools be written once and reused across many hosts (Claude Desktop, IDE, your app) without changing integration code each time.

5. Comparing MCP, RAG, and AI Agent

Criterion RAG AI Agent MCP
Solves what problem Model lacks private/fresh data Model can’t act independently, can’t plan multi-step Tool/data integration code duplicates, not standardized
Unit of operation Retrieve + augment prompt Reason-act-observe loop Protocol connecting host ↔ server
Calls tools/actions No (read-only) Yes Yes (expose tools for agent to call)
Risk level Low (only wrong answers if retrieval is poor) High (may execute wrong actions) Depends on underlying tools, protocol itself is neutral
Relationship to each other Is a “data source” Agent can use Can use RAG as a tool, use MCP to call standardized tools Is the “pipeline” for Agent to call RAG/tools in unified way

In short: RAG gives model knowledge, Agent gives model ability to act, MCP is the communication standard so that acting/retrieving doesn’t need rewriting each time you change host or tool.

6. Combining All Three in a Real-World System

In reality, large projects don’t choose one of the three, but stack them on top of each other:

User → AI Agent (planning, ReAct loop)
          ├─ calls MCP Client → MCP Server "knowledge-base" → returns RAG similarity search result
          ├─ calls MCP Client → MCP Server "order-service"  → tool getOrderById
          └─ calls MCP Client → MCP Server "payment-service" → tool refundPayment (needs confirmation)

In this architecture:

  • RAG sits inside a separate MCP server (e.g., server “knowledge-base” exposes resource as similarity search result).
  • Agent is the orchestrator running the reasoning loop, doesn’t care how tools are written, only cares tools follow MCP.
  • MCP is the unified communication layer between Agent and all servers, helping you add new servers (e.g., “shipping-service”) without modifying Agent code.

7. Common Mistakes

  • Using RAG when action is needed: stuffing data into prompt but expecting model to update database — RAG can’t do this, you need Agent + tool with write permissions.
  • Agent without tool permission limits: letting agent directly call tools like deleteOrder or refundPayment without human confirmation step, leading to damage when model reasons wrongly.
  • Mistaking MCP for an “AI framework”: MCP is only a connection protocol, doesn’t think on its own, doesn’t retrieve on its own. If your server doesn’t expose useful tools/resources, having MCP doesn’t make the model smarter.
  • Top-k too small or too large in RAG: small top-k (1-2) easily misses context, large top-k (10+) makes prompt long, costs more tokens and dilutes important signals. Start with 3-5 and evaluate with real eval set.
  • Not versioning MCP tool schema: changing tool input schema without bumping version, other hosts still using old server will call with wrong parameters.

8. Conclusion

RAG, AI Agent, and MCP don’t compete — they sit at three different layers of an LLM system: RAG provides data, Agent provides reasoning and action capability, MCP provides the connection standard so rewriting isn’t needed each time you change tool or host. When designing real-world AI systems, start with “what does the model need to know” (RAG), “what does the model need to do” (Agent), then think about “how can this tool/data be reused in many places” (MCP) — this order helps avoid over-engineering from the start.

mcpragai-agentspring-aillm
App by the authorDeclutterClear similar photos, big videos and inbox clutteriOS · Android

More on this topic