Microsoft Ship AI Agent Framework: Detailed Explanation with Examples

1. Overview

Microsoft has officially shipped (GA – General Availability) a unified framework for building, orchestrating, and operating AI Agents by merging its two previous products: Semantic Kernel and AutoGen. This is more than just a product announcement — it reshapes how you should design a system where an LLM can “autonomously decide actions” instead of just answering a single question.

This article explains how AI Agents differ from regular chatbots, what Microsoft has specifically shipped, and then you’ll build a minimal agent yourself using Java 21 + Spring Boot 3 + Spring AI (integrated with Azure OpenAI) to understand the underlying mechanism, even though Microsoft’s original framework runs on .NET and Python.

2. What is an AI Agent?

A regular chatbot receives a question, calls the LLM once, and returns an answer. An Agent differs in that it repeats a reason-act loop multiple times, autonomously deciding whether it needs to call a tool (API, database, file…) or not, before providing the final answer.

Feature	Chatbot	AI Agent
Number of LLM calls	1 per question	Multiple, self-looping
Using external tools	Usually not	Yes, tool chosen by model
State	Stateless or chat history only	Has memory, can checkpoint
Latency	Low	Higher (multiple round-trips)
Reliability	Easy to control	Needs guardrails, human-in-the-loop

2.1. Reason–Act (ReAct) Loop

The core mechanism most agent frameworks use today is ReAct: the model “thinks” (Thought), chooses an action (Action) by calling a tool, receives the result (Observation), then repeats until it has enough information to answer (Final Answer).

Thought: User is asking about order ORD-001 status, I need to look it up.
Action: lookupOrder("ORD-001")
Observation: Order ORD-001: status SHIPPED, total amount 450000 VND
Thought: I have enough information.
Final Answer: Your order ORD-001 is being delivered, total 450,000đ.

This is the part that Microsoft Agent Framework (and Spring AI which you’ll use below) automates — you don’t write this loop manually, just declare tools and system prompt.

3. What Microsoft Has “Shipped” for AI Agent

Before the merger, Microsoft had two parallel development tracks:

• Semantic Kernel: production-focused SDK emphasizing type-safe plugins, enterprise integration (Azure, .NET), suitable for stable deployments.
• AutoGen: research-oriented, strong in multi-agent conversation — multiple agents “chat” with each other to solve complex tasks.

The new framework combines both strengths, including main capabilities:

• Built-in orchestration patterns: sequential (agent A completes, then B), concurrent (multiple agents run in parallel then aggregate), hand-off (agent transfers task to a specialized agent), group chat (multiple agents discuss before answering).
• Durable/stateful execution: agent can checkpoint mid-process, resume if process crashes — important for long-running tasks (minutes to hours).
• Built-in observability: tracing per OpenTelemetry standard, you can see each Thought/Action/Observation step in production.
• Human-in-the-loop: pause agent to wait for human approval before executing sensitive actions (refunds, data deletion…).
• Azure AI Foundry Agent Service: hosted runtime to deploy agents, comes with ready-made tool catalog (Bing search, code interpreter, file retrieval) and supports Model Context Protocol (MCP) to connect external tools by open standard, without vendor lock-in.

The original framework runs on .NET/Python, but the pattern is universal. The next section rebuilds that exact pattern in Java so you see how it works at the code level.

4. Core Architecture of an AI Agent

A minimal agent needs 4 components: Planner (essentially the LLM), Tool (functions the agent can call), Memory (chat history/tool results), Executor (orchestration loop). Spring AI packages Executor and Planner in ChatClient, you just declare Tools.

4.1. Maven Dependency

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-azure-openai-spring-boot-starter</artifactId>
    <version>1.0.0</version>
</dependency>

Configuration to connect to Azure OpenAI (Microsoft’s GPT service, deployed via Azure):

spring.ai.azure.openai.api-key=${AZURE_OPENAI_KEY}
spring.ai.azure.openai.endpoint=${AZURE_OPENAI_ENDPOINT}
spring.ai.azure.openai.chat.options.deployment-name=gpt-4o-mini

4.2. Define Tools for the Agent

A tool is a regular Java method, marked with @Tool so the model knows it can call it and understands what parameters to use.

public record Order(String id, String status, BigDecimal total) {}

public interface OrderService {
    Order findById(String orderId);
}

@Service
public class OrderTools {

    private final OrderService orderService;

    public OrderTools(OrderService orderService) {
        this.orderService = orderService;
    }

    @Tool(description = "Look up the status and total amount of an order by order ID")
    public String lookupOrder(String orderId) {
        Order order = orderService.findById(orderId);
        return "Order %s: status %s, total amount %s VND"
            .formatted(order.id(), order.status(), order.total());
    }
}

The description in @Tool is what the model reads to decide whether to call lookupOrder — write clear, specific descriptions, and the agent will choose the right tool more accurately.

4.3. Agent Calls Tools Automatically via ChatClient

@Service
public class SupportAgentService {

    private final ChatClient chatClient;

    public SupportAgentService(ChatClient.Builder builder, OrderTools orderTools) {
        this.chatClient = builder
            .defaultSystem("""
                You are a customer support assistant for an online shop.
                If the question relates to an order, use the lookupOrder tool.
                Answer briefly, in a friendly tone, in Vietnamese.
                """)
            .defaultTools(orderTools)
            .build();
    }

    public String handle(String userMessage) {
        return chatClient.prompt()
            .user(userMessage)
            .call()
            .content();
    }
}

The line .defaultTools(orderTools) registers the toolset, ChatClient automatically runs the ReAct loop inside: sends the prompt with the tool list to the model, if the model requests calling lookupOrder, Spring AI automatically executes the corresponding Java method, returns the result (Observation) back to the model, then receives the final answer. You don’t write the loop manually — this is exactly what Microsoft Agent Framework and Spring AI do in the same way fundamentally.

> handle("Where is my order ORD-001?")
Your order ORD-001 is being delivered (SHIPPED), total 450,000đ!

Note: The model only calls a tool when it “decides” it needs to — not every question triggers tool calls. If the user asks “do you ship internationally?”, the model will answer directly without calling lookupOrder.

5. Multi-agent: When One Agent Isn’t Enough

One of the standout capabilities Microsoft shipped is hand-off — transferring conversation between specialized agents. For small problems, you can simulate this with a simple router, without needing complex multi-agent frameworks:

@Service
public class AgentRouter {

    private final SupportAgentService orderAgent;
    private final RefundAgentService refundAgent;

    public AgentRouter(SupportAgentService orderAgent, RefundAgentService refundAgent) {
        this.orderAgent = orderAgent;
        this.refundAgent = refundAgent;
    }

    public String route(String userMessage) {
        String lower = userMessage.toLowerCase();
        if (lower.contains("refund") || lower.contains("return")) {
            return refundAgent.handle(userMessage);
        }
        return orderAgent.handle(userMessage);
    }
}

This router plays the role of minimal “orchestrator”. At larger scale, the choice of which agent handles the request should also be decided by the LLM itself (intent classification by model), rather than hard-coded contains() — manual coding is only suitable for demos or very small, stable intent sets.

6. Writing Tests for Tools and Agents

The easiest part to test in an agent system is the tool — because it’s a pure Java method, independent of the LLM. You should test this thoroughly, because the agent is only “intelligent” as the quality of tools it calls.

class OrderToolsTest {

    @Test
    void shouldReturnOrderSummaryWithStatusAndTotal() {
        OrderService fakeOrderService = orderId ->
            new Order(orderId, "SHIPPED", new BigDecimal("450000"));

        OrderTools tools = new OrderTools(fakeOrderService);

        String result = tools.lookupOrder("ORD-001");

        assertThat(result)
            .contains("ORD-001")
            .contains("SHIPPED")
            .contains("450000");
    }
}

Since OrderService has only one method findById, I use a lambda to create a fake implementation instead of needing an additional mocking framework — concise and sufficient for this unit test.

class AgentRouterTest {

    @Test
    void shouldRouteRefundKeywordToRefundAgent() {
        SupportAgentService orderAgent = mock(SupportAgentService.class);
        RefundAgentService refundAgent = mock(RefundAgentService.class);
        when(refundAgent.handle(anyString())).thenReturn("Refund request recorded");

        AgentRouter router = new AgentRouter(orderAgent, refundAgent);

        String result = router.route("I want a refund for order ORD-002");

        assertThat(result).isEqualTo("Refund request recorded");
        verify(refundAgent).handle(anyString());
        verify(orderAgent, never()).handle(anyString());
    }
}

Note: Don’t write unit tests that call the real LLM end-to-end — results aren’t deterministic (model may answer differently each time), costs money for API calls, and makes CI slow/flaky. Mock ChatClient or test tools/router separately as above; keep a separate integration test suite (run manually or nightly) to verify the agent chooses the right tool in real scenarios.

7. Common Mistakes and When Not to Use Agents

• Using agents for tasks solvable with if-else: if the logic for choosing actions is known in advance and fixed (e.g., always look up the order when given a code like ORD-\d+), writing code is cheaper, faster, and much easier to test than letting an LLM “think” each time.
• Skipping guardrails for sensitive actions: agents can call tools to delete data, issue refunds, send customer emails — without approval steps (human-in-the-loop) or tool permission limits, a prompt injection or model reasoning error could cause real harm.
• Not limiting loop iterations: if the model keeps choosing the wrong tool or lacks data to conclude, the ReAct loop may iterate infinitely, wasting tokens and money. Always set a maximum step count and timeout.
• Missing observability: when an agent answers incorrectly, you need to review the entire Thought/Action/Observation chain to debug — without detailed logs/traces, it’s nearly impossible to find what the model “thought” leading to the wrong result.
• Latency and cost: each step in the loop is an LLM call. A simple question may cost 2-3 model calls (each taking hundreds of ms to seconds). For APIs needing instant response (e.g., under 200ms), this kind of agent isn’t suitable.

8. Conclusion

Microsoft’s merger of Semantic Kernel and AutoGen into a single framework shows a clear trend: AI Agents are moving from “interesting demo” to production infrastructure needing orchestration, state, observability and strict permission control like any other enterprise system. Whether you use .NET, Python or Java/Spring AI like the examples in this article, the fundamental architecture still revolves around four blocks: Planner, Tool, Memory, Executor — master these four, and you can read and apply any agent framework that appears next.