Multi-agent: when AI agents delegate to each other
1. Overview
When LLM applications go beyond “ask a question, get an answer,” we start delegating multi-step tasks like order lookups, policy checks, and response drafting. Cramming everything into one agent hits limits sooner or later. Multi-agent divides work among specialized agents, with one orchestrating agent handing out tasks.
This post explains what multi-agent is, why it’s getting attention, and common patterns. Then you’ll build a simple orchestrator in Java 21 with tests and Spring Boot config. We’ll also cover when not to use this approach.
2. What is multi-agent?
An agent is a loop of three parts: an LLM receives a goal, decides on an action (call a tool, answer, ask back), then observes the result. Multi-agent is a system with many such agents. Each has its own system prompt, tools, and scope of responsibility, and they delegate work to each other.
A few coordination patterns are common:
- Orchestrator–worker (supervisor): a coordinating agent analyzes the request, breaks it into subtasks, delegates to workers, then synthesizes results.
- Pipeline: agents are chained in sequence, with one agent’s output feeding into the next, e.g. “research → draft → edit.”
- Handoff: an agent handling a task realizes it’s outside its scope and transfers the entire conversation to another agent, like phone routing.
- Generator–critic: one agent generates output, another evaluates and requests fixes, repeating until good enough or max rounds reached.
This post focuses on orchestrator–worker. It’s the easiest to understand and applies to many problems.
3. Why multi-agent is getting attention
Several technical reasons:
- Context is bounded. An agent with too many tools and instructions gets a long prompt and easily picks the wrong tool. A specialized agent only needs to know its part.
- Specialization. Each agent can use its own prompt, model, and parameters. Simple classification uses a small, cheap model. Complex reasoning uses a stronger one.
- Parallel execution. Independent subtasks run concurrently, reducing total response time.
- Clear permissions. A lookup agent only has read access. Only a refund agent can call the payment API. This boundary resembles permission boundaries between services.
- Testable and measurable. You can test and measure each agent separately instead of one giant prompt block.
In short, this is the familiar “divide and conquer” and “single responsibility” principles applied to LLMs.
4. Building orchestrator–worker with Java 21
Example problem: a customer asks, “Where’s my order #1001? If it’s late, do I get a refund?” We have two workers:
order: look up order status.refund: check refund eligibility.
The orchestrator plans, delegates to both workers in parallel, then synthesizes the answer.
4.1. Dependency
The example only needs basic Spring Boot starter and test library. Versions are managed by spring-boot-starter-parent:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>spring-boot-starter-test already includes JUnit 5 and AssertJ.
4.2. Abstracting LLM and agent
We don’t tie ourselves to a specific SDK. Every LLM call goes through an interface:
@FunctionalInterface
public interface LlmClient {
String complete(String systemPrompt, String userMessage);
}In production, you write an adapter implementing this interface using your SDK or framework of choice. In tests, you pass in a lambda. That’s the biggest benefit of this abstraction.
An agent is also an interface. We add an implementation based on LLM:
public interface Agent {
String name();
String handle(String task);
}
public record LlmAgent(String name, String systemPrompt, LlmClient llm) implements Agent {
@Override
public String handle(String task) {
return llm.complete(systemPrompt, task);
}
}
public record SubTask(String agent, String description) {}Each LlmAgent differs mainly in systemPrompt, and in practice also in its tools. A worker can also be pure code without LLM calls, e.g. an agent that just queries OrderRepository. The orchestrator doesn’t need to know what’s inside the worker.
4.3. Orchestrator
public class Orchestrator {
private static final String PLANNER_PROMPT = """
You are a customer service coordinator.
Break the request into subtasks, each line as: <agent>|<description>
Valid agents: %s. Maximum %d lines. Write nothing else.
""";
private static final String SYNTH_PROMPT = """
You synthesize results from agents into a brief customer-facing answer.
Use only information in the results.
""";
private final LlmClient llm;
private final Map<String, Agent> workers;
private final int maxSubtasks;
private final Duration timeout;
public Orchestrator(LlmClient llm, List<Agent> agents, int maxSubtasks, Duration timeout) {
this.llm = llm;
this.workers = agents.stream().collect(Collectors.toMap(Agent::name, a -> a));
this.maxSubtasks = maxSubtasks;
this.timeout = timeout;
}
public String run(String request) {
String plan = llm.complete(PLANNER_PROMPT.formatted(workers.keySet(), maxSubtasks), request);
List<String> results = execute(parse(plan));
return llm.complete(SYNTH_PROMPT,
"Request: " + request + "\nResults:\n" + String.join("\n", results));
}
List<SubTask> parse(String plan) {
return plan.lines()
.map(String::strip)
.filter(line -> line.contains("|"))
.map(line -> line.split("\\|", 2))
.filter(parts -> workers.containsKey(parts[0].strip()))
.limit(maxSubtasks)
.map(parts -> new SubTask(parts[0].strip(), parts[1].strip()))
.toList();
}
private List<String> execute(List<SubTask> tasks) {
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
List<Future<String>> futures = tasks.stream()
.map(t -> executor.submit(() ->
t.agent() + ": " + workers.get(t.agent()).handle(t.description())))
.toList();
return futures.stream().map(this::await).toList();
}
}
private String await(Future<String> future) {
try {
return future.get(timeout.toMillis(), TimeUnit.MILLISECONDS);
} catch (TimeoutException e) {
future.cancel(true);
return "ERROR: agent timed out";
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return "ERROR: interrupted";
} catch (ExecutionException e) {
return "ERROR: " + e.getCause().getMessage();
}
}
}Key points:
runhas three steps: plan (one LLM call), execute subtasks, then synthesize (one more LLM call).parsetreats LLM output as untrusted data. Malformed lines are skipped. Agent names not in the list are skipped. Subtask count is capped atmaxSubtasks. If the LLM “invents” anadminagent, it will never be called.executeuses Java 21 virtual threads. LLM calls are mostly network I/O waits, so virtual threads fit perfectly: one thread per subtask, no pool tuning needed. Thetry-with-resourcesblock only closes the executor after all tasks complete.awaitsets a timeout for each worker. A slow or failing worker doesn’t break the entire request. Errors are recorded asERROR: …strings so synthesis still runs and can tell the customer that partial information is missing.
For the request from section 4, the plan from the LLM might look like:
order|Look up status of order #1001
refund|Check refund eligibility if order #1001 is lateNote: We use
agent|descriptionformat for readability. In practice, you should ask the LLM to return JSON per schema, or use your provider’s structured output/tool calling feature. Either way, code-side validation is mandatory.
5. Testing the orchestrator
Since LlmClient is a functional interface, we use lambdas instead of real LLMs. Tests run fast, give consistent results, and cost nothing in API calls.
class OrchestratorTest {
@Test
void givenPlanWithUnknownAgent_whenRun_thenOnlyKnownWorkersAreCalled() {
List<String> workerCalls = new CopyOnWriteArrayList<>();
LlmClient coordinatorLlm = (system, user) -> system.contains("coordinator")
? """
order|Look up status of order #1001
refund|Check refund eligibility for order #1001
admin|Delete all orders
"""
: "SYNTHESIS\n" + user;
LlmClient workerLlm = (system, task) -> {
workerCalls.add(task);
return system.startsWith("ORDER") ? "Order #1001 is in transit, 2 days late"
: "Eligible for shipping fee refund";
};
var orchestrator = new Orchestrator(coordinatorLlm, List.of(
new LlmAgent("order", "ORDER: look up order status", workerLlm),
new LlmAgent("refund", "REFUND: check refund eligibility", workerLlm)),
3, Duration.ofSeconds(5));
String answer = orchestrator.run("Where's order #1001? Do I get a refund if it's late?");
assertThat(workerCalls).hasSize(2).noneMatch(task -> task.contains("Delete"));
assertThat(answer)
.contains("order: Order #1001 is in transit, 2 days late")
.contains("refund: Eligible for shipping fee refund");
}
}This test checks two important behaviors:
- The line
admin|Delete all ordersin the plan is filtered out, so only two workers are called. - Results from both workers reach the synthesis step. The order in the answer preserves the plan’s order even though the workers ran in parallel.
The answer content that the synthesis lambda receives looks like:
SYNTHESIS
Request: Where's order #1001? Do I get a refund if it's late?
Results:
order: Order #1001 is in transit, 2 days late
refund: Eligible for shipping fee refundThis test only checks the orchestration logic, i.e., code you wrote. Real LLM answer quality needs separate evaluation via a set of sample questions and scoring criteria.
6. Declaring agents via Spring Boot config
As the number of workers grows, declaring them in application.yml lets you add or change prompts without touching orchestration code:
agents:
max-subtasks: 4
timeout: 20s
workers:
- name: order
system-prompt: "You look up order status. Answer only based on system data."
- name: refund
system-prompt: "You check refund eligibility per policy. Don't make refund promises yourself."Map the configuration to a record:
@ConfigurationProperties(prefix = "agents")
public record AgentProperties(int maxSubtasks, Duration timeout, List<Worker> workers) {
public record Worker(String name, String systemPrompt) {}
}Then create an Orchestrator bean:
@Configuration
@EnableConfigurationProperties(AgentProperties.class)
public class AgentConfig {
@Bean
Orchestrator orchestrator(LlmClient llm, AgentProperties props) {
List<Agent> agents = props.workers().stream()
.<Agent>map(w -> new LlmAgent(w.name(), w.systemPrompt(), llm))
.toList();
return new Orchestrator(llm, agents, props.maxSubtasks(), props.timeout());
}
}Spring Boot automatically converts 20s to Duration and max-subtasks to maxSubtasks. The LlmClient bean is an adapter you write around your SDK. If you want, you can create multiple LlmClient beans for different models, letting each worker pick via configuration.
7. Common mistakes and when NOT to use this
7.1. Common mistakes
- Blindly trusting the LLM’s plan. The plan is unvalidated input. You must whitelist agent names, cap steps, and validate parameters before executing, especially for agents with write permissions.
- Unbounded loops. If agents can delegate to each other, the system can loop: A → B → A. Always set a max step count, timeout, and token budget per request.
- Lost tracing. A request goes through many LLM calls. If you don’t log and trace with a common request ID, you won’t know which agent caused the wrong answer. Each step should record the prompt, result, time, and cost.
- Insufficient context. A worker only sees the
descriptionthe orchestrator sends it. If the description is too brief, e.g., just “check refund” without the order ID, the worker will guess. Have the planner include all necessary details in each subtask. - Amplified errors. A worker answers incorrectly, and synthesis restates it confidently. The synthesis prompt must insist on using only information in the results and reporting errors.
7.2. When NOT to use multi-agent
Note: Multi-agent isn’t automatically “better” than single agent. Each added agent means more LLM calls, more latency, more cost, and more things that can break.
Reconsider if any of these apply:
- A single agent with a few tools works well. Start with single agent. Only split when you see a clear problem: prompt too long, wrong tool picks, or permission needs.
- Fixed workflows. If steps are always A then B then C, plain code (calling LLM at each step if needed) is more predictable, testable, and cheaper than LLM planning.
- Low latency needed. The model above alone requires at least two sequential LLM calls (planning and synthesis), not counting worker time.
- Tightly dependent steps. If each step needs full context from the prior one, splitting into agents only fragments and loses information.
8. Conclusion
Multi-agent applies divide-and-conquer to LLM applications. Each agent has narrow scope, its own prompt and permissions, while an orchestrator handles planning and synthesis. In this post, you built an orchestrator–worker model in Java 21: abstract LlmClient for testability, validate LLM-generated plans, run workers in parallel with timeouts using virtual threads, and declare workers via Spring Boot config.
The key takeaway: treat LLM output as untrusted data, and always set limits on steps, time, and cost. Start with a single agent. Move to multi-agent only when you have a specific reason and can measure the benefit.
AI code review: how to use effectively and safely
Understand what AI code review is, integrate it into review processes and CI pipelines, hide secrets before sending diffs, prevent prompt injection, and know when not to use it.
Local-first software: offline apps and sync with CRDT
Learn about local-first software, why CRDTs let apps work offline and merge data automatically, with Java 21 and Spring Boot 3 examples.
Observability for LLM Applications: Tracking Prompts, Costs, and Quality
How to measure latency, tokens, costs, and response quality of LLM applications using Micrometer, OpenTelemetry, and Spring Boot 3.
Postgres Does Everything: Queue, Search, Vector in One Database
Use PostgreSQL as a job queue, full-text search, and vector search with Spring Boot. Includes code examples, tests, and guidance on when to move to specialized systems.
Are Vector Databases Still Necessary With Longer Context Windows?
Longer context windows let you fit more documents in prompts, but they don't fully replace retrieval. This analysis shows when you need a vector database and when you don't.