ExecutorService and thread pool: choosing the right type

1. Overview

Creating new Thread() for each task is easy to write, but hard to control when load increases. ExecutorService solves this by reusing threads and queuing tasks. The Executors class provides many different pool types, and choosing the wrong one can cause OutOfMemoryError or hang the application.

This article goes through each pool type, how ThreadPoolExecutor handles tasks internally, how to choose pool size and how to configure it in Spring Boot 3.x. After reading, you’ll know which pool to use for CPU-bound tasks, I/O-bound tasks, or scheduled tasks.

2. ExecutorService basics

The following example uses a simple pool to run a task with a return value:

import java.util.concurrent.*;

public class OrderReportApp {

    public static void main(String[] args) throws Exception {
        try (ExecutorService executor = Executors.newFixedThreadPool(4)) {
            Future<Integer> total = executor.submit(() -> countOrders("2026-10"));
            System.out.println("Total orders: " + total.get());
        }
    }

    static int countOrders(String month) {
        return 128; // simulate database query
    }
}

Total orders: 128

The code works as follows:

• submit() takes a Callable and returns a Future<Integer> immediately. The task runs on one of the pool’s threads.
• total.get() blocks the current thread until there’s a result.
• Since Java 19, ExecutorService implements AutoCloseable. When exiting the try block, close() calls shutdown() and waits for all tasks to finish.

Note: If you don’t close the executor, non-daemon threads will keep the JVM alive. The main program won’t exit.

3. Pool types in Executors

3.1. newFixedThreadPool(n)

This pool has exactly n threads. Extra tasks are put into an unbounded LinkedBlockingQueue.

ExecutorService invoiceExecutor = Executors.newFixedThreadPool(8);

This type is suitable when you want to limit the number of tasks running concurrently, for example CPU-bound tasks. The risk lies in the queue. If the producer pushes tasks faster than they’re processed, the queue grows until heap runs out.

3.2. newCachedThreadPool()

This pool has no fixed threads. It uses a SynchronousQueue, meaning there’s no real queue. Each new task is given to an idle thread if available, otherwise the pool creates a new thread. Idle threads are terminated after 60 seconds.

ExecutorService notificationExecutor = Executors.newCachedThreadPool();

This pool is good for many short tasks with low, uneven load. When load spikes, it can create thousands of platform threads. Each thread takes about 1 MB of stack by default, so the application can easily hit OutOfMemoryError: unable to create native thread.

3.3. newSingleThreadExecutor()

This pool has only one thread, and tasks run sequentially in the exact order they were submitted.

ExecutorService auditLogExecutor = Executors.newSingleThreadExecutor();
auditLogExecutor.submit(() -> System.out.println("Order #1 created"));
auditLogExecutor.submit(() -> System.out.println("Order #1 paid"));

Order #1 created
Order #1 paid

The two log lines always print in the correct order. This type is useful when you need to log, write to files, or update state in order without locks. If the thread dies due to an exception, the pool creates a new thread to replace it. Its queue is also unbounded.

3.4. newScheduledThreadPool(n)

This type is for scheduled or repeating tasks:

ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);

scheduler.scheduleAtFixedRate(
        () -> System.out.println("Syncing inventory at " + java.time.LocalTime.now()),
        0, 5, TimeUnit.SECONDS);

scheduleAtFixedRate runs the task every 5 seconds, measured from the start of the previous run. In contrast, scheduleWithFixedDelay measures 5 seconds from when the previous run ended. You should choose scheduleWithFixedDelay when task execution time is variable.

Note: If a scheduled task throws an exception, all subsequent runs are silently cancelled with no logging. Always wrap task code in try/catch.

3.5. newWorkStealingPool()

This method returns a ForkJoinPool with a number of threads equal to the number of CPUs. Each thread has its own queue. Idle threads will “steal” work from other threads’ queues.

ExecutorService pricingExecutor = Executors.newWorkStealingPool();

This type is good for CPU-bound tasks that can be recursively subdivided, like parallel computation over large datasets. Pool threads are daemon threads, and the pool doesn’t guarantee execution order.

3.6. newVirtualThreadPerTaskExecutor() (Java 21)

With this type, each task runs on a new virtual thread. Virtual threads are very cheap, so you can create hundreds of thousands.

try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 10_000; i++) {
        int customerId = i;
        executor.submit(() -> {
            Thread.sleep(100); // simulate HTTP call to CustomerService
            return customerId;
        });
    }
}

The code above runs 10,000 tasks blocking for 100 ms and completes in a few hundred milliseconds. If you use a fixed pool of 10 threads, the same work takes about 100 seconds.

This is the right default for I/O-bound tasks like HTTP calls, database queries, or file reads. You need to remember three things:

• Don’t pool virtual threads. They’re created to be used once and discarded.
• To limit load on external resources, use Semaphore. For example, if the database only handles 20 connections, don’t rely on pool size to limit it.
• In Java 21, synchronized blocks with blocking operations will “pin” the virtual thread to a carrier thread. Then virtual thread benefits disappear. Prefer ReentrantLock in such sections.

Example of limiting concurrent queries with Semaphore:

Semaphore dbPermits = new Semaphore(20);

executor.submit(() -> {
    dbPermits.acquire();
    try {
        return orderRepository.findByCustomerId(customerId);
    } finally {
        dbPermits.release();
    }
});

4. Configuring ThreadPoolExecutor manually

The factory methods in Executors usually hide a dangerous parameter: an unbounded queue or unlimited thread count. For production systems, you should create ThreadPoolExecutor yourself to control everything:

ThreadFactory factory = Thread.ofPlatform().name("payment-", 0).factory();

ThreadPoolExecutor paymentExecutor = new ThreadPoolExecutor(
        4,                                   // corePoolSize
        8,                                   // maximumPoolSize
        60, TimeUnit.SECONDS,                // keepAlive for threads above core
        new ArrayBlockingQueue<>(100),       // bounded queue
        factory,                             // naming: payment-0, payment-1...
        new ThreadPoolExecutor.CallerRunsPolicy());

When a new task arrives, ThreadPoolExecutor handles it in this exact order:

1. If thread count is less than corePoolSize, the pool creates a new thread.
2. Otherwise, the pool puts the task in the queue.
3. If the queue is full and thread count is less than maximumPoolSize, the pool creates another thread.
4. If both limits are reached, the pool calls RejectedExecutionHandler.

The consequence is that maximumPoolSize only takes effect when the queue is full. If you use an unbounded queue, the pool will never exceed corePoolSize. This is a very common source of confusion.

Java provides four built-in rejection policies:

Policy	Behavior
AbortPolicy (default)	Throws RejectedExecutionException
CallerRunsPolicy	The thread calling execute() runs the task itself, helping to slow down the producer (back-pressure)
DiscardPolicy	Discards the task silently
DiscardOldestPolicy	Discards the oldest task in the queue and retries

Setting thread names via ThreadFactory makes thread dumps and logs much more readable. Comparing payment-3 with pool-7-thread-3 shows the difference immediately.

5. Choosing the right type

5.1. Pool size

For CPU-bound tasks, the thread count should be approximately the number of cores:

int cores = Runtime.getRuntime().availableProcessors();
ExecutorService cpuExecutor = Executors.newFixedThreadPool(cores);

More threads than cores just increases context switching overhead.

For I/O-bound tasks on platform threads, you can use an approximation formula:

thread count ≈ number of cores × (1 + wait time / compute time)

Example: machine has 8 cores, each request waits 90 ms for I/O and computes for 10 ms. Result is 8 × (1 + 9) = 80 threads. This is just a starting point. You need to measure actual performance with load testing.

5.2. Decision table

Problem	Choice
Heavy CPU computation	newFixedThreadPool(cores) or ForkJoinPool
Calling APIs, querying DB at scale	newVirtualThreadPerTaskExecutor() + Semaphore
Need sequential order	newSingleThreadExecutor()
Scheduled tasks	newScheduledThreadPool(n)
Production, need back-pressure	ThreadPoolExecutor with bounded queue
Spiky load, no limits	Avoid newCachedThreadPool()

6. Thread pool in Spring Boot 3.x

First, you need the Maven dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>

Then declare a dedicated executor for sending emails:

@Configuration
@EnableAsync
public class AsyncConfig {

    @Bean(name = "emailExecutor")
    public ThreadPoolTaskExecutor emailExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4);
        executor.setMaxPoolSize(8);
        executor.setQueueCapacity(200);
        executor.setThreadNamePrefix("email-");
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        executor.setWaitForTasksToCompleteOnShutdown(true);
        executor.setAwaitTerminationSeconds(30);
        return executor;
    }
}

@Service
public class EmailService {

    @Async("emailExecutor")
    public void sendOrderConfirmation(String customerEmail) {
        System.out.println(Thread.currentThread().getName() + " sending email to " + customerEmail);
    }
}

email-1 sending email to an.nguyen@example.com

A few important points:

• queueCapacity defaults to Integer.MAX_VALUE for ThreadPoolTaskExecutor. If you don’t set this value, maxPoolSize becomes meaningless, as explained in section 4.
• setWaitForTasksToCompleteOnShutdown(true) ensures emails being sent aren’t cut off when the application shuts down.
• If you want to use virtual threads for @Async, Tomcat, and @Scheduled, you just need to add one config line:

spring.threads.virtual.enabled=true

7. Testing pool behavior

The following test confirms the exact execution order described in section 4: core threads first, then queue, then extra threads, finally rejection.

import org.junit.jupiter.api.Test;
import java.util.concurrent.*;
import static org.assertj.core.api.Assertions.*;

class PaymentExecutorTest {

    @Test
    void shouldRejectWhenThreadsAndQueueAreFull() throws Exception {
        CountDownLatch release = new CountDownLatch(1);
        ThreadPoolExecutor executor = new ThreadPoolExecutor(
                1, 2, 30, TimeUnit.SECONDS,
                new ArrayBlockingQueue<>(1),
                new ThreadPoolExecutor.AbortPolicy());

        Runnable blockingPayment = () -> {
            try {
                release.await();
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        };

        try {
            executor.execute(blockingPayment); // 1: create core thread
            executor.execute(blockingPayment); // 2: into queue
            executor.execute(blockingPayment); // 3: queue full -> create thread 2

            assertThat(executor.getPoolSize()).isEqualTo(2);
            assertThat(executor.getQueue()).hasSize(1);
            assertThatThrownBy(() -> executor.execute(blockingPayment))
                    .isInstanceOf(RejectedExecutionException.class);
        } finally {
            release.countDown();
            executor.shutdown();
            assertThat(executor.awaitTermination(5, TimeUnit.SECONDS)).isTrue();
        }
    }
}

CountDownLatch keeps all tasks in a blocking state, so the pool’s state remains static when you inspect it. The fourth task has no room, so AbortPolicy throws an exception. The finally block releases the latch and closes the pool, so the test doesn’t leave behind stray threads.

8. Common mistakes

• Using Executors.newFixedThreadPool with uncontrolled producers. An unbounded queue will grow and cause OOM. Use ArrayBlockingQueue with a limit.
• Creating a new pool for each request. Pools must be shared, for example declared as a bean or static final field. Creating and forgetting to close leaks threads.
• Swallowing exceptions. With submit(), exceptions are held in the Future. If you don’t call get(), you’ll never see the error. Call get() or catch errors inside the task.
• Deadlock from tasks waiting for subtasks on the same small pool. Example: pool has 2 threads, both parent tasks call get() on child tasks sitting in the queue. No thread left to run the child tasks, so the whole system hangs.
• Putting heavy ThreadLocal into virtual threads. You can have millions of virtual threads, each holding a copy, so memory grows very quickly.
• Calling shutdownNow() when tasks don’t handle interruption. The task will keep running. Long-running tasks should check Thread.currentThread().isInterrupted().

Note: Don’t use ExecutorService for complex multi-step workflows with dependencies. That case should use CompletableFuture, which can still run on your configured executor.

9. Conclusion

Each thread pool type solves a specific problem:

• Fixed pool or ForkJoinPool for CPU-bound tasks.
• Virtual threads with Semaphore for I/O-bound tasks on Java 21.
• Single thread executor when order matters.
• Scheduled pool for periodic tasks.

In production, configure ThreadPoolExecutor or ThreadPoolTaskExecutor yourself with a bounded queue, clear rejection policy, and recognizable thread names. Finally, validate pool size with load testing rather than just formulas.