Counting Tokens and Estimating Costs When Calling LLMs in Java

1. Overview

Each time you call an LLM like GPT-4o, you pay by token, not by character count or number of requests. Without measurement, your month-end bill can be a nasty surprise. Especially when the prompt contains long conversation history or RAG documents.

In this post, you’ll count tokens directly in the JVM using JTokkit, calculate costs with BigDecimal, and block requests that exceed budget. Finally, you’ll reconcile your estimate with the actual token count that Spring AI returns.

2. What Are Tokens and Why Count Them

A token is the unit that a model’s tokenizer breaks text into. In English, one token averages around 4 characters. Vietnamese with diacritics typically consumes more tokens for the same meaning, because characters like “ệ”, “ữ” appear rarely in the tokenizer’s training data.

A request’s cost has two parts with different pricing:

• Input tokens (prompt tokens): system prompt, chat history, question, attached documents.
• Output tokens (completion tokens): the part the model generates. Its unit price is typically many times higher than input.

There are two moments when you need to count:

1. Before calling: to estimate, trim context, or reject an overly expensive request.
2. After calling: to record actual cost based on data the provider returns.

3. Counting Tokens Locally with JTokkit

JTokkit is a pure Java library that implements the same BPE encoding tables as OpenAI’s tiktoken. The library requires no network calls.

<dependency>
    <groupId>com.knuddels</groupId>
    <artifactId>jtokkit</artifactId>
    <version>1.1.0</version>
</dependency>

Count tokens for a string:

import com.knuddels.jtokkit.Encodings;
import com.knuddels.jtokkit.api.Encoding;
import com.knuddels.jtokkit.api.EncodingRegistry;
import com.knuddels.jtokkit.api.EncodingType;

public class TokenCountDemo {

    public static void main(String[] args) {
        EncodingRegistry registry = Encodings.newDefaultEncodingRegistry();
        Encoding encoding = registry.getEncoding(EncodingType.O200K_BASE);

        String english = "Hello, world!";
        String vietnamese = "Xin chào, thế giới! Đơn hàng của bạn đã được xác nhận.";

        System.out.println(english + " -> " + encoding.countTokens(english));
        System.out.println(vietnamese + " -> " + encoding.countTokens(vietnamese));
    }
}

Explanation of each part:

• Encodings.newDefaultEncodingRegistry() loads the encoding tables. This loading takes time and memory, so create it once and reuse it.
• O200K_BASE is the encoding for the GPT-4o family. Older models like GPT-4 and GPT-3.5 use CL100K_BASE.
• countTokens returns only the count. It’s faster than encode because it doesn’t need to keep the token list.

Results for the English sentence look like this. The count for Vietnamese depends on the encoding but will be significantly higher than the word count.

Hello, world! -> 4
Xin chào, thế giới! Đơn hàng của bạn đã được xác nhận. -> (around 15–20)

If you want to select encoding by model name, use registry.getEncodingForModel(ModelType.GPT_4O).

Note: JTokkit is accurate only with OpenAI’s tokenizer. Claude, Gemini, and Llama use different tokenizers. With those models, use JTokkit only for rough estimation and add a 10–20% safety margin.

3.1. Counting Tokens for a Conversation

The Chat API doesn’t send a plain string. It sends a list of messages with role, and each message has formatting tokens attached. According to OpenAI’s description for chat models, each message costs about 3 extra tokens. The reply is also “primed” with 3 tokens.

import com.knuddels.jtokkit.api.Encoding;
import java.util.List;

public record ChatMessage(String role, String content) {}

public class ChatTokenCounter {

    private static final int TOKENS_PER_MESSAGE = 3;
    private static final int REPLY_PRIMING = 3;

    private final Encoding encoding;

    public ChatTokenCounter(Encoding encoding) {
        this.encoding = encoding;
    }

    public int count(List<ChatMessage> messages) {
        int total = REPLY_PRIMING;
        for (ChatMessage message : messages) {
            total += TOKENS_PER_MESSAGE
                    + encoding.countTokens(message.role())
                    + encoding.countTokens(message.content());
        }
        return total;
    }
}

This is an approximation. If the request has tool/function definitions or images, they also count toward input. JTokkit can’t count those directly. So you always need to reconcile with actual data (section 6).

4. Configuring Pricing Table and Calculating Cost

Model pricing changes frequently, so don’t hard-code prices in your code. Put prices in a configuration file. The numbers below are examples, in USD per million tokens.

pricing:
  models:
    gpt-4o:
      input-per-million: 2.50
      output-per-million: 10.00
    gpt-4o-mini:
      input-per-million: 0.15
      output-per-million: 0.60

Bind the configuration to a record:

import java.math.BigDecimal;
import java.util.Map;
import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "pricing")
public record PricingProperties(Map<String, ModelPrice> models) {

    public record ModelPrice(BigDecimal inputPerMillion, BigDecimal outputPerMillion) {}

    public ModelPrice priceOf(String model) {
        ModelPrice price = models.get(model);
        if (price == null) {
            throw new IllegalArgumentException("No pricing configured for model: " + model);
        }
        return price;
    }
}

Remember to enable binding with @EnableConfigurationProperties(PricingProperties.class) or @ConfigurationPropertiesScan on your @SpringBootApplication class. If the model name contains a dot (e.g., gpt-4.1), write the key in YAML as "[gpt-4.1]". Otherwise, Spring will interpret the dot as a nested level.

Now for the cost calculation service:

import java.math.BigDecimal;
import java.math.RoundingMode;
import org.springframework.stereotype.Service;

@Service
public class CostEstimator {

    private static final BigDecimal ONE_MILLION = BigDecimal.valueOf(1_000_000);

    private final PricingProperties pricing;

    public CostEstimator(PricingProperties pricing) {
        this.pricing = pricing;
    }

    public BigDecimal estimate(String model, long inputTokens, long outputTokens) {
        PricingProperties.ModelPrice price = pricing.priceOf(model);

        BigDecimal inputCost = price.inputPerMillion()
                .multiply(BigDecimal.valueOf(inputTokens));
        BigDecimal outputCost = price.outputPerMillion()
                .multiply(BigDecimal.valueOf(outputTokens));

        return inputCost.add(outputCost)
                .divide(ONE_MILLION, 8, RoundingMode.HALF_UP);
    }
}

A few things to note:

• Use BigDecimal instead of double. One request costs a fraction of a cent, and accumulating millions with double will introduce rounding errors.
• Multiply before dividing to maintain precision.
• Round to 8 decimal places. That’s small enough not to lose cheap requests.

Example with gpt-4o, 1,200 input tokens and 300 output tokens:

input : 1200 × 2.50 / 1_000_000 = 0.003
output:  300 × 10.00 / 1_000_000 = 0.003
total : 0.00600000 USD

You see that output has only 1/4 the token count but costs the same. That’s why you should limit response length.

5. Blocking Requests Over Budget Before Calling

Before calling, you don’t know how long the model will respond. But you know the upper limit: the max_tokens parameter. Worst-case cost equals counted input plus max_tokens output.

import java.math.BigDecimal;
import java.util.List;
import org.springframework.stereotype.Component;

@Component
public class BudgetGuard {

    private final ChatTokenCounter tokenCounter;
    private final CostEstimator costEstimator;

    public BudgetGuard(ChatTokenCounter tokenCounter, CostEstimator costEstimator) {
        this.tokenCounter = tokenCounter;
        this.costEstimator = costEstimator;
    }

    public BigDecimal checkWorstCase(String model, List<ChatMessage> messages,
                                     int maxOutputTokens, BigDecimal limitUsd) {
        int inputTokens = tokenCounter.count(messages);
        BigDecimal worstCase = costEstimator.estimate(model, inputTokens, maxOutputTokens);

        if (worstCase.compareTo(limitUsd) > 0) {
            throw new IllegalStateException(
                    "Request estimated at %s USD, exceeds budget of %s USD"
                            .formatted(worstCase.toPlainString(), limitUsd.toPlainString()));
        }
        return worstCase;
    }
}

ChatTokenCounter needs to be declared as a bean with a shared Encoding:

import com.knuddels.jtokkit.Encodings;
import com.knuddels.jtokkit.api.EncodingType;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class TokenizerConfig {

    @Bean
    public ChatTokenCounter chatTokenCounter() {
        var encoding = Encodings.newDefaultEncodingRegistry()
                .getEncoding(EncodingType.O200K_BASE);
        return new ChatTokenCounter(encoding);
    }
}

You can use BudgetGuard in several ways. You can reject the request. You can also trim the oldest chat history until budget is available. Or switch to a cheaper model like gpt-4o-mini.

6. Getting Actual Token Count from Spring AI

Estimation is only for decision-making. To calculate actual cost, use the data the provider returns.

Add Spring AI:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>1.0.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-model-openai</artifactId>
    </dependency>
</dependencies>

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-4o
spring.ai.openai.chat.options.max-tokens=500

Call the model and read Usage:

import java.math.BigDecimal;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.metadata.Usage;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.stereotype.Service;

@Service
public class SupportAssistant {

    private static final Logger log = LoggerFactory.getLogger(SupportAssistant.class);

    private final ChatClient chatClient;
    private final CostEstimator costEstimator;

    public SupportAssistant(ChatClient.Builder builder, CostEstimator costEstimator) {
        this.chatClient = builder.build();
        this.costEstimator = costEstimator;
    }

    public String answer(String question) {
        ChatResponse response = chatClient.prompt()
                .user(question)
                .call()
                .chatResponse();

        Usage usage = response.getMetadata().getUsage();
        String model = response.getMetadata().getModel();

        BigDecimal cost = costEstimator.estimate(
                "gpt-4o", usage.getPromptTokens(), usage.getCompletionTokens());

        log.info("model={} prompt={} completion={} cost={} USD",
                model, usage.getPromptTokens(), usage.getCompletionTokens(),
                cost.toPlainString());

        return response.getResult().getOutput().getText();
    }
}

The log output will look like:

model=gpt-4o-2024-08-06 prompt=412 completion=187 cost=0.00290000 USD

Note that getModel() returns the name with version, e.g., gpt-4o-2024-08-06. That’s why the example uses a fixed key "gpt-4o" for price lookup. In practice, normalize the model name before looking up the price.

When comparing actual prompt with your count from section 3.1, a difference of a few tokens is normal. If the difference is large, it’s usually because a default system prompt, tool definitions, or advisor (e.g., memory, RAG) injected content you didn’t count.

7. Testing CostEstimator

Cost calculation logic must not have bugs. Write standalone tests without needing Spring context:

import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;

import com.knuddels.jtokkit.Encodings;
import com.knuddels.jtokkit.api.EncodingType;
import java.math.BigDecimal;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;

class CostEstimatorTest {

    private final PricingProperties pricing = new PricingProperties(Map.of(
            "gpt-4o", new PricingProperties.ModelPrice(
                    new BigDecimal("2.50"), new BigDecimal("10.00"))));

    private final CostEstimator estimator = new CostEstimator(pricing);

    @Test
    void givenInputAndOutputTokens_whenEstimate_thenSumsBothParts() {
        BigDecimal cost = estimator.estimate("gpt-4o", 1_200, 300);

        assertThat(cost).isEqualByComparingTo("0.006");
    }

    @Test
    void givenUnknownModel_whenEstimate_thenThrows() {
        assertThatThrownBy(() -> estimator.estimate("unknown-model", 10, 10))
                .isInstanceOf(IllegalArgumentException.class)
                .hasMessageContaining("unknown-model");
    }

    @Test
    void givenVietnameseText_whenCount_thenUsesMoreTokensThanWords() {
        var encoding = Encodings.newDefaultEncodingRegistry()
                .getEncoding(EncodingType.O200K_BASE);
        var counter = new ChatTokenCounter(encoding);
        String content = "Đơn hàng của bạn đã được xác nhận";

        int tokens = counter.count(List.of(new ChatMessage("user", content)));

        assertThat(tokens).isGreaterThan(content.split(" ").length);
    }
}

The first test uses isEqualByComparingTo instead of isEqualTo. Because 0.00600000 and 0.006 have different scale, so BigDecimal.equals() returns false.

8. Common Mistakes

• Creating EncodingRegistry per request. Loading encoding tables is expensive. Create it once and reuse it, since Encoding is thread-safe.

• Using the wrong encoding. Counting GPT-4o with CL100K_BASE will give wrong numbers. Choose encoding by model.

• Counting only input. Output typically costs 4x or more than input. Ignoring this is the biggest error.

• Forgetting “hidden” tokens. Reasoning models (o-series) generate reasoning tokens. These don’t appear in the answer but still cost like output. Conversely, cached input tokens get a discount. Check getNativeUsage() for details.

• Using double for money. Accumulated rounding errors will skew cost reports vs. invoices.

• Blindly trusting estimation. Estimation is for blocking and alerts. Recording cost must be based on actual Usage.

Note: You don’t need to count tokens beforehand for every request. If the prompt is short, fixed, and has max_tokens, cost is already bounded. Then just logging Usage after the call is enough. Count ahead when context varies per user, like chat history, RAG, or file uploads.

9. Conclusion

In this post, you counted tokens locally with JTokkit, including overhead per message. You calculated cost accurately with BigDecimal and pricing in config. You used max_tokens to cap worst-case cost before calling.

The core principle: estimate to decide, actual data to bill. Combining BudgetGuard before the request with logging Usage after the response, you’ll have enough data to track costs by model, feature, or customer.