Why Coding Agents Make Mistakes: Common Errors and Prevention

1. Overview

Coding agents are AI tools that can read a codebase on their own, edit files, run build/test commands, and repeat until the job is “done”. They clearly speed up work, but they also create errors that are harder to spot than human-made mistakes.

This article categorizes common coding agent errors, illustrates them with concrete Java examples, then presents mitigation techniques: writing tests first, a conventions file for the agent, automated gates, and review processes. After reading, you’ll know where to watch out in an agent-generated diff, and how to set up your environment to make it harder for the agent to go wrong.

2. Why Coding Agents Make Mistakes

You need to understand the nature of the problem before diving into specific errors. Language models generate code that looks reasonable based on patterns they’ve learned. “Looks reasonable” doesn’t mean “correct for your project”. There are three root causes:

• Lack of context: the agent only sees the code it reads in a working session. It doesn’t know your team’s implicit conventions, the historical reasons behind unfamiliar code, or business constraints that haven’t been written down.
• Optimizing for the wrong goal: when you ask to “make the test pass”, the agent might do exactly that literally, even by fixing the test.
• Misplaced confidence: agents rarely say “I’m not sure”. Incorrect and correct code are presented with the same tone of voice.

This topic matters now because agents are no longer limited to suggesting a few lines. More and more teams are handing them entire tasks. The larger the diff, the harder to review, and the costlier it is when mistakes slip through.

All the examples below only need spring-boot-starter-test. The version of this starter is managed by the Spring Boot parent, and the starter already includes JUnit 5, AssertJ, and Mockito:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>

3. Common Error Types

3.1. Hallucinated APIs

The agent calls a method that sounds very reasonable but doesn’t actually exist:

Customer customer = customerRepository.findById(id)
        .orElseThrowIfEmpty(() -> new CustomerNotFoundException(id)); // does not exist

Optional only has orElseThrow(...), not orElseThrowIfEmpty. Fortunately, the compiler catches this right away.

The lesson here: compile errors are the cheapest kind of error. If the agent is allowed to run mvn compile, it will fix itself. The more dangerous variant is when the method exists but behaves differently than the agent “thinks”. For example, the agent assumes List.of(...) returns a mutable list, when actually the list is immutable and the error only shows up at runtime.

3.2. Code Compiles, Runs, but Violates Business Logic

This is the most dangerous kind of error. Suppose you ask the agent to “write a function to calculate the total amount of an order”:

public record OrderLine(String sku, double unitPrice, int quantity) {}

public class OrderCalculator {
    public double total(List<OrderLine> lines) {
        return lines.stream()
                .mapToDouble(line -> line.unitPrice() * line.quantity())
                .sum();
    }
}

The code is concise and readable. Yet it’s still wrong, because it uses double for money:

var lines = List.of(new OrderLine("PEN", 0.1, 1), new OrderLine("BOOK", 0.2, 1));
System.out.println(new OrderCalculator().total(lines));

0.30000000000000004

The agent didn’t make a syntax error. It failed because it missed a rule your team takes for granted: money always uses BigDecimal. Here’s the correct version:

public record OrderLine(String sku, BigDecimal unitPrice, int quantity) {}

public class OrderCalculator {
    public BigDecimal total(List<OrderLine> lines) {
        return lines.stream()
                .map(line -> line.unitPrice().multiply(BigDecimal.valueOf(line.quantity())))
                .reduce(BigDecimal.ZERO, BigDecimal::add);
    }
}

3.3. Dummy Tests: Verifying the Mock Instead of the Code

When asked to write tests, the agent sometimes creates tests that always pass, regardless of what the actual code does:

@Test
void shouldChargeCustomer() {
    PaymentService payment = mock(PaymentService.class);
    when(payment.charge("C1", new BigDecimal("100"))).thenReturn(true);

    assertThat(payment.charge("C1", new BigDecimal("100"))).isTrue();
}

This test only confirms that Mockito works. It doesn’t call any actual business logic. The correct test must go through the class under test, such as CheckoutService:

public interface PaymentService {
    boolean charge(String customerId, BigDecimal amount);
}

public class CheckoutService {
    private final PaymentService paymentService;

    public CheckoutService(PaymentService paymentService) {
        this.paymentService = paymentService;
    }

    public String checkout(String customerId, BigDecimal amount) {
        if (amount.signum() <= 0) {
            throw new IllegalArgumentException("Amount must be positive");
        }
        return paymentService.charge(customerId, amount) ? "PAID" : "FAILED";
    }
}

class CheckoutServiceTest {

    private final PaymentService payment = mock(PaymentService.class);
    private final CheckoutService checkout = new CheckoutService(payment);

    @Test
    void returnsPaidAndChargesExactAmount() {
        when(payment.charge("C1", new BigDecimal("100.00"))).thenReturn(true);

        assertThat(checkout.checkout("C1", new BigDecimal("100.00"))).isEqualTo("PAID");
        verify(payment).charge("C1", new BigDecimal("100.00"));
    }

    @Test
    void rejectsNonPositiveAmountWithoutCharging() {
        assertThatThrownBy(() -> checkout.checkout("C1", BigDecimal.ZERO))
                .isInstanceOf(IllegalArgumentException.class);
        verifyNoInteractions(payment);
    }
}

The first test verifies both the returned result and that the correct amount was charged. The second test checks an error path and ensures no transaction occurs. A simple trick to detect dummy tests: deliberately break the code and run the test again. If the test still passes, the test is useless.

3.4. Fixing Tests to Make Them Pass

You ask to “fix the failing test”. The agent might fix the assertion instead of the code:

// Before
assertThat(calculator.total(lines)).isEqualByComparingTo("0.30");

// After agent "fixes" it
assertThat(calculator.total(lines)).isNotNull();

The build turns green and the task is marked complete, but the bug remains. Other variants of this error: adding @Disabled, deleting the test entirely, or loosening the tolerance in the assertion.

3.5. Swallowing Exceptions to Make Errors Disappear

When an exception is encountered during a run, the agent might take the quickest way to make it disappear:

public Customer loadCustomer(String id) {
    try {
        return customerClient.fetch(id);
    } catch (Exception e) {
        return null; // "handled the error"
    }
}

The root cause—say, a timeout or misconfiguration—gets hidden. The actual error surfaces later as a NullPointerException elsewhere, much harder to trace back.

3.6. Ignoring Existing Conventions and Making Widespread Changes

Your project already has MoneyUtils.round(...), but the agent writes its own rounding function with a different RoundingMode. Or you ask for a small bug fix, and the diff comes back touching 15 files: renaming variables, reformatting code, “while we’re at it” bumping dependency versions. Large diffs dilute review attention, and real bugs easily hide among harmless changes.

4. Mitigation Strategies

4.1. Write Tests First, Hand Implementation to the Agent

The most effective way is to let you define what “correct” means, while the agent handles “how”. Tests are executable specifications:

class OrderCalculatorTest {

    private final OrderCalculator calculator = new OrderCalculator();

    @Test
    void sumsLinesWithoutFloatingPointError() {
        var lines = List.of(
                new OrderLine("PEN", new BigDecimal("0.10"), 1),
                new OrderLine("BOOK", new BigDecimal("0.20"), 1));

        assertThat(calculator.total(lines)).isEqualByComparingTo("0.30");
    }

    @Test
    void returnsZeroForEmptyOrder() {
        assertThat(calculator.total(List.of())).isEqualByComparingTo("0");
    }
}

The first test forces the agent to use BigDecimal through the type of OrderLine. The second test pins down the behavior for an empty order. When tests are written by humans and the agent is forbidden from modifying them, the errors in sections 3.2 and 3.4 become much harder to trigger.

4.2. A Conventions File for the Agent

Most coding agents can read an instructions file placed at the repository root. The file name varies by tool; AGENTS.md is a fairly common convention. Write down the things your team takes for granted:

# Conventions for Agents

## Build & Test
- Run `./mvnw -q verify` before reporting completion.

## Required
- Money: always use BigDecimal, round via MoneyUtils.round().
- Never catch Exception broadly; never return null to hide errors.
- DO NOT modify, delete, or @Disabled existing tests. If you think a test is wrong, stop and explain.

## Scope
- Only modify files directly related to the task.
- Do not change dependency versions, do not reformat unrelated files.

This file is short, concrete, and each line maps to a known error type. Writing vague instructions like “write clean code” has almost no effect.

4.3. Automated Gates

Written conventions can be ignored, but a build failure cannot. You can enable compiler warnings and fail the build on any warning:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <configuration>
        <compilerArgs>
            <arg>-Xlint:all</arg>
        </compilerArgs>
        <failOnWarning>true</failOnWarning>
    </configuration>
</plugin>

Next, add a script to CI to detect when tests are modified or disabled:

#!/usr/bin/env bash
set -euo pipefail

./mvnw -q verify

changed_tests=$(git diff --name-only origin/main...HEAD -- 'src/test/**' || true)
if [[ -n "$changed_tests" ]]; then
  echo "Warning: PR modifies tests, reviewer must confirm:"
  echo "$changed_tests"
fi

if git diff origin/main...HEAD -- 'src/test/**' | grep -E '^\+.*@Disabled'; then
  echo "Error: new @Disabled found." >&2
  exit 1
fi

The script runs the full build and tests first. Then it lists modified test files for the reviewer to notice, and blocks the PR outright if any @Disabled is newly added. Here’s an example output when the agent sneakily disables a test:

Warning: PR modifies tests, reviewer must confirm:
src/test/java/com/shop/OrderCalculatorTest.java
+    @Disabled
Error: new @Disabled found.

You can expand in the same way to block patterns like catch (Exception, or add static analysis and architecture tests to the pipeline.

4.4. Small Tasks, Small Diffs, Focused Review

• Break down tasks: instead of “implement shopping cart feature”, assign “add CartService.addItem to make the existing test pass”. Each task should produce a diff readable in a few minutes.
• Read the diff, not the summary: the agent might report “added error handling” when the reality is return null.
• Use a separate review checklist for agent-generated code: were any tests modified? are there any catch blocks swallowing errors? do any functions duplicate existing utilities? are there changes outside the task scope?
• Run the actual code: for important changes, run the application or integration scripts yourself instead of just trusting the test results.

Note: Don’t let the agent write both the test and the implementation for the same behavior and then review itself. When it does, all its misunderstandings get “confirmed” twice, and you’ve lost an independent source of truth.

5. When Not to Hand Work to a Coding Agent

Agents are strongest when specs are clear, tests exist, and results are easy to verify. You should think hard, or use the agent only for suggestions rather than full autonomy, in these cases:

• Security code: authentication, authorization, encryption, handling untrusted input. Errors here typically don’t make tests fail.
• Concurrency: race conditions rarely surface in unit tests, and “looks correct” code is especially risky.
• Production data migration: irreversible operations shouldn’t be left to an agent trying things and retrying.
• Untested code areas: the agent has no feedback signal, and you have no way to verify. Write characterization tests first, then hand off the work.
• Major design decisions: choosing architecture, module boundaries, data models. The agent can propose options for reference, but you must be the one responsible.

6. Conclusion

Coding agents fail in fairly predictable ways: calling nonexistent APIs, writing syntactically correct but logically wrong code, creating dummy tests, fixing tests to make the build green, swallowing exceptions, and expanding changes beyond scope. The root causes are lack of context, optimizing for the wrong goal, and misplaced confidence.

The mitigation doesn’t come from finding the perfect prompt, but from building a working environment for the agent. You write tests as specs, record specific conventions in a guide file, place automated gates in CI, keep tasks small, and review diffs with focus. That way, the agent becomes a real productivity tool instead of a source of hard-to-find bugs.