The Emperor’s New Unit Tests: Why LLMs Can’t Just ‘Poof!’ Out Perfect Code Checks

Ah, the siren song of effortless automation! We’ve all seen those alluring prompts, “Write unit tests for utils/date.ts,” promising a world where our tireless AI companions simply conjure up robust test suites. And, for the most part, they deliver: tests compile, run, and often, miraculously, pass. A round of applause, everyone’s satisfied, right?

But hold your horses, fellow code connoisseurs. This perceived paradise of automated testing raises a rather prickly question that keeps me up at night, pondering the digital abyss: are those tests actually any good?

Let’s strip back the layers of AI hype for a moment and revisit the foundational ‘why’ of unit testing. We craft these meticulous checks to validate our code against a predefined set of requirements. So, when we offload this task to an LLM, are we genuinely confident that our digital scribe is privy to all those intricate requirements?

If we simply toss the code into the AI’s gaping maw, it’s left with the code itself, maybe some inline documentation, and comments. Is that truly enough to grasp the nuanced intent behind every line? Or is it merely sophisticated pattern matching? To uncover the truth, we embarked on a little investigative journey with a simple set of requirements.

The Blueprint: Our Requirements for a Product Service

Imagine we’re building a new `ProductService` in PHP (though, as our research shows, the language is largely irrelevant to the conclusions). Here’s what this service absolutely must do:

• Implement a `getProductPrice` method to fetch a product’s price by its ID.
• If the provided `productId` is empty or just whitespace, it must throw an `EmptyProductIdException` with an error code of 0.
• It should dutifully consult a `ProductRepository` to retrieve the product.
• Should the `ProductRepository` come up empty-handed (i.e., product not found), an `ProductNotFoundException` with an error code of 1 should be thrown.
• Finally, if all goes well, it simply returns the product’s price.
• Our `Product` entity itself is a straightforward affair: ID, name, price, and, crucially, a ‘cost price’. Keep that last one in mind; it’s a bit of a trickster.

The Starting Line: Our ProductService & Its Docblock

Our `ProductService` was built to these specifications. Its `getProductPrice` method looked perfectly innocent, complete with a helpful docblock that explicitly stated its return type and the exceptions it might throw (including their error codes).

For instance, the method clearly expected to `return $product->getPrice();`. This documentation, you’d think, would be a golden compass for any test-generating entity.

The First Round: Naive LLM Tests

Armed with our pristine `ProductService`, we turned to OpenAI Codex (v0.25, GPT-5 high model) and posed the most straightforward of requests: “Write unit tests for all public methods of the ProductService.”

Codex, to its credit, delivered a seemingly solid suite: two tests for successful price retrieval (one even checking for ID trimming!), and two for the expected exceptions, diligently checking for the correct error codes and messages. It appeared to infer these details brilliantly from our docblock.

On the surface, these tests were impeccable. But here’s the rub: they only mirrored what was already explicitly laid out in the code and its comments. The real question lurked beneath: could these tests truly guard against deeper, more insidious coding blunders, the kind that might pass a superficial glance?

Into the Crucible: Code Mutations!

To really poke and prod at the LLM’s understanding, we decided to play mad scientist. We introduced deliberate “mutations” into our `ProductService` code. Each time, we wiped the slate clean, restarted Codex to purge any lingering context, and fired off the same innocent prompt: “Write unit tests for all public methods of the ProductService.”

Case 1: The Flipped Empty ID Condition

Our first sabotage involved subtly inverting the condition that checks for an empty product ID: instead of throwing an exception if the ID is empty, we made it throw if the ID is not empty. A classic “oops” moment.

Codex, bless its digital heart, was unfazed. It churned out tests that immediately flagged this logical inversion. Not only did it correctly identify the bug through its tests, but it even proposed the exact fix for the method! Chalk one up for obvious logical errors.

Case 2: The Flipped Product Search Condition

Next, we messed with the product search logic, flipping the condition that checks if the `ProductRepository` returned a product. Again, a simple logical inversion.

The outcome? Identical to Case 1. Codex saw right through our shenanigans, generated correct tests, and offered the appropriate fix. It seemed our LLM was a whiz at spotting when `if ($a === null)` accidentally became `if ($a !== null)`. Good to know it’s not entirely gullible.

Case 3: The Sneaky ‘Cost Price’ Swap

This is where things got juicy. Remember that `costPrice` we mentioned in the `Product` entity? We changed the `getProductPrice` method to return `return $product->getCostPrice();` instead of `return $product->getPrice();`.

And here’s the kicker: Codex generated tests that confirmed the mutated behavior! It happily asserted that the method now returned the cost price, not the actual price, even though our docblock explicitly stated the method’s purpose was to return the ‘price’. The LLM followed the flawed code like a loyal puppy, accepting the cost price as gospel. This, dear readers, was the bug that slipped through the cracks, a stark reminder that intent and documentation don’t always translate into LLM understanding.

The ‘Context Is King’ Revelation: Single Coding Session

Before drawing any harsh conclusions, we conducted a final, more nuanced experiment. What if the LLM had a deeper understanding of the initial requirements? This time, we asked Codex to write the code itself first, then generate the tests, all within a single, continuous session.

We fed it the initial requirements, and Codex dutifully produced a `ProductService` (though, interestingly, without the detailed docblock this time). It then generated a set of four correct unit tests, including the exception code checks.

Then, the moment of truth: we re-introduced our mutations within this same session, asking Codex to “Check whether tests for all public methods of ProductService still exist, and write them if they are missing.”

Remarkably, the game changed. Codex successfully handled the flipped conditions, automatically fixing the bugs and regenerating correct tests. And most tellingly, when we swapped `getPrice` for `getCostPrice`, Codex not only generated correct tests but also proactively fixed the `ProductService` code back to `return $product->getPrice();`!

This single-session run revealed a profound insight: an LLM, when given the full initial context of requirements and tasked with both code generation and testing, can indeed ‘remember’ and enforce those original requirements, acting as a more vigilant guardian against subtle business logic errors.

The Unvarnished Truth: LLM Tests Are a Double-Edged Sword

Our little experiment illustrates a crucial point: simply asking an LLM to “write tests” is often a naive gamble. While these AI-generated tests might compile and pass, they often merely mirror the existing code, warts and all. They’re fantastic at catching blatant logical inversions but can spectacularly miss the mark when the code deviates from complex, implicit business rules or explicit requirements not directly represented in code structure.

So, what’s a savvy developer to do?

• Context, Context, Context: The more context you provide—be it inline comments, detailed documentation, or even verbose prompt engineering describing the requirements—the better your LLM’s chances. But don’t mistake “better” for “perfect.”
• The Genesis Session: If your LLM writes the code, have it write the tests in the same session. This seems to imbue the AI with a deeper, more persistent understanding of the original requirements, allowing it to act as a more effective enforcer of intent.
• Never Skip the Human Review: This is the golden rule. LLM-generated tests are a fantastic starting point, a potent productivity booster. But they are not a substitute for human intellect, domain knowledge, and that critical eye that spots what the machine cannot. Blindly committing AI-generated tests is a recipe for certifying bugs, not squashing them.

Ultimately, LLMs are powerful tools for the modern developer’s arsenal. But like any tool, they must be wielded with understanding and discretion. They can absolutely help you write tests faster, but without clear requirements and the invaluable human touch, they’ll simply certify the code you have, rather than the behavior you truly need. So, the next time your LLM proudly presents a shiny new test suite, ask yourself: is this truly bulletproof, or is it just the emperor’s latest fashionable, yet ultimately transparent, garment?