Mutation Testing Explained: Why Code Coverage Lies to You
95% code coverage tells you almost nothing about whether your tests would catch a real bug. Mutation testing does. How it works, how to run it with Stryker.NET, and how AI closes the gap it finds.
Code coverage answers one question: did any test execute this line? It says nothing about whether a test would notice if that line's behavior were wrong. A test suite can have 100% coverage and catch zero real bugs — every line runs, nothing actually asserts the right thing happened. Mutation testing is the tool that exposes this gap.
How mutation testing works
A mutation testing tool takes your source code and automatically introduces small, deliberate bugs — mutants — one at a time: flipping a `>` to `>=`, changing `&&` to `||`, deleting a line, negating a boolean. It then reruns your test suite against each mutant.
- Mutant killed — at least one test failed. Good: your tests would have caught this bug.
- Mutant survived — every test still passed. Bad: this exact class of bug could ship and nothing would notice.
Every surviving mutant is a concrete, specific gap: "if this comparison operator were wrong, nothing would fail." That's a precise, actionable finding — far more useful than a vague "coverage is only 82%" number.
Running it with Stryker.NET
dotnet tool install -g dotnet-stryker
cd YourProject.Tests
dotnet strykerThe output is a mutation score (percentage of mutants killed) plus a line-by-line report. A codebase with 90% line coverage frequently sits at 60-70% mutation score on first run — that gap is real, uncovered risk that coverage numbers were hiding.
Closing the gap: feeding survivors back to AI
This is where mutation testing and AI-assisted testing combine well. Instead of manually reasoning about why a mutant survived, paste the specific surviving mutation and the code around it, and ask for a test that would kill it.
Context: Stryker.NET reports this surviving mutant in OrderService.cs:
Original: if (quantity <= 0) throw new ArgumentException(...)
Mutant: if (quantity < 0) throw new ArgumentException(...)
No existing test in OrderServiceTests.cs fails against this mutant.
Task: Write the minimal xUnit test that fails against the mutant above but passes against the original code.
Constraints:
- Target exactly this boundary condition (quantity == 0) — don't write a broad test that happens to also cover it.
- Follow the existing test file's naming convention and AAA (Arrange-Act-Assert) structure.
- One test method, no test-data-driven [Theory] unless the existing file already uses that pattern for similar cases.Common mutant operators
| Operator | What it changes | What surviving it usually means |
|---|---|---|
| Conditional boundary | `<` ↔ `<=`, `>` ↔ `>=` | Boundary/edge-case values aren't tested |
| Logical operator | `&&` ↔ `||` | Compound conditions aren't tested with mixed true/false inputs |
| Arithmetic operator | `+` ↔ `-`, `*` ↔ `/` | No test checks the actual computed value, only that a value exists |
| Statement deletion | Removes a line entirely | The line has no observable effect any test checks for — possibly dead code, possibly a real gap |
| Return value | Returns a fixed/default value instead of computed one | No test asserts on the specific return value, only its type or non-null-ness |
Where to use it (and where not to)
Mutation testing is slow — it reruns your suite once per mutant, so full-codebase runs on a large project can take hours. Run it selectively: on core domain logic, financial calculations, and security-sensitive code where a silent behavioral bug is expensive; skip it for UI glue and thin controller code where the risk-to-cost ratio doesn't justify it. Most teams run it in CI on a schedule (nightly/weekly) rather than on every commit.
This pairs directly with property-based testing — properties are frequently what's needed to kill the mutants that example-based tests miss, because they check general invariants rather than specific recorded outputs.