Mutation Score

Mutation score measures how effectively a test suite detects seeded code changes, called mutants. It is typically expressed as the proportion of non-equivalent mutants that the tests kill.

Mutation testing evaluates test quality by making small, deliberate changes to production code. Each changed version is a mutant, and the test suite is run against it to see whether the change is detected.

A mutant is killed when at least one test fails because of the injected change. A surviving mutant means the test suite did not detect that change, although the mutant may be behaviorally equivalent to the original program and therefore impossible for the tests to distinguish.

The mutation score is commonly calculated as `killed mutants / (total mutants - equivalent mutants)`. For example, if 80 mutants are created, 5 are considered equivalent, and tests kill 60 of the remaining 75, the score is `60 / 75 = 80%`.

For a Java backend engineer, mutation score is useful because ordinary line or branch coverage can show that code executed without showing whether tests would fail when behavior changes. Mutation testing therefore focuses on the fault-detection capability of the tests rather than execution alone.

Key points

• A mutant is a deliberately modified version of the program.
• A killed mutant causes at least one test to fail.
• A surviving mutant is not detected by the test suite.
• Equivalent mutants behave indistinguishably from the original for the relevant behavior.
• Mutation score commonly excludes equivalent mutants from its denominator.
• High code coverage does not necessarily imply a high mutation score.

Interview questions

What is mutation score?

Mutation score measures how many relevant mutants are detected by a test suite. A mutant is killed when executing the tests causes a failure that exposes the modification. The denominator normally excludes equivalent mutants because those cannot be distinguished from the original behavior. A higher score generally indicates that the tests detect more of the simulated implementation changes.

How do you calculate mutation score?

A common formula is `killed mutants / (total mutants - equivalent mutants)`. For example, 60 killed mutants out of 75 non-equivalent mutants gives an 80% mutation score. The exact reporting convention can vary between mutation-testing tools, so the tool's treatment of equivalent or excluded mutants should be checked.

Why isn't high code coverage enough to demonstrate good tests?

Coverage can show that a statement or branch was executed, but execution alone does not prove that the test verifies the resulting behavior. A test can execute a line while making weak or no assertions about its result. Mutation testing changes the implementation and checks whether the tests notice. This makes mutation score a complementary measure of test effectiveness rather than a replacement for coverage.

What is an equivalent mutant, and why is it a problem?

An equivalent mutant is a modified program whose externally observable behavior remains equivalent to the original for the relevant inputs. Because the tests cannot legitimately distinguish it from the original, it cannot be killed through an ordinary behavioral test. Equivalent mutants make mutation scores harder to interpret because counting them as ordinary surviving mutants can unfairly reduce the score. Detecting equivalent mutants can also require substantial human analysis.

How would you use mutation testing in a Java backend project?

I would apply mutation testing to important production logic, especially areas where incorrect behavior could escape ordinary tests. I would examine surviving mutants to determine whether they expose missing assertions, missing edge cases, or merely unimportant behavior. I would use the results alongside unit, integration, branch, and other quality signals rather than treating one mutation threshold as sufficient. For Java projects, I would also consider build-time cost and configure the mutation scope so feedback remains practical.