Mutation

Mutation Testing
• Error seeding
– Introduce artificial faults into a program in some suitable random fashion – Test the program with a test suite – Let r the ratio defined by r = n1 / n2 where
• n1 = # of the artificial faults found during error seeding • n2 = # of the artificial faults introduced during error seeding
1

Mutation Testing
• Use error seeding to estimate the number of faults
– Assume that we test a program with a test suite
• Let n be # of faults found during testing

– How many faults remain in the program?
• n1 / r

2

1

Mutation Testing
• Mutants
int getMax(int x, int y) { int max; if (x > y) max = x; else max = y; return max; } } int getMax(int x, int y) { int max; if (x >= y) max = x; else max = y; return max; } int getMax(int x, int y) { int max; if (x > y) max = x; else max = x; return max;

3

Mutation Testing
• Equivalent mutants
int getMax(int x, int y) { int max; if (x > y) max = x; else max = y; return max; } } int getMax(int x, int y) { int max; if (x >= y) max = x + 1; else max = y; return max; } int getMax(int x, int y) { int max; if (x > y) max = x * 1; else max = y; return max;

4

2

Mutation Testing
• Assumptions
– Competent programmer assumption
• Programmers are not so stupid • They produce programs that are close to being correct • Faults should be detectable as small deviations from the intended program

– Coupling effect assumption
• If a test suite kills a mutant, it also can kill mutants of mutants
5

Mutation Testing
• Mutation analysis
– We want to evaluate the quality of a test suite
• Assume that we are given a program with a test suite • Make a set of mutants • Test the program and a mutant with the test suite
– If the outputs are different, the mutant is said to be killed by the test suite – Otherwise, the mutant is live » The test suite is inadequate » The mutant is equivalent to the original program
6

3

Mutation Testing
• Mutation Adequacy
– Evaluate the...