7/23/2021 What Is Mutation Testing: Tutorial With Examples

What Is Mutation Testing:

Tutorial With Examples

This tutorial explains what is Mutation Testing, how to perform it, and
Mutation Testing types with examples:

What is Mutation Testing?

Mutation testing is a fault-based testing technique where variations of a

software program are subjected to the test dataset. This is done to
determine the effectiveness of the test set in isolating the deviations.

It sounds a little complicated, isn’t it? 

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 1/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

White-label Embedd

Self-Service BI - Designed for

Bold BI

What You Will Learn:

Mutation Testing
Mutation Testing Concepts
How To Do Mutation Testing
Mutation Testing types
#1) Value Mutation
#2) Statement Mutation
▾
#3) Decision Mutation
https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 2/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

Mutation Testing Tools

More on Mutation Testing
Points To Note
Conclusion
Recommended Reading

Mutation Testing
Mutation Testing (MT) goes a long way, back to the 70s where it was first proposed as
a school project. It was written off as it was very resource-intensive. However, as
humans continued to develop more advanced computers, it slowly made a comeback
and is now one of the most popular testing techniques.

Mutation testing definition

MT is also known as fault-based testing, program mutation, error-based

testing, or mutation analysis.

As the name suggests, mutation testing is a software testing type that is based on
changes or mutations. Miniscule changes are introduced into the source code to check
whether the defined test cases can detect errors in the code.

The ideal case is that none of the test cases should pass. If the test passes, then it
means that there is an error in the code. We say that the mutant (the modified version
of our code) lived. If the test fails, then there is no error in the code, and the mutant
was killed. Our goal is to kill all mutants.

Mutation testing also helps to test the quality of the defined test cases or the test
suites with a bid to write more effective test cases. The more mutants we can kill, the
higher the quality of our tests.

Mutation Testing Concepts

Before we discuss mutation testing further, let us explore some concepts
that we will come across.
▾

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 3/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

#1) Mutants: It is simply the mutated version of the source code. It is the code that
contains minute changes. When the test data is run through the mutant, it should
ideally give us different results from the original source code. Mutants are also called
mutant programs.

There are different types of mutants. These are as follows:

Survived Mutants: As we have mentioned, these are the mutants that are
still alive after running test data through the original and mutated variants of the
source code. These must be killed. They are also known as live mutants.
Killed Mutants: These are mutants that are killed after mutation testing.
We get these when we get different results from the original and mutated
versions of the source code.
Equivalent Mutants: These are closely related to live mutants, in that, they
are ‘alive’ even after running test data through them. What differentiates them
from others is that they have the same meaning as the original source code, even
though they may have different syntax.

#2) Mutators/mutation operators: These are what makes mutations possible,

they are on the ‘driver’s seat’. They basically define the kind of alteration or change to
make to the source code to have a mutant version. They can be referred to as faults
or mutation rules.

▾
#3) Mutation score: This is a score based on the number of mutants.

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 4/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

It is calculated using the below formula:

Note that, equivalent mutants are not considered when calculating the mutation
score. Mutation score is also known as mutation adequacy. Our aim should be to
achieve a high mutation score.

How To Do Mutation Testing

Step #1: Let’s write our Jasmine unit test.

Test Suite (Jasmine)

describe("User", function() {

it("should compare the two numbers from user input", function(){

expect(20).toBeGreaterThan(5);

})

});

Step #2: Write the original code.

Original code (Javascript)

const user_info = () => {

▾
mother_age = parseInt(prompt("Enter mother's age"))

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 5/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

daughter_age = parseInt(prompt("Enter daughter's age"))

if (mother_age > daughter_age) {

alert(`Daughter's age is ${daughter_age}. Mother's age is

${mother_age}. Welcome to the Mother-Daughter program`)

} else {

alert(`Daughter's age: ${daughter_age}, is more than mother's

age: ${mother_age}.

Please enter correct ages`)

user_info();

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 6/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

Step #3: We will then run the test through the original code to ensure that we don’t
have failed tests from the get-go. We should have some output that communicates
that we have indeed written a test with zero failures.

For example:

finished in 0.019s 1 spec, 0 failures, randomized with seed 31435

Original code’s result:

Daughter's age is 5. Mother's age is 20. Welcome to the Mother-

Daughter program

Step #4: Introduce the mutant. In this case, we change the greater-than operator
(mother_age > daughter_age) to a lesser than operator (mother_age <
daughter_age)
▾
Mutant code (Javascript)

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 7/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

const user_info = () =&amp;amp;gt;{

mother_age = parseInt(prompt("Enter mother's age"))

daughter_age = parseInt(prompt("Enter daughter's age"))

if (mother_age &amp;amp;lt; daughter_age) {

alert(`Daughter's age is ${daughter_age}. Mother's age is

${mother_age}. Welcome to the Mother-Daughter program`)

} else {

alert(`Daughter's age: ${daughter_age}, is more than mother's

age: ${mother_age}.

Please enter correct ages`)

user_info();

Step #5: We then run the test through the mutant code.

Here are the test results:

▾

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 8/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

finished in 0.017s 1 spec, 0 failures, randomized with seed 89555

Mutant code’s result:

Daughter's age: 5, is more than mother's age: 20. Please enter

correct ages

Step #6: Compare the results from the original version and the mutant version. In
this case, they are different, even if the same test suite was used. We have therefore
killed our mutant. Our test suite is therefore good to go.

Mutation Testing types

There are several types of mutations. These are explained below.

#1) Value Mutation

Here, we introduce a mutation by changing the parameter and/or
constant values, usually by +/- 1. ▾

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 9/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

Example: Original code (Javascript)

let arr = [2,3,4,5]

for(let i=0; i&amp;amp;lt;arr.length; i++){

if(i%2===0){

console.log(i*2)

If the above code was meant to multiply the even numbers where i<4, then value
mutation would mean changing the initialization to let i=1.

Example: Mutant code (Javascript)

let arr = [2,3,4,5]

for(let i=1; i&amp;amp;lt;arr.length; i++){

if(i%2===0){
▾
console.log(i*2)

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 10/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

#2) Statement Mutation

Here, we delete or duplicate a statement in a code block. We could also rearrange
statements in a code block.

In an if-else block, for example, we could delete the else part or even the
entire if-else block.

Example: Original code (Javascript)

let arr = [2,3,4,5]

for(let i=0; i&amp;amp;lt;arr.length; i++){

if(i%2===0){

console.log(i*2)

Example: Mutant code (Javascript)

let arr = [2,3,4,5]

for(let i=0; i&amp;amp;lt;arr.length; i++){

if(i%2===0){

console.log(i*2)

▾
console.log(i*2)

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 11/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

#3) Decision Mutation

The target here is the code that makes decisions, for example, value comparisons.
We can change > to < as in our Mother-Daughter program example.

Other operators that we can switch include the following:

Original operator Mutant operator

1 >=

2 >= ==

3 === ==

4 and or

5 || &&

Advantages of Mutation Testing(MT) includes:

MT covers a large part of the code’s logic.

We get to test specific parts of the code, and not just paths, branches, or
statements.
With MT, we can catch even errors that are easy to bypass.
It helps us to evaluate the quality of our test suite and adjust accordingly.
▾
Disadvantages of Mutation Testing (MT) includes:

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 12/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

It is resource-intensive as there is a huge number of mutants that are usually

generated for every code block.
We need to counter check the surviving mutants, as some are invalid.

Mutation Testing Tools

Tools come in handy to speed up the process of mutant generation. Here are some
tools that we can use in MT: Stryker, Jumble, PIT, and Insure++.

More on Mutation Testing

Let’s learn from an example: 

Say, there is a hospital site that lets new users register. It reads the Date of birth or
age of the patient. If it is greater than 14, assigns a general physician as their main
doctor.  To do so, it invokes the ‘general physician’ function that finds the available
doctor.

Now, there might be other functionality. Maybe, patients below 13 get assigned to a
pediatrician and so on. But we will only take the age-over-14 case.

This is what the code might look like:

1) Read Age

2) If age>14

3) Doctor= General Physician()

4) End if

Please note that the above lines of code are not specific to any programming language
and won’t run. It is just hypothetical.

As a tester, if my data-set is 14, 15, 0, 13 – some random numbers.

The target is to check if the data-set of the 4 values (14, 15, 0, and 3) is adequate to
identify all possible problems with this code.
▾
Also read => Tips to design test data before executing your test cases
https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 13/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

How does Mutation Testing achieve this?

First and foremost, you create mutants- variations of the program. A mutant is
nothing but a program that is written as a deviation. It contains a self-seeded fault.

Examples are:

Arithmetic operator replacement

Logical connector replacement
Statement removal
Relational operator replacement
Absolute value insertion, etc.

These replacements are also called ‘Mutation Operators.’

Let me show you examples:

Mutant #1: Relational operator replacement

1) Read Age

2) If age<14 ‘Changing the > with <’

3) Doctor= General Physician()

4) End if

Mutant #2:

1) Read Age

2) If age=14 ‘Changing the > with =’

3) Doctor= General Physician()

4) End if

Mutant #3:

1) Read Age
▾
2) If age>=14 ‘Changing the > with >=’

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 14/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

3) Doctor= General Physician()

4) End if

Mutant #4:

1) Read Age

2) If age<=14 ‘Changing the > with <=’

3) Doctor= General Physician()

4) End if

Mutant #5: Statement Removal

1) Read Age

2) If age=14

3) ‘remove the doctor assignment statement’

4) End if

Mutant #6: Absolute Value Insertion

1) Read Age

2) If age>14

3) Doctor= Mr.X (Absolute value insertion- let’s say X is a pediatrician)

4) End if

Mutant #7: Incorrect syntax

1) Read Age

2) If age%%14 (incorrect syntax)

3) Doctor=General Physician()

4) End if

Mutant #8: Does the same thing as the original test

1) Read Age
▾
2) If age> 14 & age>14 ‘means the same thing as age>14’

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 15/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

3) Doctor= General Physician()

4) End if

Once, all the mutants are created. They are subjected to the test data-set. Our set is
14, 15, 0 and 13. Which of these mutants will our data-set find?

Find out in the below table:

(Click on image for an enlarged view)

As you can see our data value 14 finds failures when it runs against, Mutant 2, 3 and
4. Or, 14 kills mutants 2, 3 & 4. But, it is ineffective against, 1, 6 and 8.

If your data-set kills all mutants, it is effective. Otherwise, include more or

better test data. It is not necessary for the each value in the data-set to kill all
mutants. But together, they should kill all. For example: 14 kills 2, 3 and 4. 15 kills 1, 2
and 4. And, so on.

What about 5, 7, and 8?

Mutant #5 – is the program instance that will fail irrespective of any data value you
give. This is because it will not do any programming for both valid and invalid values. ▾

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 16/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

Mutant #7 – will be a compile error. Or in the case of a scripting language an error

that will prevent execution.

Mutant #8 – is the same thing as the main program.

As you can see, the above mutants are not useful at all.

Therefore, mutants to avoid are:

Syntactically incorrect/‘Still-Born’ mutants. : You need syntactically

correct mutants ONLY. Example: Mutant 7
Equivalent Mutants: The ones that do the exact same thing as the original
program. Example: Mutant 8.
Trivial Mutant: Can be killed by any data-set. Example: Mutant 5

Points To Note

The number of mutants, even for a small program, can be many. It is a finite
number, but still very large. Due to this, a subset of mutants is usually used. It is
common to choose mutants randomly.
The mutation operators listed above is not an exhaustive list. There can be
many other variations. I have oversimplified the concept for easy understanding.
The mutation operators also differ with programming languages, design, and
specifications of the application.
If there are some mutants alive at the end of the test. It means either it is an
invalid mutant (like 5, 7, and 8) or the data-set was inadequate. If it is the later
one, go back and change it.
Mutation Test is a structural, white-box, and unit testing method. It uses
fault-injection or fault-seeding to generate its mutants.
There are many unit testing frameworks and tools that aid in automatic
mutation testing. Some of them are:
Jester for JUnit.
Pester for Python
MuClipse for Eclipse, etc.
▾
Are you thinking, if it takes this much effort, what is going to happen when I have to
test large samples of code?
https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 17/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

Mutation testing relies on two things:

Competent Programmer Assumption: If a programmer is competent

enough to write and test a small piece of code, he will be good at writing larger
programs too.
Coupling effect Assumption: If 2 units are combined to form a program
and each one is good in itself, then the combination is going to be good too.

So, it focuses on the smallest unit of code and places its faith in the
programmer’s skill to scale mutation testing to larger programs.

Conclusion
This tutorial covered Mutation testing definition, types, and steps to perform this
testing in detail with examples. We hope you have enjoyed reading and learning about
this interesting testing technique- Mutation Testing.

About the author: This article is written by STH team member Swati S.

Please share your comments, questions, and thoughts below.

Recommended Reading

Structural Testing Tutorial - What Is Structural Testing

What is Defect Based Testing Technique?

What Is Orthogonal Array Testing Technique (OATS)?

A Simple Approach for XML to Database Testing

Load Testing with HP LoadRunner Tutorials

Volume Testing Tutorial: Examples and Volume Testing Tools

▾
Functional Testing Vs Non-Functional Testing
https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 18/19
7/23/2021 What Is Mutation Testing: Tutorial With Examples

Pairwise Testing or All-Pairs Testing Tutorial with Tools and Examples

Last Updated: June 28, 2021

Next Read: What Scares Testers Most? - The Tester's Horror Stories (Halloween Special) »

© 2021 SOFTWARE TESTING HELP

Contact us | Advertise | Copyright Policy | Privacy Policy | Cookie Policy | Affiliate Disclaimer

https://www.softwaretestinghelp.com/what-is-mutation-testing/amp/ 19/19