Lecture 1 : Introduction to Software Testing 1

Lecture 1 : Introduction to Software Testing

CA 7503 – Software Testing

Deivamani Mallayya
Department of Information Science and Technology
CEG Campus, Anna University

July 4, 2019
Lecture 1 : Introduction to Software Testing 2

Software is Everywhere
Lecture 1 : Introduction to Software Testing 3

Software Engineering – An Overview

▪ Phases
▪ 3Ps : Process, Product and
Project
▪ Process Models
▪ Traditional Vs Object-Oriented
methodology

Source:
https://www.skyindya.com/blog/web-development/software-development-methodologies-
infographics/
Lecture 1 : Introduction to Software Testing 4

Process Models
▪ Waterfall Model
▪ Incremental Models
▪ Incremental
▪ RAD
▪ Evolutionary
▪ Evolutionary Models
▪ Prototype
▪ Spiral

Source: https://www.lumitex.com/blog/prototyping-methodology
Lecture 1 : Introduction to Software Testing 5

As a Programmer ASK these Questions??

▪ Does my software work?

▪ Does my software meet its specification?
▪ I’ve changed something, does it still work?
▪ How can I become a better programmer?

Testing
Lecture 1 : Introduction to Software Testing 6

Motivation (1)
Disney Lion King, 1994 – 1995
▪ Disney’s first multimedia CD-ROM game for
children.
▪ Sales were huge.
▪ On December 26, Disney's customer support
phones began to ring, and ring, and
ring….from angry parents with crying
children who couldn't get the software to
work. Numerous stories appeared in
newspapers and on TV news.
▪ Disney failed to properly test the software
on the many different PC models available
on the market.
Source: https://greekcity.com/product/the-lion-king-2/
Lecture 1 : Introduction to Software Testing 7

Motivation (2)
Intel Pentium Floating-Point Division Bug,
1994
▪ Enter the following equation into your PC's calculator:

(4195835 / 3145727) * 3145727 - 4195835

▪ If the answer is zero, your computer is just fine. If you get

anything else, you have an old Intel Pentium CPU with a

floating-point division bug.

▪ Dr. Thomas R. Nicely of Lynchburg (Virginia) College

Source:
https://en.wikipedia.org/wiki/Pentium_FDIV_bug
unexpected result from one of his experiments to an

incorrect answer by a division problem solved on his

Pentium PC.
Lecture 1 : Introduction to Software Testing 8

Motivation (3)
NASA Mars Polar Lander, 1999
▪ On December 3, 1999, NASA's Mars Polar Lander disappeared during its
landing attempt on the Martian surface.
▪ A Failure Review Board investigated the failure and determined that the most
likely reason for the malfunction was the unexpected setting of a single data
bit.
▪ Most alarming was why the problem wasn't caught by internal tests.
Lecture 1 : Introduction to Software Testing 9

Motivation (3)
▪ In theory, the plan for landing was this: As the lander fell to the surface, it was to deploy a parachute to

slow its descent.

▪ A few seconds after the chute deployed, the probe’s three legs were to snap open and latch into position

for landing. When the probe was about 1,800 meters from the surface, it was to release the parachute and

ignite its landing thrusters to gently lower it the remaining distance to the ground.

▪ To save money, NASA simplified the mechanism for determining when to shut off the thrusters. In lieu of

costly radar used on other spacecraft, they put an inexpensive contact switch on the leg’s foot that set a

bit in the computer commanding it to shut off the fuel. Simply, the engines would burn until the legs

“touched down.”

▪ Unfortunately, the Failure Review Board discovered in their tests that in most cases when the

legs snapped open for landing, a mechanical vibration also tripped the touch-down switch, setting the fatal

bit.
Lecture 1 : Introduction to Software Testing 10

Motivation (4)
Malaysia Airlines jetliner, August 2005
▪ As a Malaysia Airlines jetliner cruised from Perth, Australia, to

Kuala Lumpur, Malaysia, it suddenly took on a mind of its own

and zoomed 3,000 feet upward.

▪ The captain disconnected the autopilot and pointed the Boeing

777’s nose down to avoid stalling, but was jerked into a steep

dive. He controlled back sharply on both engines, trying to slow

the plane. Instead, the jet raced into another climb.
▪ The crew eventually regained control and manually flew their

177 passengers safely back to Australia.

▪ Investigators quickly discovered the reason for the plane’s roller-

coaster ride 38,000 feet above the Indian Ocean. A defective

software program had provided incorrect data about the

aircraft’s speed and acceleration, confusing flight computers.

The computers had also failed, at first, to respond to the pilot’s

commands.
Lecture 1 : Introduction to Software Testing 11

Motivation (5)
The Y2K (Year 2000) Bug, circa 1974
▪ Dave was working on a payroll system for his company.
The computer he was using had very little memory for
storage, forcing him to conserve every last byte he
could.
▪ One method he used was to shorten dates from their 4-
digit format, such as 1973, to a 2-digit format, such as
73. Dave could save lots of expensive memory space.
▪ 1995, Dave’s program was still being used, Dave was
retired, and no one knew how to get into the program to
check if it was Y2K compliant, let alone how to fix it.
▪ It’s estimated that several hundred billion dollars were
spent, worldwide, to replace or update computer Source:
https://whysurfswim.com/2017/03/05/5-daily-dose-the-y2k-disaster/

programs such as Dave’s, to fix potential Year 2000

failures.
 Lecture 1 : Introduction to Software Testing 12

Need for Software Testing

Software testing is Software testing Software testing It is required to At the end software No matter how well
required to check ensures that the ensures that the make sure that the is developed by a the software design
the reliability is free
system product is in line final product is team of human looks on paper,
user friendly once the
of the software from any bug with the developers development starts
that can cause any requirement of with different and you start
kind of failure the client perceptions. It testing the product
is not possible to you will definitely
create software find lots of
with zero defects in the
defects without design.
incorporating
software testing in
the development
cycle.
Lecture 1 : Introduction to Software Testing 13

What is software testing?

▪ IEEE defines software testing as
▪ A process of analyzing a software item to detect the differences
between existing and required conditions (that is
defects/errors/bugs) and to evaluate the features of the software
item.
▪ Note that...
...one needs to know the required conditions
...one needs to be able to observe the existing conditions
▪ Testing focuses on behavioral (what the program does) and
structural (how the program is) aspects
Lecture 1 : Introduction to Software Testing 14

Program behavior

Fault of commission occurs when we enter something into a representation that is

incorrect.
Faults of omission occur when we fail to enter correct information.
Of these two types, faults of omission are more difficult to detect and resolve
Lecture 1 : Introduction to Software Testing 15

Program behavior and testing

Lecture 1 : Introduction to Software Testing 16

Functional vs structural testing

Lecture 1 : Introduction to Software Testing 17

What is a Software Bug?

▪ Product Specification: “an agreement among the software development
team. It defines the product they are creating, detailing what it will be,
how it will act, what it will do, and what it won't do”
▪ Formally, we say that a software bug occurs when one or more of the
following five rules are true: when the software
▪ doesn't do something that the product specification says it should do.
▪ does something that the product specification says it shouldn't do.
▪ does something that the product specification doesn't mention.
▪ doesn't do something that the product specification doesn't mention but
should.
▪ is difficult to understand, hard to use, slow, or in the software tester's eyes
will be viewed by the end user as just plain not right.
Lecture 1 : Introduction to Software Testing 18

What is a Software Bug?

▪ Apply the rules to Windows calculator

▪ Note about Rule 5:

▪ It would be impossible to write software that every user
thought was perfect.

▪ As a software tester: be thorough, use your best judgment,

and be reasonable
Lecture 1 : Introduction to Software Testing 19

Bugs a.k.a. …

▪ Defect ▪ Failure
▪ Fault ▪ Inconsistency
▪ Problem
▪ Product Anomaly
▪ Error
▪ Product Incidence
▪ Incident
▪ Feature :-)
▪ Anomaly
▪ Variance
Lecture 1 : Introduction to Software Testing 20

Why do Bugs Occur?

▪ Bugs are caused for numerous reasons, but in this sample project
analysis, the main cause can be traced to the specification.
Lecture 1 : Introduction to Software Testing 21

Why do Bugs Occur?

1. The number one cause of software bugs is the specification, why?
▪ It is not written
▪ It is not complete
▪ It is not clear
▪ It is constantly changing
▪ It is not communicated well to the development team
2. The next largest source of bugs is the design. (why?)
3. Coding errors
▪ Software complexity
▪ Poor documentation
▪ Limited time
▪ Programmer skills
Lecture 1 : Introduction to Software Testing 22

Symptoms and Root Causes of Bugs

▪ Inaccurate understanding of end user needs
▪ Inability to deal with changing requirements
▪ Modules that don’t fit together
▪ Software that is hard to maintain or extend
▪ Late discovery of serious project flows
▪ Poor software quality
▪ Unacceptable software performance
▪ Team members in each other’s way, making it impossible to
reconstruct who changed what, when, where and why
▪ An untrustworthy build-and-release process
Lecture 1 : Introduction to Software Testing 23

Root Causes of Project Failures

▪ Ad hoc requirement management
▪ Ambiguous and imprecise communication
▪ Brittle (Fragile) architectures
▪ Overwhelming complexity
▪ Undetected inconsistencies in requirement, design, and implementation
▪ Insufficient testing
▪ Subjective assessment of project status
▪ Failure to attack risk
▪ Uncontrolled change propagation
▪ Insufficient automation
Lecture 1 : Introduction to Software Testing 24

The Cost of Bugs

Lecture 1 : Introduction to Software Testing 25

What Does a Software Tester Do?

▪ The goal of a software tester is to find bugs, find them as

early as possible, and make sure they get fixed.
Lecture 1 : Introduction to Software Testing 26

What Makes a Good Software Tester?

▪ Explorer: loves to get a new piece of software, install it and see what
happens
▪ Troubleshooter: loves puzzles
▪ keep trying
▪ Make decisions about what to test and how long it will take?
▪ Diplomatic: bearer of bad news
▪ Creative
▪ Strive for perfection
▪ Persuasive: convenes management about the importance of a bug
▪ Software programming skills
▪ Expert in some non-computer field
Lecture 1 : Introduction to Software Testing 27

Fault, Error, and Failure

▪ Error: a static defect in the software’s source code

▪ Cause of a problem
▪ Fault: is a deviation from the required operation of system or
subsystem
▪ Erroneous program state caused by execution of the defect
▪ Failure: External, incorrect behavior with respect to the
requirements or other descriptions of the expected behavior
▪ Propagation of erroneous state to the program outputs
Lecture 1 : Introduction to Software Testing 28

Fault, Error, and Failure

Source:
Foundations of Software Testing by Aditya P. Mathur
Lecture 1 : Introduction to Software Testing 29

Fault & Failure Model

Three conditions necessary for a failure to be observed
▪ Reachability : The location or locations in the program that contain the fault
must be reached
▪ Infection : The state of the program must be incorrect
▪ Propagation : The infected state must propagate to cause some output of
the program to be incorrect
Lecture 1 : Introduction to Software Testing 31

How Important is Testing?

Lecture 1 : Introduction to Software Testing 32

Testing in the 21st Century

▪ Safety critical, real-time software
▪ Embedded software
▪ Enterprise applications
▪ Security
▪ Web
▪ Mobile

Software testing is getting more important

What are we trying to do when we test ?

What are our goals ?
Lecture 1 : Introduction to Software Testing 33

How would you test a ballpoint pen?

▪ Does the pen write?
▪ Does it work upside down?
▪ Does it write in the correct color?
▪ Do the lines have the correct thickness?
▪ Does the click-mechanism work? Does it work after 100,000 clicks?
▪ Is it safe to chew on the pen?
▪ Is the logo on the pen according to company standards?
▪ Does the pen write in -40 degree temperature?
▪ Does the pen write underwater?
▪ Does the pen write after being run over by a car?
▪ Which are relevant? Which are not relevant? Why (not)?
Lecture 1 : Introduction to Software Testing 34

To ponder...
▪ Discuss: Who should write tests? Developers? The person who
wrote the code? An independent tester? The customer? The user?
Someone else?
▪ Discuss: When should tests be written? Before the code? After the
code? Why?
Lecture 1 : Introduction to Software Testing 35

Summary

▪ Testing is the most time consuming and expensive part of software

development

▪ Not testing is even more expensive

▪ Having too little testing effort early increases the testing cost

▪ Planning for testing after develop is prohibitively expensive

▪ A tester’s goal is to eliminate faults as early as possible

▪ Testing improves software quality, reduce cost, and preserve customer

satisfaction
Lecture 1 : Introduction to Software Testing 36

Recall
1. What is a fault of commission?

2. What is a fault of omission?

3. What may Faults (defect, bugs) lead to?

4. What may Errors (mistakes) lead to?

5. What may Failures lead to?

6. Can tests prove the existence of failures?

7. Can tests prove the absence of failures?

8. What's the goal of a software tester?

9. Give several reasons why the product specification is usually the largest
source of bugs in a software product.