TA Lecture-2

Test Designing & Development

 The Test Development Process
• Developing test material can be split into two distinct stages:
– Defining “what” needs to be tested
– Defining “how” the system should be tested

• This process can vary from organization to organization, can be very

formal or very informal with little documentation.

• The more formal, the more repeatable the tests, but it does depend
on the context of the testing being carried out.

• The Test Development Process consists of the following steps:

1. Defining test conditions
2. Specifying test cases
3. Specifying test procedures
4. Developing a Test execution schedule
 Test Conditions, Cases, Procedures and Schedule
What How How When

Test Test
Procedure Execution
Sourced Documentation

Specification Schedule

Manual Test
Test Script
Test Test Test Procedure
Cases
Condition Cases or Specifications
Automated
Priority Test Script
 Categories of Test Design Techniques
1. Specification-based Techniques (Black Box
Testing)
– derive test cases from the specification or a
model of the system
– testing of what a system should do

2. Structure-based Techniques (White Box

Testing)
– derive test cases from the code written
– testing based on how the specified behaviour
was implements

3. Experience-based Techniques
– derive test cases from the tester’s experience of
similar systems and general experience of
testing
Specification-based Techniques
(Black Box Testing)
Specification-based Techniques – Black Box Testing
• Testing without knowing the internal workings of the code.

• WHAT a system does, rather than HOW it does it.

• Typically used at System Test phase, although can be useful

throughout the test lifecycle.

• Also known as specification based testing.

• Applies for Functional and Non-Functional testing.

If Output = Expected Result, then pass

 Types of Black Box Testing
1. Equivalence Partitioning
2. Boundary Value Analysis
3. Decision Table Testing
4. State Transition Testing
5. Use Case Testing
 Equivalence Partitioning
• Equivalence Partitioning (EP) ─ A black box design technique in
which test cases are designed to execute representatives from
equivalence partitions.

• The equivalence partitions are ususally derived from the

requirements specification for input attributes that influence the
processing of the test obejct.

• Idea: Dividing the test input data into a range of values and
selecting one input value from each range.
 Equivalence Partitioning: Example-1
• There is a college that gives admissions to students based upon
their percentage only between 50 to 90 %. More and even less than
this range will not be accepted, and application will redirect user to
an error page.

Percentage: * Accept percentage value between 50 - 90

Equivalence Partitioning
Invalid Valid Invalid
< 50 50 – 90 > 90
 Equivalence Partitioning: Example-2
• Consider a system for postal services’ payment. Letters up to 100g
are called as ‘light’. Postal rates for sending the light letters up to
10g are $25. The next 40g will be charged by $35. Each next 25g up
to 100g should be charged by an extra $10.

Equivalence class Price Define and execute the test cases:

Letter has ‘-5’ g  Invalid
0–9g $25
Letter has ‘6’ g  $25
10 – 49 g $35
Letter has ‘33’ g  $35
50 – 74 g $45 Letter has ‘64’ g  $45
75 – 100 g $55 Letter has ‘88’ g  $55
100+ g Invalid Letter has ‘105’ g  Invalid
 Boundary Value Analysis
• Boundary value analysis (BVA) focuses on the boundary of the input
space to identify test cases.

• The rationale behind boundary value analysis is that errors tend to

occur near the extreme values of an input variable.

int[] arr = new int[10];

for(i = 1; i < 10; i++)
{
arr[i] = 1;
}
cout<<arr[0]; // 0

• Many bugs occur due to careless usage of indexes, operators (for

example < instead of <= ), etc.
 Value Selection in Boundary Value Analysis
• The basic idea in boundary value analysis is to select input variable
values at their:
– Minimum
– Just above the minimum
– A nominal value
– Just below the maximum

– Maximum
 Boundary Value Analysis: Example-1
• Consider a system that accepts ages from 18 to 56.

Age: * Accept age between 18 - 56

Boundary Value Analysis

Invalid Valid Invalid
(min-1) (min, min + 1, nominal, max - 1, max) (max + 1)
17 18, 19, 37, 55, 56 57
Boundary Value Analysis: Single Fault Assumption
• Boundary value analysis is also augmented by the single fault
assumption principle.

“Failures occur rarely as the result of the

simultaneous occurrence of two (or more) faults”

• Let us assume a program P accepting two inputs y1 and y2 such

that a ≤ y1 ≤ b and c ≤ y2 ≤ d.

y2
d

a b y1
Boundary Value Analysis: Single Fault Assumption
• In this respect, boundary value analysis test cases can be obtained
by holding the values of all but one variable at their nominal values,
and letting that variable assume its extreme values.

T = { <y1nom , y2min>,
<y1nom , y2min+1>,
<y1nom , y2nom>, For n variable to be checked:
<y1nom , y2max-1>, Maximum of 4n+1 test cases
<y1nom , y2max>,
<y1min , y2nom>,
<y1min+1 , y2nom>,
<y1max-1 , y2nom>,
<y1max , y2nom> }
 Boundary Value Analysis: Example-2
• Consider a Program for determining the Previous Date.
• Input: Day, Month, Year with valid ranges as-
1 ≤ Month≤12
1 ≤ Day ≤31
1900 ≤ Year ≤ 2000
Design Boundary Value Test Cases.

Expected Expected
Month Day Year Month Day Year
Output Output
6 15 1990 14/6/1990 6 30 1950 29/6/1950
6 15 1991 14/6/1991 6 31 1950 Invalid
6 15 1950 14/6/1950 1 15 1950 14/6/1950
6 15 1999 14/6/1999 2 15 1950 14/2/1950
6 15 2000 14/6/2000 11 15 1950 14/11/1950
6 1 1950 31/5/1950 12 15 1950 14/12/1950
6 2 1950 1/6/1950
 Decision Table Testing
• A Decision Table is the method used to build a complete set of test
cases without using the internal structure of the program in
question.

• This testing technique is useful when a variety of actions exists and

we need to take a particular action depending on the existing
conditions.

• Conditions = Input.

• Actions = Output.
Decision Table Testing: Log In Facebook Example

Email address or phone number

Password

Email or Phone False True False True

Password False False True True

Error:
Error:
Error:
Error: Error: Facebook
Please
Please
Expected Results Incorrect
Incorrect Incorrect Page is
Confirm
Confirm
Email
Email Email Processed
Password
Password
 Components of a Decision Table
• A decision table has 4 parts:
1. Condition stub
2. Condition entry
3. Action stubs
4. Action entry
 Types of Condition Entries
1. Limited Entry : In the limited entry decision tables, the
condition entries are restricted to binary values.

2. Extended Entry : In the extended entry decision table, the

condition entries have more than two values.
Decision Table Testing: Log In Facebook Example

Condition Stubs Condition Entry

Conditions Rule 1 Rule 2 Rule 3 Rule 4

Email or Phone False True False True

Password False False True True

Error:
Error:
Error:
Error: Error: Facebook
Please
Please
Expected Results Incorrect
Incorrect Incorrect Page is
Confirm
Confirm
Email
Email Email Processed
Password
Password

Action Stubs
Action Entry

A rule = A test case

No. of combination = No. of Condition1 value * No. of condition2 value
= 2 * 2 = 4 rules or 4 test cases
 Decision Table Testing: Log In Facebook Example
Conditions Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6 Rule 7 Rule 8 Rule 9

Email or Invalid Invalid Valid Valid Valid

Blank Blank Blank Invalid
Phone

Password Blank Invalid Valid Blank Invalid Valid Blank Invalid Valid

Error: Error: Error:

Error: Error: Error: Error: Error:
Expected Please Please Please
Incorrect
Confirm
Incorrect Incorrect Incorrect Incorrect
Confirm Confirm ─
Results Email Email Email Email Email
Password Password Password

Login Login Login Login Login Login Login Login Facebook

Show page page page page page page page page page page

No. of Condition1 and 2 are blank, Invalid, valid as 3 possible values.

No. of combination = No. of Condition1 value * No. of condition2 value

= 3 * 3 = 9 rules or 9 test cases
 Decision Table Testing: Triangle Problem
• Conditions:
– a, b, c form a triangle?
– a = b?
– a = c?
– b = c?

• Actions:
– Not a Triangle
– Scalene
– Isosceles
– Equilateral
– Impossible
 Decision Table Testing: Triangle Problem
Conditions Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6 Rule 7 Rule 8 Rule 9
c1: a, b, c form a ∆? F T T T T T T T T
c2: a = b? - T T T T F F F F
c3: a = c? - T T F F T T F F
c4: b = c? - T F T F T F T F
a1: Not a triangle X
a2: Scalene X
a3: Isosceles X X X
a4: Equilateral X
a5: Impossible X X X
 State Transition Testing
• It is a test design techniques based on the theory of finite state
machine.

• State represents attributes and behaviour

• Transition represents movement from one valid state to another

valid state.

• Example: On > Off > On Turn switch on

OFF ON

Turn switch off

 State Transition Testing
• State transition testing is used when the system is defined in a finite
number of states.

• On giving input to the System Under Test (SUT):

– Change the state of the SUT
– Produce outputs

• State transition testing tests the valid transitions from one state to
another and also the invalid transitions.
 State Transition Testing: Car Example
• States are Stopped, Accelerating, Constant speed and Braking

Accelerating

Constant
Stopped
Speed

Braking
 State Transition Testing: Car - State Table
States Inputs Next State
Stopped Press brake pedal more Accelerating
Accelerating Keep gas pedal constant Constant speed
Accelerating Switch to brake pedal Braking
Constant Speed Press gas pedal more Accelerating
Constant Speed Switch to brake pedal Braking
Braking Switch to gas pedal; Press Accelerating
gas pedal more
Braking Switch to gas pedal Constant Speed
Braking Keep braking Stopped
Stopped ? Constant speed
Stopped ? Braking
Accelerating ? Stopped
Constant Speed ? Stopped
 State Transition Testing: Car – Test Case
• Test case 1 – Stopped > Accelerating > Constant speed > Braking > Stopped
• Test case 2 – Stopped > Accelerating > Braking > Accelerating > Braking >
Stopped
• Test case 3 – Stopped > Accelerating > Constant Speed > Accelerating >
Braking > Accelerating > Constant speed > Braking > Constant speed >
Braking > Stopped

Accelerating

Constant
Stopped
Speed

Braking
 Use Case Testing
• This technique helps identify test cases that cover the entire
system, on a transaction by transaction basis, from start to finish.

• A use case is a description of a particular use of the system by an

actor.

• Used widely in developing tests at system or acceptance level.

 Use Case Testing
What is a use case?
• It is a list of steps to achieve a goal in the system.

• These steps define the interactions between the actor and the
system.

• The use case captures the functional requirements of the

system.

• It defines the outcomes of errors during system use.

• Use case defines the main scenario (and optional exceptional

scenarios).
Use Case Testing: ATM Example
 Use Case Testing: ATM Example
Step Description
Main Success Scenario: 1 A: Inserts card
2 S: Validates card and asks for pin
A: Actor
S: System 3 A: Enters pin
4 S: Validates pin
5 S: Allows access to account

Extensions: 2a Card not valid

S: Display message and reject card
4a Pin not valid
S: Display error, go back to step 3
4b Pin invalid 3 times
S: Eat card and exit
Structure-based Techniques
(White Box Testing)
Structure-based Techniques (White Box Testing)
• White box testing (also known as glass box and structural testing) is
a method of testing software that tests the internal structure or
working of an application.

• Test cases can be designed to reach every branch in the code and to
exercise each condition.

• Typically done during unit testing.

Input Output
 Types of White Box Testing
1. Statement Coverage
2. Decision/Branch Coverage
3. Condition Coverage

4. Path Coverage
 Statement Coverage
• Statement coverage methodology:
– design test cases so that every statement in a program is executed at
least once.

• The principal idea:

– unless a statement is executed, we have no way of knowing if an error
exists in that statement.

• Observing that a statement behaves properly for one input value:

– no guarantee that it will behave correctly for all input values.
 Statement Coverage: Example-1
Read A
Read B
IF A > B THEN
Print “A is greater than B”
ELSE
Print “B is greater than A”
ENDIF

Case 1: If A = 7, B= 3 Case 2: If A = 4, B= 8
No of statements Executed = 5 No of statements Executed = 6
Total statements = 7 Total statements = 7
Statement coverage = 5 / 7 * 100 Statement coverage = 6 / 7 * 100
= 71.42 % = 85.20 %
∴ Minimum number of test cases required for 100% statement coverage = 2
 Statement Coverage: Example-2
Wait for card to be inserted
IF card is valid THEN
display “Enter PIN number”
IF PIN is valid THEN
select transaction
ELSE
display “Invalid PIN”
ELSE
reject card

No. of test cases required for

100% Statement coverage= 3

Number of test cases required for 100% statement coverage = 1 + count of ELSE
 Statement Coverage: Example-3
Read A
IF A > 0 THEN
IF A == 21 THEN
Print “Key”
ENDIF
ENDIF

No. of test cases required for 100% Statement coverage = 1

 Statement Coverage: Example-4
Read A
Read B
IF A > 0 THEN
IF B == 0 THEN
Print “No values”
ELSE
Print B
IF A > 21 THEN
Print A
ENDIF
ENDIF
ENDIF

No. of test cases required for 100% Statement coverage = 2

 Statement Coverage: Example-5
Read A
Read B
IF A < 0 THEN
Print “A negative”
ELSE
Print “A positive”
ENDIF
IF B < 0 THEN
Print “B negative”
ELSE
Print “B positive”
ENDIF

No. of test cases required for

100% Statement coverage = 2 (∵ un-nested conditions)
 Decision/Branch Coverage
• “Branch” in a programming language is like the “IF statement”.

• An IF statement has two branches: True and False.

• So in Branch coverage (also called Decision coverage), we validate

whether each branch is executed at least once.

• Test cases are designed such that:

– different branch conditions is given true and false values in turn.

• Branch testing guarantees statement coverage:

– a stronger testing compared to the statement coverage-based testing.
 Branch Coverage: Example-1
Read A
Read B
IF A > B THEN
Print “A is greater than B”
ELSE
Print “B is greater than A”
ENDIF

∴ Minimum number of test cases required for 100% branch coverage = 2

 Branch Coverage: Example-2
Wait for card to be inserted
IF card is valid THEN
display “Enter PIN number”
IF PIN is valid THEN
select transaction
ELSE
display “Invalid PIN”
ELSE
reject card

No. of test cases required for

100% Branch coverage= 3

Number of test cases required for 100% branch coverage = 1 + count of IF

 Branch Coverage: Example-3
Read A
IF A > 0 THEN
IF A == 21 THEN
Print “Key”
ENDIF
ENDIF

No. of test cases required for 100% branch coverage = 3

 Branch Coverage: Example-4
Read A
Read B
IF A > 0 THEN
IF B == 0 THEN
Print “No values”
ELSE
Print B
IF A > 21 THEN
Print A
ENDIF
ENDIF
ENDIF

No. of test cases required for 100% branch coverage = 4

 Branch Coverage: Example-5
Read A
Read B
IF A < 0 THEN
Print “A negative”
ELSE
Print “A positive”
ENDIF
IF B < 0 THEN
Print “B negative”
ELSE
Print “B positive”
ENDIF

No. of test cases required for

100% branch coverage = 2 (∵ un-nested conditions)
 Condition Coverage
• Condition coverage testing is a type of testing that tests all
the conditional expressions in a program for all possible
outcomes of the conditions.

• Condition coverage (also called predicate coverage) testing

tests the conditions independently of each other.

• Condition coverage vs. branch coverage:

– In branch coverage, all conditions must be executed at least
once.
– On the other hand, in condition coverage, all possible outcomes
of all conditions must be tested at least once.
 Condition Coverage: Example-1
Read A
Read B
IF A > B THEN
Print “A is greater than B”
ELSE
Print “B is greater than A”
ENDIF

Minimum number of test cases required for 100% branch coverage = 2

Minimum number of test cases required for 100% condition coverage = 2

 Condition Coverage: Example-2
Read A Test case Final
A>0 B < 10
Read B number output
IF (A>0 || B<10) THEN Don’t
1 True True
Print “Valid input” care
ELSE 2 False True True
Print “Invalid input” 3 False False False
ENDIF

Minimum number of test cases required for 100% branch coverage = 2

Minimum number of test cases required for 100% condition coverage = 3

 Condition Coverage: Example-3
Read A
Read B Min. test cases for:
IF ((A>0 || B<10) && (A+B < 15)) THEN
Branch Coverage = 2
Print “Valid input”
ELSE Condition Coverage = 5
Print “Invalid input”
ENDIF

Test case No A>0 B < 10 A+B < 15 Final output

1 True Don't care True True
2 True Don't care False False
3 False True True True
4 False True False False
5 False False Don't care False
 Path Coverage
• Design test cases such that:
– all linearly independent paths in the program are executed at least
once.

• Defined in terms of
– control flow graph (CFG) of a program.

• A control flow graph (CFG) describes:

– the sequence in which different instructions of a program get
executed.
– the way control flows through the program.

• No of test cases = Cyclomatic Complexity of CFG = E – N + 2P

 Path Coverage: Example-1
1 Read A
2 Read B
3 IF A > B THEN
4 Print “A is greater than B”
5 ELSE
Print “B is greater than A”
6 ENDIF

Cyclomatic complexity = E – N + 2P = 6 - 6 + 2*1 = 2

∴ Minimum number of test cases required for 100% path coverage = 2

 Path Coverage: Example-2
1 Read A
2 Read B
3 IF A > 0 THEN
4 IF B == 0 THEN
5 Print “No values”
6 ELSE
Print B
7 IF A > 21 THEN
8 Print A
9 ENDIF
10 ENDIF
11 ENDIF

Cyclomatic complexity = E – N + 2P = 13 - 11 + 2*1 = 4

∴ Minimum number of test cases required for 100% path coverage = 4

Experience-based Techniques
 Experience-based Techniques
• Experience-based Techniques uses tester’s skill, intuition and
experience of similar application and techniologies.

• These techniques are useful in identifying special tests which are

not easily captures by formal techniques.

• These techniques are usually applied after formal techniques.

• There are few scenarios where these are the only option to apply:
– Specifications are poorly defined.
– Time pressure.
– Testing team is not formally trained on testing fundamentals.
 Types of Experience-based Techniques
• Error guessing is a technique that should always be used as a
complement to other more formal techniques. The success of error
guessing is very much dependent on the skill of the tester, as good
testers know where the defects are most likely to lurk.

• Exploratory testing is a hands-on approach in which testers are

involved in minimum planning and maximum test execution.