Testing Essentials

Ayman M. Kamal

Course Outline
Introduction

Software Development Phases

Software Testing Process

Bug Life Cycle

Testing Levels

Testing Types

Requirements Testing

Writing Test Cases

Bug Reporting
Introduction
What is Testing?

Why Testing?

What is Bug?

When Do We Start Testing?

When Do We Finish Testing?

Different Roles of Tester

How to be a Good Tester?

Software Testing Certificates

 What is Testing?

Comparing the actual behavior of the software against the

intended behavior

The process of executing a program or application with the intent

of finding errors

The process of verifying that a software meets the business and

technical requirements

The process of validating that a software works as expected

Software testing, depending on the testing method, can be

implemented at any time in the development phases

What is Bug?

An error in a computer program that produces an

incorrect or unexpected result

A mistake in a system that causes it to behave in

unintended way

Types of bugs:
Arithmetic bugs
Logical bugs
Syntax bugs
 Why Testing?

A primary purpose for testing is to detect software failures

Testing can measure the software quality

Testing can reduce the cost of errors

A study conducted by NIST (National Institute of Standards and

Technology) in 2002 reports that software bugs cost the U.S. economy
$59.5 billion annually. More than a third of this cost could be avoided if
better software testing was performed

Why Testing?

In May 1992, PEPSI told customers in Philippines they could win

$40,000 if they bought a bottle and found number 349 underside of the
bottle cap. Due to a software error, 800,000 bottle caps were produced
with number 349 instead of one, which was an equivalent of $42 billion
in prize money

Another story was printed in the New York Times in February 1994.
Chemical Bank managed to allow $15 million to be withdrawn
incorrectly from 100,000 accounts - a single line error in the program
caused every ATM on their network to process the transaction twice
 Why Testing?

When We Start Testing?

Once we are getting requirements baseline

Once we are getting requirements from client

Once the requirements are frozen

Conventional testing starts after the coding part in the

software development life cycle

Unconventional testing starts right from the first phase

of the software development life cycle

It’s highly recommended to start testing from the initial

phase of software development
 When We Finish Testing?

Test cases completed with certain percentage passed

When bug rate falls to certain level

Coverage of code, functionality, or requirements

reaches a specified point

Depending on testing budget and deadlines

You never finish your testing, may be you can decide

whether this can be launched or released

We can execute a certain subset of tests in a given

time, but testing will never be done. There will always
be bugs to be found

How to be a Good Tester?

Apart from the technical knowledge and testing

skills, you have to:

Ask about everything, Don’t assume

Explain your problem well to others

Analytical skills

Communication skills

Presentation skills
 How to be a Good Tester?

Apart from the technical knowledge and testing

skills, you have to:

Listen carefully

Don’t speak too fast

Speak clearly and to the point

Be honest and Responsive

Be open and available

Be prepared

Don’t take it personally

Different Roles of Tester

Beginner Level:
Test the requirements document
Create checklist
Create test cases
Execute test cases
Report bugs
Send status report

Intermediate Level:
Create test plan
Review test cases
Perform functional/non-functional testing
Supervise junior testers
Estimate testing effort
 Different Roles of Tester

Expert Level:
Review testing estimate
Allocate and utilize the testing team
Evaluate the testing team
Communicate with the project manager

Testing Certificates

International Software Testing

Qualifications Board (ISTQB)

Certified Software Tester

(CSTE)

Software Quality Engineer

Certification (CSQE)
Development Phases

Initiation Phase

Analysis Phase

Requirements Phase

Design Phase

Implementation Phase

Testing Phase

Installation Phase

Maintenance Phase
 Initiation Phase

High-level view of the intended project

Determine the goals of the project

Evaluate the project from different areas (i.e.

Financially and Technically)

Draft plan

Draft schedule

Analysis Phase

Talk to the customer

Understand customer need

Determine the problem

Requirements gathering

High level business requirements

 Requirements Phase

Communicate business requirements

to the technical team

Generate technical requirements

Requirements review

Some iterations between the

technical team and customer

Get customer signoff on the

requirements

Design Phase

Describe the system into details

Prepare business rules, diagrams and documentation

Prepare some layouts and prototypes

Describe the system as list of modules

Describe each module as

a list of features
 Implementation Phase

Start coding

Develop components

Integrate components

Deploy builds to be tested

Module testing is done in this

stage by the developers

Testing Phase

Start testing the application

The code is tested at various levels

Some testing iterations are done

Different types of testing are done

Most of the bugs are detected

 Installation Phase

The system deployed at the customer side

Some tests performed to make sure that the

system deployed successfully

Get user acceptance

The system is ready to use

Some documents are generated according to the

agreement (i.e. release notes and user guide)

Maintenance Phase

Corrective maintenance: Reactive modification

of a software product performed after delivery
to correct discovered problems

Adaptive maintenance: Modification of a

software product performed after delivery to
keep a software product usable in a changed or
changing environment
 Maintenance Phase

Perfective maintenance: Modification of a

software product after delivery to improve
performance or maintainability

Preventive maintenance: Modification of a

software product after delivery to detect and
correct latent faults in the software product
before they become effective faults
Course Outline
Introduction

Software Development Phases

Software Testing Phases

Bug Life Cycle

Testing Levels

Testing Types

Requirements Testing

Writing Test Cases

Bug Reporting
 Software Testing Process

Planning Phase

Designing Phase

Execution Phase

Completion Phase

Planning Phase

What things need to be tested

Which testing types we need

What type of documentation we need

Evaluate the budget

Calculate the risk

Create test plan

Create test schedule

It’s an ongoing process

 Designing Phase

Review all testing documents

Prepare test data if needed

Create test conditions

Create test cases

Create test scripts

Execution Phase

Execute test cases

Check the output

Report bugs

Update test cases if needed

 Completion Phase

Retest the resolved bugs

Complete testing iterations

Close all bugs

Complete documentation

Evaluate and document lessons

learnt
 Bug Life Cycle

Open

Assign

Test

Close/Reject

Open the Bug

Log new bug on tracking tool

Define all bug attributes

Verify the bug

Notify whoever concerned

Send the bug

 Assign the Bug

Is it real bug?

Assign to developer

Reproduce the bug

Fix/Reject the bug

Return the bug for testing

Test the Bug

Verify the changes made by the developer

Reproduce the bug

Test the bug again with the

same scenario

Update testing documents

if needed
 Close/Reject the Bug

Indicate the fix impact

Reject the bug if it still exists

Close the bug if it’s fixed

Notify whoever concerned

 Testing Levels

Unit Testing

Integration Testing

System Testing

Acceptance Testing

Unit Testing

Verify the functionality of a specific section of code

Usually made at the function/class level

Usually made by developers to ensure that the specific

function is working as expected

Unit testing alone cannot verify the functionality of a

piece of software

Assure that the software blocks

are working independently

Unit testing is also called

“Component Testing”
 Integration Testing

Verify the interfaces between components against the

design

Software components may be integrated in an iterative

way or all together

It uses to expose defects in the interfaces and

interaction between integrated components

System Testing

It tests a complete integrated system

It verifies that the system meets its requirements

It verifies that a system is integrated to any external

system

It verifies that a system is integrated to any third party

system
 Acceptance Testing

It’s used as a test prior to introducing a new build to the

main testing process

Acceptance testing may be performed by the customer

on their own environment

It called User Acceptance Testing (UAT) if it made by

the customer

It may be performed between

any two development phases
Course Outline
Introduction

Software Development Phases

Software Testing Phases

Bug Life Cycle

Testing Levels

Testing Types

Requirements Testing

Writing Test Cases

Bug Reporting
 Functional Testing

Black Box Testing

White Box Testing

Exploratory Testing

Boundary Testing

Regression Testing

Smoke Testing

Black Box Testing

Treats the software as a "black box" without any

knowledge of internal implementation

Aims to test the functionality of software according to

the applicable requirements

The tester inputs data and only sees if the output is

either as expected or not

The black box tester has no

relation with the code

The tester doesn't know how

the software was constructed
 Black Box Testing

Think about the windows calculator. If you type

“4” and press the “sqrt” button, you will get “2”.

With black box testing, it doesn't matter what the

software goes through to compute the square root

of “4”. It just does it.

White Box Testing

The tester has access to the internal data structures

The tester has access to the code

Creating tests to satisfy some criteria of code coverage

It includes syntax checking of the code

It includes review on the code to find errors

 White Box Testing

Let's say you have an entity that is stored across

multiple tables in the DB and the case is to delete

this entity. With white box testing, you need to

check if all the related rows are deleted from all

tables. If the deletion happens to delete only the

parent record and leave behind other rows, the case

is considered as failed.

Exploratory Testing

It’s a style of testing that gives the tester freedom to

continually optimize the quality of work

Seeks to find out how the software actually works, and

to ask questions about how it will handle difficult and
easy cases

Expectations are open. Results

may be expected and may be not

Less preparation is needed,

important bugs are found quickly

It’s suitable if requirements

incomplete or if there is lack of time
 Boundary Testing

It’s called boundary value analysis too

It focuses on corner cases

It tests correctness of application work when entering

the boundary values of input data

It’s performed using the extremes of the input domain,

for example:
Maximum Value
Minimum Value
Inside Range
Outside Range

Regression Testing

It seeks to detect errors by partially retesting a

modified program

The intent of regression testing is to make sure that no

additional errors introduced while retesting other
problems

It aims to make sure that no new bugs appeared during

the bug fixing phase

Its scope depends on the

scope of the modified or
added features
 Smoke Testing

It aims to detect failure which is severe enough to

reject the software release

A subset of all test cases that cover the main

functionality without going into details

Smoke testing is done by testers before accepting a

build for further testing

It aims to ensure that the basic

components of a software are
working fine

It’s considered as done successfully

if it does not catch fire

Non-Functional Testing

Performance Testing

Load Testing

Stress Testing

Usability Testing

Localization Testing
 Performance Testing

It’s used to verify performance behaviors for application

functions under the normal work conditions

It can demonstrate that the system meets performance

criteria

It’s performed using a test tool to simulate large

number of users

It measures concurrency, throughput

and response time

It evaluates the compliance of a

system with specified performance
requirements

Load Testing

It’s the process of putting heavy demand on a system

and measuring its response

It’s very important for client-server systems including

Web application and other business systems with
numerous concurrent users

It’s concerned with measuring the behavior of a

component or system with increasing the load

It’s testing the web site in order to

find out at what point it fails
 Stress Testing

It’s used to determine the stability of the system

It involves testing beyond normal operational capacity,

often to a breaking point, in order to observe the
results

Testing at conditions higher than what would be

encountered during normal operation

Stress Testing

The application is tested against heavy load such as

complex numerical values, large number of inputs and
large number of queries

It’s better to be prepared for extreme conditions rather

than letting the system/software/website crash, when
the limit of normal operation is crossed
 Usability Testing

It’s used to evaluate a product by testing it on users

level

It gives direct input on how real users use the system

It aims to determine the extent to which the software is

understood, easy to learn, easy to use and attractive to
the users

It aims to determine how the software is user friendly

Usability Testing

Goals of usability testing:

Performance (How much time? and How many steps?)
Accuracy (How many mistakes?)
Recall (How much do you remember after periods of non-
use?)
Emotional Response (How do you feel after completing the
task? Confident or Stressed?)
 Localization Testing

It’s focused on internationalization and localization

aspects of software

It’s the process of adapting a globalized application to a

particular culture

It’s making software specifically designed for a specific

locality

Localization Testing

It includes the translation of the application UI, content

and adapting graphics for a specific culture/locale

The following needs to be considered:

Language
Spelling Rules
Sorting Rules
Date Formats
Course Outline
Introduction

Software Development Phases

Software Testing Phases

Bug Life Cycle

Testing Levels

Testing Types

Requirements Testing

Writing Test Cases

Bug Reporting
Requirements Testing

Requirements Gathering

Business Requirements Definition

Types of Requirements

Why Requirements Testing?

Inputs and Outputs

Requirements Testing Criteria

 Requirements Gathering

Clarify what the project does and does not cover

State the objective of the project and define its scope

You can use written document, screen diagrams,

prototyping or use cases

The author should understand the client's business and the process
of software development

Reviews are needed as it costs

many times more to fix things
in testing than in the
requirements phase

Business Requirements

Identify the needs of stakeholder

List the business goals

Define the high-level needs of the project

Define the features of the project

Focus on the capabilities needed by the target users

 Types of Requirements

Functional requirements:
Define the functions of the software and its components
A function is described as a set of inputs, outputs and
Behaviors
Define what a system is supposed to accomplish
Specify particular results of the system

Types of requirements

Non-Functional requirements:
Specifies criteria that can be used to judge the operation of a
system
Specifies measurements that the software must meet
Defines how a system is supposed to be
It’s not concerned with specific behavior
 Why Requirements Test?

Most of the bugs in software are due to incomplete or inaccurate

requirements

The software code can’t do anything if there are ambiguities in

requirements

It’s better to catch the requirement issues and fix them in early
development life cycle

Cost of fixing the bug after completion of development or product

release is too high

Why Requirements Test?

 Inputs and Outputs

Requirement Test Criteria

Technical requirements should be:

Clear

Specific

Consistent

Measurable

Testable

Complete
Course Outline
Introduction

Software Development Phases

Software Testing Phases

Bug Life Cycle

Testing Levels

Testing Types

Requirements Testing

Writing Test Cases

Bug Reporting
Writing Test Cases

Validation Test Cases

Test Cases Criteria

Test Cases Attributes

Samples of Test Cases

 Validation Test Cases

Data type:
Characters
Integers
Decimals
Alphanumeric

Maximum length

Required/Optional

Special characters

Test Cases Criteria

Test cases should describe how your system works

Test cases should describe what kind of errors can you expect

Test cases should validate the testing coverage of the application

Test cases should be fast to execute

Test cases should be simple and easy to understand

 Test Cases Criteria

The purpose of each test case should be clear

You should be able to run test cases in any order

Write test case for every condition

Write test case to test only one thing

Don’t write explanations like essays

Be to the point

Test Cases Attributes

ID

Title

Description

Expected Results
 Samples of Test Cases

Open File Test Case:

Steps to reproduce:

1. Launch Application
2. Select “File” menu
File menu pulls down
3. Choose “Open”
"Open" dialog box appears
4. Select a file to open
5. Click OK

Result: File should open

Samples of Test Cases

Verify B - bold formatting to the text:

Test Case ID: B001

Test Execution:
1. Open program
2. Open new document
3. Type any text
4. Select the text to make bold.
5. Click B

Expected Result: Applies bold formatting to the text

Course Outline
Introduction

Software Development Phases

Software Testing Phases

Bug Life Cycle

Testing Levels

Testing Types

Requirements Testing

Writing Test Cases

Bug Reporting
Bug Reporting

Bug Criteria

Bug Attributes

Types of Bug

Types of Bug Severity

Example
 Bug Criteria

Each bug should have unique number

The bug should be reproducible

The bug should be described step by step

The bug steps should be mentioned clearly

Do not assume or skip any reproducing step

Do not write essay about the problem, be specific

Bug Criteria

Do not combine multiple problems even if they seem to be similar

Write different bugs for each problem

Report the problem immediately

Read the bug before hitting submit button

Do not use abusive language

 Bug Attributes

ID Title

Description Actual Results

Expected Results Type

Component Build Number

Priority Severity

Status Assign To

Types of Bug

Coding error

Design error

New suggestion

Documentation issue

Hardware problem
 Types of Bug Severity

Blocker: No further testing work can be done

Critical: Application crash, Loss of data

Major: Major loss of function

Minor: minor loss of function

Trivial: Some UI enhancements

Enhancement: Request for new feature or some

enhancement in existing one

Example

Lets assume in your application under test you want to create a new

user with user information, for that you need to logon into the

application and navigate to USERS menu > New User, then enter all

the details in the ‘User form’ like, First Name, Last Name, Age, Address

and Phone. Once you enter all these information, you need to click on

‘SAVE’ button in order to save the user. Now you can see a success

message saying, “New User has been created successfully”

 Example

But when you entered into your application by logging in and

navigated to USERS menu > New user, entered all the required

information to create new user and clicked on SAVE button. BANG!

The application crashed and you got an error page on screen.

(Capture this error message window and save as a Microsoft paint file)

Example

Bug Title: Application crash on clicking the SAVE button while

creating a new user

Build Number: Version Number 5.0

Severity: HIGH (High/Medium/Low)

Priority: HIGH (High/Medium/Low)

Assigned To: Developer-X

Status: New/Open/Active (Depends on the Tool you are using)

 Example

Description:
1) Logon into the application

2) Navigate to the Users Menu > New User

3) Fill all the user information fields

4) Click on ‘Save’ button

Actual Result:
The application crashed and you got an error page
on the screen

Example

Notes:
1) See the attached logs for more information (Attach more logs
related to the bug..IF any)

2) And also see the attached screenshot of the error page

Expected result: On clicking SAVE button, you

should be prompted to a success message “New User
has been created successfully”. (Attach ‘application
crash’ screen shot.. IF any)
 Contact Us

 +2 0222756841
 +2 01021902447

3 Ahmed El-Sayfy St., from Mostafa El-Nahhas,

Nasr City, Cairo
Egypt

E-Mail: info@testproeg.com

www.testproeg.com

TestPRO | Your Quality Partner