FUNDAMENTALS OF TESTING

CHAPTER 1
BEFORE WE START…..

 https://universititenaga.padlet.org/Ramona/oua6time638r3n1p
 Introduce yourself :
 Name
 Age
 Programme
 Location
 Life Quote
 Latest picture of yourself
TOPICS

Why is Seven The

What is Test
Testing Testing Psychology
Testing? Process
Necessary Principles of Testing
Subtopics Learning Objectives

Typical Objectives of Identify typical objectives of

Testing testing
WHAT IS
TESTING?
Testing and Debugging Differentiate testing from
debugging
WHAT IS TESTING?

The process consisting of all lifecycle activities, both static and dynamic,
concerned with planning, preparation and evaluation of a component or system
and related work products to determine that they satisfy specified requirements,
to demonstrate that they are fit for purpose and to detect defects.

ISQTB Foundation V3.1 -2018

OBJECTIVES OF TESTING
To prevent defects by evaluate
work products such as
requirements, user stories, design,
and code
To build confidence in the level of
quality of the test object
To verify whether all specified
requirements have been fulfilled
To find defects and failures thus
reduce the level of risk of
inadequate software quality
To comply with contractual, legal, or
regulatory requirements or
standards, and/or to verify the test
object’s compliance with such
requirements or standards
To check whether the test object
is complete and validate if it works
as the users and other
stakeholders expect
To provide sufficient information to
stakeholders to allow them to make
informed decisions, especially
regarding the level of quality of the
test object
Depends upon the context of the
component or system being tested,
the test level, and the software
development lifecycle model
 LET’S THINK!

Provide examples for the objectives of testing in different phases of the software life cycle

Plan Analysis

Implementation Design

7
TESTING AND DEBUGGING

Testing Debugging

Show failures that are caused by defects in the Part of development activity that finds, analyzes, and fixes
software defects
Done by tester Done by developer
TOPICS

Why is Seven The

What is Test
Testing Testing Psychology
Testing? Process
Necessary Principles of Testing
Subtopics Learning Objectives

Testing’s Contributions to Give examples of why testing is

Success necessary

Quality Assurance and Testing Describe the relationship between

testing and quality assurance and give
examples of how testing contributes to
WHY IS TESTING
higher quality NECESSARY
Errors, Defects, and Failures Distinguish between error, defect, and
failure

Defects, Root Causes and Distinguish between the root cause of a

Effects defect and its effects
EXAMPLES OF
SOFTWARE
FAILURES
TESTING’S CONTRIBUTIONS TO SUCCESS
testers involved in testers work closely with
requirements reviews or user system designers to increase
story refinement to identify understanding of the design and
requirements defects how to test it

reduces the risk of reduce the risk of

fundamental design defects
incorrect or untestable enable tests to be identified
features being developed at an early stage.

testers work closely with testers verify and validate

developers to understanding the software prior to
of the code and how to test release
it.

increases the likelihood

reduce the risk of defects that the software meets
within the code and the tests stakeholder needs and
satisfies requirements.
QUALITY ASSURANCE AND TESTING

Quality Assurance ≠ Testing

Proper execution of the
entire process.
Quality Assurance Ensure that appropriate
levels of quality will be
achieved.
Quality Management
Proper testing.
Quality Control Ensure the achievement of
appropriate levels of
quality.
ERRORS, DEFECTS, AND FAILURES

Error/ Mistake Fault / Defect/ Bugs Failure

Human action that produces an A flaw in a component or system Deviation of the component or
incorrect result that can cause the component or system from its expected delivery,
system to fail to perform its required service or result
function
COMMON EXAMPLE

Daily Life example

 Failures: Student get grade D for the subject
 Defects: Assignment mark = 0
 Errors: Student did not submit assignment file because he forgot about it

System Example
 Failures : Student unable to access the padlet
 Defects : Padlet setting = private
 Errors : lecturer did not change the setting to shared
WHY ERRORS OCCURS?

Miscommunication
Inexperienced or between project
Time pressure Human fallibility insufficiently skilled project participants, including
participants miscommunication about
requirements and design

Misunderstandings about
Complexity of the code,
intra-system and inter-
design, architecture, the
system interfaces, especially New, unfamiliar
underlying problem to be
when such intra system and technologies
solved, and/or the
inter-system interactions
technologies used
are large in number
LET’S THINK! PART 1
Identify error, defect, and failure

Disney’s Lion King, 1994 –1995

In the fall of 1994, the Disney company released its first multimedia CD-ROM game for children, The Lion
King Animated Storybook. Although many other companies had been marketing children’s programs for
years, this was Disney’s first venture into the market, and it was highly promoted and advertised. Sales
were huge. It was “the game to buy” for children that holiday season. What happened, however, was a
huge debacle. On December 26, the day after Christmas, Disney’s customer support phones began to ring,
and ring, and ring. Soon the phone support technicians were swamped with calls from angry parents with
crying children who couldn’t get the software to work. Numerous stories appeared in newspapers and on
TV news. It turns out that Disney failed to properly test the software on the many different PC models
available on the market. The software worked on a few systems—likely the ones that the Disney
programmers used to create the game—but not on the most common systems that the general public had.
 LET’S THINK! PART 1I
Identify error, defect, and failure
NASA Mars Polar Lander, 1999
On December 3, 1999, NASA’s Mars Polar Lander disappeared during its landing attempt on the Martian surface. A Failure Review Board
investigated the failure and determined that the most likely reason for the malfunction was the unexpected setting of a single data bit. Most
alarming was why the problem wasn’t caught by internal tests.
In theory, the plan for landing was this: As the lander fell to the surface, it was to deploy a parachute to slow its descent. A few seconds after the
chute deployed, the probe’s three legs were to snap open and latch into position for landing. When the probe was about 1,800 meters from the
surface, it was to release the parachute and ignite its landing thrusters to gently lower it the remaining distance to the ground.
To save money, NASA simplified the mechanism for determining when to shut off the thrusters. In lieu of costly radar used on other spacecraft,
they put an inexpensive contact switch on the leg’s foot that set a bit in the computer commanding it to shut off the fuel. Simply, the engines
would burn until the legs “touched down.”
Unfortunately, the Failure Review Board discovered in their tests that in most cases when the legs snapped open for landing, a mechanical
vibration also tripped the touch-down switch, setting the fatal bit. It’s very probable that, thinking it had landed, the computer turned off the
thrusters and the Mars Polar Lander smashed to pieces after falling 1,800 meters to the surface.
The result was catastrophic, but the reason behind it was simple. The lander was tested by multiple teams. One team tested the leg fold-down
procedure and another the landing process from that point on. The first team never looked to see if the touch-down bit was set—it wasn’t their
area; the second team always reset the computer, clearing the bit, before it started its testing. Both pieces worked perfectly individually, but not
when put together.
 DEFECTS, ROOT CAUSES AND EFFECTS

 Root cause analysis  Analyzing of Defects

 Actions or conditions that  to identify their root
contributes to the causes
creation of defects.
 lead to process  to reduce the occurrence
improvements that of similar defects in the
prevent a significant future.
number of future defects
from being introduced.
 LET’S THINK!
Identify the root cause of the problem and the improvements need to be taken in future.
Disney’s Lion King, 1994 –1995
In the fall of 1994, the Disney company released its first multimedia CD-ROM game for children, The Lion King
Animated Storybook. Although many other companies had been marketing children’s programs for years, this
was Disney’s first venture into the market, and it was highly promoted and advertised. Sales were huge. It was
“the game to buy” for children that holiday season. What happened, however, was a huge debacle. On
December 26, the day after Christmas, Disney’s customer support phones began to ring, and ring, and ring.
Soon the phone support technicians were swamped with calls from angry parents with crying children who
couldn’t get the software to work. Numerous stories appeared in newspapers and on TV news. It turns out that
Disney failed to properly test the software on the many different PC models available on the market. The
software worked on a few systems—likely the ones that the Disney programmers used to create the game—but
not on the most common systems that the general public had.
However, nobody reported the defect even after the creation & selling of thousands of CD-ROMs. Disney ended up paying by
product returns and replacement CD-ROM. They also incurred other costs associated with debugging, repair, and test cycle.
This defect affected the company’s reputation and financial status.
TOPICS

Why is Seven The

What is Test
Testing Testing Psychology
Testing? Process
Necessary Principles of Testing
Subtopics Learning Objectives
SEVEN TESTING
Seven Testing Explain the seven testing PRINCIPLES
Principles principles
SEVEN TESTING PRINCIPLES
TOPICS

Why is Seven The

What is Test
Testing Testing Psychology
Testing? Process
Necessary Principles of Testing
Subtopics Learning Objectives

Test Process in Context Explain the impact of context on the

test process

Test Activities and Tasks Describe the test activities and

respective tasks within the test process
TEST PROCESS
Test Work Products Differentiate the work products that
support the test process

Traceability between the Test Explain the value of maintaining

Basis and Test Work Products traceability between the test basis and
test work products
 Factors that influence the test process for an organization
 Software development lifecycle model and project methodologies
being used
 test levels and test types being considered
 Product and project risks
 Business domain
TEST PROCESS  Operational constraints, including but not limited to:
IN CONTEXT  Budgets and resources
 Timescales
 Complexity
 Contractual and regulatory requirements
 Organizational policies and practices
 Required internal and external standards
TEST ACTIVITIES AND TASKS

Test
Test Test Test Test
monitoring Test analysis Test design
planning implementation execution completion
and control
 Define
 the objectives of testing
TEST  the approach for meeting test
objectives within constraints imposed
PLANNING by the context
 may be revisited based on feedback
from monitoring and control activities.
TEST ACTIVITIES AND TASKS

Test
Test Test Test Test
monitoring Test analysis Test design
planning implementation execution completion
and control
TEST MONITORING AND CONTROL

Test Monitoring
Test control
The on-going comparison of
actual progress against Take actions necessary to
planned progress using any meet the objectives of the
test monitoring metrics test plan Test progress
defined in the test plan. supports • communicated to
stakeholders
• Includes
• deviations from the plan
• information to support
any decision to stop
Evaluation of exit criteria testing
TEST ACTIVITIES AND TASKS

Test
Test Test Test Test
monitoring Test analysis Test design
planning implementation execution completion
and control
TEST ANALYSIS

Analyzing the test basis
appropriate to the test level being
considered
Evaluating the test basis and test
Identifying features and sets of
items to identify defects of various
features to be tested
types

Defining and prioritizing test

conditions for each feature based
Capturing bi-directional
on analysis of the test basis, and
traceability between each element
considering functional, non-
of the test basis and the associated
functional, and structural
test conditions
characteristics, other business and
Test basis
technical factors, and levels of risks The body of knowledge used as the basis for test
analysis and design.
Test item
A part of a test object used in the test process
Test condition
Test Objectives A testable aspect of a component or system
identified as a basis for testing.
Requirement specifications - business requirements, functional requirements,
system requirements, user stories, epics, use cases, or similar work products that
specify desired functional and non-functional component or system behavior

Design and implementation information, such as system or software architecture

diagrams or documents, design specifications, call flow graphs, modelling diagrams
(e.g., UML or entity-relationship diagrams), interface specifications, or similar
work products that specify component or system structure
TEST BASIS

The implementation of the component or system itself, including code, database

metadata and queries, and interfaces

Risk analysis reports, which may consider functional, non-functional, and

structural aspects of the component or system
TYPES OF DEFECTS IN TEST BASIS

 Ambiguities
 Omissions
 Inconsistencies
 Inaccuracies
 Contradictions
 Superfluous statements
THE OVERVIEW

Test basis
SRS for Food
Ordering System
Test Item Test Condition
New Customer To check that user should be able to
Registration submit the new customer
registration form

Test object
Food Ordering
System
TEST ACTIVITIES AND TASKS

Test
Test Test Test Test
monitoring Test analysis Test design
planning implementation execution completion
and control
 Designing and Identifying Designing the Capturing bi-
prioritizing test necessary test test environment directional
cases and sets of data to support and identifying traceability
test cases test conditions any required between the test
and test cases infrastructure basis, test
and tools conditions, and
test cases

Test case: A set of preconditions, inputs, actions (where applicable), expected results and postconditions, developed based on test conditions.
Test data : Data needed for test execution.
Test environment: An environment containing hardware, instrumentation, simulators, software tools, and other support elements needed to conduct a test.

TEST DESIGN
 THE OVERVIEW….
High level Test Case Test data
TC1 : User submit form with valid data Alice
Alice@gmail.com
Alice123
TC2: User submit form with invalid data Alice123
Test basis Alice@gmail
SRS for Food Alice123
Ordering System
Test Item Test Case ID TC1
New Customer Test Condition
To check that user Test Case Description User submit form with valid data
Registration Form
-Name should be able to submit Test Case Pre-Condition User must not register before
-Email Address the new customer
-Password registration form Test Data Alice
Alice@gmail.com
Alice123
Test Steps 1. Use clicks on new customer link
Test object 2. System displays the new customer
Food Ordering registration form
3. User fills up the form
System
4. User submit the submit button
5. System performs validation data and save in
the database
6. System displays the successful message
Expected result Successful message is display
TEST ACTIVITIES AND TASKS

Test
Test Test Test Test
monitoring Test analysis Test design
planning implementation execution completion
and control
  Developing and prioritizing test procedures, and,
potentially, creating automated test scripts
 Creating test suites from the test procedures and (if
any) automated test scripts
 Arranging the test suites within a test execution
schedule in a way that results in efficient test execution
TEST  Building the test environment (including, potentially,
test harnesses, service virtualization, simulators, and
IMPLEMENTATION other infrastructure items) and verifying that everything
needed has been set up correctly
 Preparing test data and ensuring it is properly loaded in
test procedure
A sequence of test cases in execution order, and any associated actions that
the test environment
may be required to set up the initial preconditions and any wrap up activities
post execution.  Verifying and updating bi-directional traceability
test suite
A set of test scripts or test procedures to be executed in a specific test run. between the test basis, test conditions, test cases, test
test execution
The activity that runs a test on a component or system producing actual
procedures, and test suites
results.
test harness
A test environment comprised of stubs and drivers needed to execute a test
suite.
THE OVERVIEW…. High level Test Case Test data
TC1 : User submit form Alice
with valid data Alice@gmail.com
Alice123

Test basis TC2: User submit form Alice123

SRS for with invalid data Alice@gmail
Food Alice123
Ordering
System Test Case ID TC1
Test Scenarios

Test Case User submit form with valid

Test Item Test Condition Description data Test TP1
New To check that Procedure
user should be Pre-Condition User must not register before
Customer ID
able to submit Test Case
Registration Test Data Alice
the new Test To check the functionality of
Form Alice@gmail.com
Procedure new customer registration form
-Name customer Alice123
Description
-Email registration
Test Test Steps 1. Use clicks on new
Address form Pre-Condition User must not register before
object customer link
-Password 2. System displays the new Test 1. User submit form with invalid
Food
Ordering customer registration Procedure data (TC2)
form Steps 2. User submit form with valid
System 3. User fills up the form data (TC1)
4. User submit the submit
button
5. System performs
validation data and save in
the database
6. System displays the
successful message
Expected Successful message is display
result
TEST ACTIVITIES AND TASKS

Test
Test Test Test Test
monitoring Test analysis Test design
planning implementation execution completion
and control
  Recording the IDs and versions of the test item(s) or test
object, test tool(s), and testware
 Executing tests either manually or by using test execution tools
 Comparing actual results with expected results
 Analyzing anomalies to establish their likely causes (e.g.,
failures may occur due to defects in the code, but false
TEST positives also may occur
 Reporting defects based on the failures
EXECUTION  Logging the outcome of test execution (e.g., pass, fail, blocked)
 Repeating test activities either as a result of action taken for an
anomaly, or as part of the planned testing (e.g., execution of a
corrected test, confirmation testing, and/or regression testing)
 Verifying and updating bi-directional traceability between the
test basis, test conditions, test cases, test procedures, and test
results.
TEST ACTIVITIES AND TASKS

Test
Test Test Test Test
monitoring Test analysis Test design
planning implementation execution completion
and control
  Checking whether all defect reports are closed,
entering change requests or product backlog
items for any defects that remain unresolved at
the end of test execution
 Creating a test summary report to be
communicated to stakeholders

TEST  Finalizing and archiving the test environment,

the test data, the test infrastructure, and other
testware for later reuse

COMPLETION  Handing over the testware to the maintenance

teams, other project teams, and/or other
stakeholders who could benefit from its use
 Analyzing lessons learned from the completed
test activities to determine changes needed for
future iterations, releases, and projects
 Using the information gathered to improve test
process maturity
 ISO/IEC/IE
TEST WORK PRODUCTS EE 29119

Related work Traceability

Test planning Test plan A Test plan B Test basis info Exit criteria
product info

Test monitoring & Resource allocation,

control
Test progress report Test summary reports Task completion
usage & effort

defined test prioritized test bidirectionally

Test analysis Test charter Test basis defects
conditions conditions traceability

Identification of
Sets of high-level Identification and Design of test
Test design Test cases infrastructure and
test cases design of test data environment
tools

Sequence of Creation and

Test Test Test execution Executable low-
those test Test suites verification of
implementation procedures schedule level test cases
procedures test data

Status of Test item(s) Test object(s)

Status of test Test tools Testware
Test execution individual test Defect reports
procedures Documentation Documentation Documentation documentation
cases

Test completion test summary reports

action items for change requests
product backlog
finalized testware
improvement items
TEST DOCUMENTATION ISO/IEC/IEEE 29119- 3
 Good traceability supports:
 Analyzing the impact of changes

TRACEABILITY  Making testing auditable

BETWEEN THE  Meeting IT governance criteria

 Improving the understandability of test progress
TEST BASIS reports and test summary reports to include the status
AND TEST of elements of the test basis
WORK  Relating the technical aspects of testing to
stakeholders in terms that they can understand
PRODUCTS
 Providing information to assess product quality,
process capability, and project progress against
business goals
TOPICS

Why is Seven The

What is Test
Testing Testing Psychology
Testing? Process
Necessary Principles of Testing
Subtopics Learning Objectives
Human Psychology and Identify the psychological factors
Testing that influence the success of
testing THE
Tester’s and Developer’s Explain the difference between
PSYCHOLOGY OF
Mindsets the mindset required for test TESTING
activities and the mindset
required for development
activities
TESTER’S AND
DEVELOPER’S
MINDSETS
TESTER’SAND DEVELOPER’S
MINDSETS

 not normally an easy one because:-

 testers point out problems with software
 developers like to think their software is perfect
 testers are perceived as delaying the project by
finding faults in the system
 when the development slips, testers normally have
to work long hours to test the product, which in
turn can cause resentment.
 Curiosity

professional pessimism

TESTER’S a critical eye

MINDSETS

attention to detail

a motivation for good and positive communications and

relationships.
DEVELOPER’S MINDSETS

designing and building solutions than in confirmation bias

contemplating what might be wrong with
those solutions.
THE PSYCHOLOGY OF TESTING

 Ways to improve communication between tester and developer:

 Start with collaboration rather than battles – remind everyone
of the common goal of better-quality systems
 Communicate findings on the product in a neutral, fact- focused
way without criticizing the person who created it, for example,
write objective and factual incident reports and review findings
 Try to understand how the other person feels and why they
react as they do
 Confirm that the other person has understood what have been
said and vice versa

55
END OF
CHAPTER1!