First Vietnamese QA Firm

LQA – Toward the Perfection 1

 Test Processes

Presented by: Tham Nguyen

Hanoi, Feb 2017

LQA – Toward the Perfection 2

Agenda
 Duration: 2 Hours
 Purpose: Introduce fundamental test process and activities
 Audience: Testers
 Content:
 Part 1: Testing lifecycle

 Planning and Control

 Analysis and Design

 Implementation and Execution

 Evaluating exit criteria and reporting

 Test closure activities

 Part 2: Testing process (in V-model and Agile/Scrum)

LQA – Toward the Perfection 3
Fundamentals test process
• Divide the activities within the fundamental test process into the following
basic steps
 Planning and control
 Analysis and design
 Implementation and execution
 Evaluating exit criteria and reporting
 Test closure activities

LQA – Toward the Perfection 4

Testing process overview

P A D I
l n e m Execution C
a a s p l
n l i l o
n y g e s
i s n m u
n i e Evaluating exit criteria
r
g s n Reporting test result
e
t

Control

Project timeline

LQA – Toward the Perfection 5

Part 1: Test planning and control
During test planning:
– Understand Product requirement and Project Requirement of intended testing
system.
• Product requirement = requirement specification (functional requirement and non
functional requirement)
• Project requirement: process, timeline, test environment, resource
– Specify the scope, approach, resources, and schedule of intended testing
activities.

LQA – Toward the Perfection 6

Test Planning
• Inputs for Test planning
– Customer requirements
– Project plan

• Output for Test planning

– Test plan document

LQA – Toward the Perfection 7

Test Planning - Flowchart
Determine scope and risk

Determine the test

approach

Implement test policy,

strategy

Determine required
resource

Schedule and define tasks

Determine exit criteria

LQA – Toward the Perfection 8

Test Planning – Major Tasks
Test planning has some major tasks:
 Determine the scope and risks and identify the objectives of testing
 Determine the test approach (techniques, test items, coverage, identifying and interfacing
with the teams involved in testing)
 Implement the test policy and/or the test strategy
 Determine the required test resources (e.g. people, test environment, PCs)
 Schedule test analysis and design tasks, test implementation, execution and evaluation
 Determine the exit criteria

LQA – Toward the Perfection 9

Test Control - Flowchart
Measure and analyze the results of
reviews and testing

Monitor, document progress

Initiate corrective actions

Make decisions

LQA – Toward the Perfection 10

Test Control – Major Tasks
Test control is an ongoing activity.
 compare actual progress against the planned progress
 report to the project manager and customer on the current status of testing including any
changes or deviations from the plan.
Test control: has the following major tasks
 Measure and analyze the results of reviews and testing:
 Monitor and document progress, test coverage and exit criteria:
 Provide information on testing
 Initiate corrective actions
 Make decisions

LQA – Toward the Perfection 11

Terms/Glossary
• Test planning: The activity of establishing or updating a test plan
• Test control: A test management task that deals with developing and applying a set of
corrective actions to get a test project on track when monitoring shows a deviation
from what was planned
• Test plan: A document describing the scope, approach, resources, and schedule of
intended test activities

LQA – Toward the Perfection 12

Terms/Glossary
• Test Approach: the implementation of the test strategy for a specific project.
• Testware: includes test cases, test plans, test data, etc..
• Test strategy: A high-level description of the test types to be performed and the
testing for a program (one or more projects)
• Test policy: A high level document describing the principles, approach and major
objectives of the organization regarding testing

LQA – Toward the Perfection 13

Part 2: Test analysis and design
Inputs for Test design
– Detail design
– SRS, User case
– Test plan
Outputs for Test design
– Test design document

LQA – Toward the Perfection 14

Test analysis and design -Flowchart
Review the test basis

Identify test conditions

Design the tests

Evaluate testability

Design the test

environment

LQA – Toward the Perfection 15

Test analysis and design – major tasks
 Review the test basis (such as the product risk analysis, requirements,
architecture, design specifications, and interfaces), examining the specifications
for the software we are testing.
 Identify test conditions based on analysis of test items, their specifications, and
what we know about their behavior and structure
 Design the tests: define test case
 Evaluate testability of the requirements and system
 Design the test environment set-up and identify any required infrastructure and
tools

LQA – Toward the Perfection 16

Terms/Glossary

• Test basis: All documents from which the requirements of a component

or system can be inferred. The documentation on which test cases are
based
• Test condition: An item or event of a component or system that could be
verified by one or more test cases, e.g. a function, transaction, feature,
quality attribute, or structural element.

LQA – Toward the Perfection 17

Terms/Glossary
• Test case: A set of input values, execution preconditions, expected results, and
execution post conditions developed for a particular objective or test condition, such
as to exercise a particular program path or to verify compliance with a specific
requirement

LQA – Toward the Perfection 18

Part 3: Test Implementation
• Inputs for Test implementation
– SRS, User case
– Test Design document
• Outputs for Test implementation
– Test case
– Test script
– Test data

LQA – Toward the Perfection 19

Test Implementation- Flowchart

Develop and prioritize test

cases

Create test suites

Implement, verify the

environment

LQA – Toward the Perfection 20

Test Implementation- Major Tasks
• Test implementation
– Develop and prioritize our test cases
– Create test suites from the test cases for efficient test execution.
– Implement and verify the environment. We make sure the test environment has
been set up correctly, possibly even running specific tests
Test suite: a logical collection of test cases which naturally work together. Test
suites often share data and a common high-level set of objectives. We'll also set
up a test execution schedule

LQA – Toward the Perfection 21

Part 4: Test Execution
• Inputs for Test execution
– Software/Product
– Test case, Test script, Test data
• Outputs for Test execution
– Defect list
– Issue list
– Test log

LQA – Toward the Perfection 22

Test Execution- Flowchart

Execute the test suites

Log and record identities

Compare actual results

LQA – Toward the Perfection 23

Test Execution- Major Tasks
• Test Execution:
 Execute the test suites and individual test cases, following our test procedures.
 Log the outcome of test execution and record the identities and versions of the software
under test, test tools and testware
 Compare actual results (what happened when we ran the tests) with expected results
(what we anticipated would happen)
 Repeat test activities - need to re-execute tests that previously failed in order to confirm
a fix (confirmation testing or re-testing).
• Re-testing ( = Confirmation testing): Testing that runs test cases that failed the last time
they were run, in order to verify the success of corrective actions.

LQA – Toward the Perfection 24

Terms/Glossary
• Test script: Commonly used to refer to a test procedure specification, especially an
automated one.
• Test procedure: A document specifying a sequence of actions for the execution of a
test. Also known as test script or manual test script.

LQA – Toward the Perfection 25

Part 4: Evaluating exit criteria and reporting
• Inputs for Test Report
– Test Log
– Test Plan
• Outputs for Test Report
– Test Report

LQA – Toward the Perfection 26

Evaluating exit criteria and reporting – Flow chart

Check test logs

Assess more tests or exit

criteria

Write a test summary report

LQA – Toward the Perfection 27

Evaluating exit criteria and reporting
• Evaluating exit criteria is the activity where test execution is assessed against the
defined objectives.
• Evaluating exit criteria has the following major tasks:
– Check test logs against the exit criteria specified in test planning:
– Assess if more tests are needed or if the exit criteria specified should be changed:
– Write a test summary report for stakeholders:
• Test summary report : A document summarizing testing activities and results. It also
contains an evaluation of the corresponding test items against exit criteria.

LQA – Toward the Perfection 28

Test Summary Report (IEEE 829 )
The IEEE 829 template includes the following sections:
 Test summary report identifier
 Summary (e.g., what was tested, what the conclusions are, etc.)
 Variances (from plan, cases, procedures)
 Comprehensive assessment
 Summary of results (e.g., final metrics, counts)
 Evaluation (of each test item vis-à-vis pass/fail criteria)
 Summary of activities (resource use, efficiency, etc.)
 Approvals

LQA – Toward the Perfection 29

Terms/Glossary
• Exit criteria: The set of generic and specific conditions, agreed upon with the
stakeholders, for permitting a process to be officially completed. The purpose of exit
criteria is to prevent a task from being considered completed when there are still
outstanding parts of the task that have not been finished. Exit criteria are used to
report against and to plan when to stop testing.
• Test closure: During the test closure phase of a test process, data is collected from
completed activities to consolidate experience, testware, facts, and numbers. The test
closure phase consists of finalizing and archiving the testware and evaluating the test
process, including preparation of a test evaluation report.

LQA – Toward the Perfection 30

Part 5: Test Closure Activities
• Collect data from completed test activities including checking and filing
testware, and analyzing facts and numbers.
• Need to do when
 Software is delivered
 The project is cancelled
 A particular milestone is achieved
 A maintenance release or update is done.

LQA – Toward the Perfection 31

Test Closure Activities - Flowchart
Check planned deliverables

Finalize and archive testware

Handover testware

Evaluate and Analyze lessions

LQA – Toward the Perfection 32

Test closure activities - Major Tasks
Test closure activities include the following major tasks:
 Check which planned deliverables we actually delivered and ensure all reports
have been resolved through defect repair
 Finalize and archive testware such as scripts, the test environment, and any
other test infrastructure, for later reuse.
 Hand over testware to the maintenance organization who will support the
software and make any bug fixes or maintenance changes, for use in
confirmation testing and regression testing.
 Evaluate how the testing went and analyze lessons learned for future releases
and projects
LQA – Toward the Perfection 33
Testing process

Software development process with

V Model
• Defined by the late Paul Rook in
1980’s.
• To improve the efficiency and
effectiveness of software
development.
• Accepted in Europe and UK as an
alternative to Waterfall model.

LQA – Toward the Perfection 34

Step in V-Shaped model

Quality is guaranteed at each project stage.

LQA – Toward the Perfection 35
 Test level
 Unit Test: Ensures the component is working according to the detailed
design/build specifications of the module. Also known as component,
module, or program testing.
 Integration Testing: Testing of more than one (tested) unit together to
determine if they function correctly. It is done using the integration test
design prepared during the architecture design phase. Helps assembling
incrementally a whole system, ensuring the correct ‘flow’ of data from the
first through the final component. It’s done by Developers or collaboration
Tester
 System Testing: Testing the system as a whole - Black-box type testing that
is based on overall requirements specifications; covers all combined parts of
a system. Ensures that system meets all functional and business
requirements with both verifying and validating test. It’s done by
Independence Test Team
 Acceptance testing: To determine whether a system satisfies its acceptance
criteria and business requirements or not.
LQA – Toward the Perfection 36
Testing in Agile

LQA – Toward the Perfection 37

Testing in Agile

LQA – Toward the Perfection 38

Testing in Agile

LQA – Toward the Perfection 39

QnA/Discussion

LQA – Toward the Perfection 40

LQA – Toward the Perfection 41
Reference
• Foundations of Software Testing_ISTQB Certification, Chapter 1
• Foundation Syllabus_2011
• ISTQB Glossary of Testing Terms v3.01
– http://www.istqb.org/
– http://www.slideshare.net/kkakkonen/agile-testing-kari-kakkonen-16062014

LQA – Toward the Perfection 42

Contact Us!

Website: http://lqa.com.vn
Email: hello@lqa.com.vn
Phone: +84 90 414 0881
LQA – Toward the Perfection 43