Test training

Version: 1.5
Author: Hoang A Na

Aug 2005

© 2005 FPT-SOFT 04e-BM/NS/HDCV/FSOFT

 Introduction

Estimated duration: 1-1.30 hour

Purpose: Practical Guide to Software System Testing
Targeted attendees: PLs, Testers, SQA

FPT-Soft Confidential TRAINING MATERIALS 2

 Objectives

After the course, students will clearly understand:

 Test Process
 Defect logging
 Software Testing Concepts

FPT-Soft Confidential TRAINING MATERIALS 3

 Agenda

A. Overview
B. Test Process
C. Defect Tracking
D. Test methods
E. Stages of Test
F. Types of Test
G. Testing Infrastructure
H. Test Measurements
I. Test Web application

FPT-Soft Confidential TRAINING MATERIALS 4

 A. Overview

• Because software is likely to have faults

• To learn about the reliability of the software
• To fill the time between delivery of the software and the release date
• To prove that the software has no faults
• Because testing is included in the project plan
• Because failures can be very expensive
• To avoid being sued by customers
• To stay in business

FPT-Soft Confidential TRAINING MATERIALS 5

 A. Overview

 Why testing?
• Because software is likely to have faults
• To learn about the reliability of the software
• To fill the time between delivery of the software
and the release date
• To prove that the software has no faults
• Because testing is included in the project plan
• Because failures can be very expensive
• To avoid being sued by customers
• To stay in business

FPT-Soft Confidential TRAINING MATERIALS 6

 QA - QC

 QA: Quality Assurance

 QC: Quality Control
 QA vs. QC
• The mission of QA is defect prevention
• The mission of QC is defect detection
• Testing is a specific method of QC
• PREVENTION IS CHEAPER THAN CURE
• Verification and Validation

FPT-Soft Confidential TRAINING MATERIALS 7

 Validation - Verification

 Validation
 Ensure that the system operates according to a plan by
executing system functions
 Did we build the system right?
 Validation: done by customer (including review and
acceptance test).
 Verification
 Methods to ensure that the system complies with an
organizational standard or process
 Did we build the right system?
 Verification: done by project team (Review & Test)
 So what do we need to do most when we test?
QC - Verification

FPT-Soft Confidential TRAINING MATERIALS 8

 Test conditions
 Testing involves monitoring and improving the process:
• making sure that standards and procedures are followed
• ensuring that problems are found and dealt with.
 Testing involves operation of a system/application under
controlled conditions and evaluating the results.
The controlled conditions = normal + abnormal
 Creatively Destroying the Product: attempt to make things go
wrong to determine if
• things happen when they shouldn’t
• or things don’t happen when they should
 Check the quality of the PRODUCT and not the quality of the
PROGRAMMER
 A Test Cycle is the period in which the Product is tested and
defects are verified. Before the release of a product, it will have
gone through numerous Test Cycles to confirm the Quality of the
product.

FPT-Soft Confidential TRAINING MATERIALS 9

 Skills

Attributes of an ideal tester:

 Observation skill (What you see is what you test)
 Communication skill
 Technical skill
 Self-confident
 Perseverance, patience
 Skepticism
 Self-motivating

FPT-Soft Confidential TRAINING MATERIALS 10

 Testing Principles
 All tests should be traceable to customer requirements
 Tests should be planned long before testing begins
 Testing should begin in the small and progress toward
testing in the large
 Exhaustive testing is not possible
 To be more effective, testing should be conducted by an
independent third party
 Understanding customer needs
• Identify business need
• Identify target customer
• Identify usage behavior and patterns
• Prioritize (impact - frequency)

FPT-Soft Confidential TRAINING MATERIALS 11

 Quality cost

Quality Cost includes:

 Staff training
 Review
 Test
 Correction
Error Removal Cost:
Requirement 1
Design 5
Code 10
System test 50
Post- Delivery 100

FPT-Soft Confidential TRAINING MATERIALS 12

 Source: IBM

FPT-Soft Confidential TRAINING MATERIALS 13

 B. Test Process

1. Test team
2. Work products of test
3. Test life cycle
4. Test tools

FPT-Soft Confidential TRAINING MATERIALS 14

 1. Test Team

Project Report direction

SQA
leader
Support direction

Infrastructure
Test leader
management

Tester Tester

FPT-Soft Confidential TRAINING MATERIALS 15

 2. Work Products of Test
 Input:
 Customer requirements and Acceptance criteria
 Software Requirement Specification (SRS)
 Design documents
 Programs (Modules)
 Output:
 Test documents: Test plan, Test cases, Test
script, Test data
 Defects in DMS
 Test results

FPT-Soft Confidential TRAINING MATERIALS 16

 What tester to do while review

• Study requirement (SRS/ Specs):

 Identify what to be tested. Is everything
clear and detailed? Is everything possible?
 Identify which requirements cannot be
tested
 Create Test plan
• Review/Study DDD:
 Verify the consistent with SRS
 Identify key functions
 Create Test cases & test data

FPT-Soft Confidential TRAINING MATERIALS 17

  Test cases:
 To cover the basic functionalities of the Product using
normal user scenarios
 Test Cases are modified during the process of the
Product Development as and when new features are
added or when the existing features are modified.
 Defect:
 Defects are scenarios that cause the Product to throw
errors or cause the Feature not to function as mentioned
in the Functional Spec
 Testers also need to keep track of the defect as to
confirm that they have been fixed and not faced there
after => Defect Verification

FPT-Soft Confidential TRAINING MATERIALS 18

 3. Testing Life Cycle

 The Traditional Lifecycle

 Requirements  Design  Code  Test
 Problem: testing is being left too late in the
process  high quality cost

 Parallel Testing Lifecycle

 Only test execution needs to wait until after coding.
 Test planning and test case design can occur in
parallel with development.
 The act of planning and designing the tests to be
applied would discover bugs

FPT-Soft Confidential TRAINING MATERIALS 19

 •Project plan •Detail design
•SRS •Software/ product
•Customer requirements
and Acceptance criteria •Test plan •Test case/ test script/ test data

Test planning Test preparation Test execution Data analysis

•Test plan •Test case •Defect list

•Test scrip •Test result
•Test data

FPT-Soft Confidential TRAINING MATERIALS 20

 Testing Life-cycle starts as soon as The whole process will be
START
the software life-cycle starts repeated for the next iteration
What: create a test plan to
identify and describe the Plan Test
testing that will be
What:
What: identify
implemented test
and
Implement model and
executed
(record, What: Evaluate test
its artifacts:
When:
generate test cases,
requirement
or program) test
analysis
the test Design Test executions
procedures
phase and test data
procedures that were defined When: Before completion of
When:
Who:
in Design design
Test phase
Leader
test (test scripts) system testing phase
Who: Implement Test
When: Coding phaseTesters
Test Leader & Who: Test Leader and
Who: Test Leader & Testers Testers

Integration Test
Evaluate Test
What: Execute test
programs/scripts that were System Test
generated in Implementation What: Generate final test
test. coverage summary
What: Execute all test When: Before delivering,
When: integration
programs/scripts testing
that were Sum up and report
phase report to SQA by the Final
generated in Implementation inspection
Who: Testers test.
test. Regression Who: Test Leader
When: system testing phase FINISH
Who: Testers

FPT-Soft Confidential TRAINING MATERIALS 21

 4. Test Tools

 Some tools in FPTSoft

 Test data management: MS Excel
 Scripting language: Rational Visual Test 6.0
Rational Test Studio (implementing)
 Defect management: DMS
 Graphic test : Virtual Ruler, Eye Dropper
 Functional Test : Rational Robot Test
 Performance test : Rational Load Test

 Templates for test documents

 Test Plan
 Test case
 Test Design

FPT-Soft Confidential TRAINING MATERIALS 22

 C. Defect Tracking

What: -To analyze the causes of the defect, find-out solution

What: To log defect into DMS
and assign What:
with clear
to person To correct
who What:
the defect
is responsible To
forretest
and perform
the function(s)
correcting it. unit containing the
information: project, module, stage,testingQCtoactivity,
make sure
defect
theand
defect
makehassure
been
that the defect has been
- To accept the defect
created date ...corrected completely of
severity, type, due date, tester,removed. if it’s acceptable (by concession
authority or customer).
Defect still exists in theWho:
Who: Tester product, change
Developer status
Who: to ERROR
Tester
Who: Project Leader
Defect’s status: ERROR Defect’s status: Defect’s
PENDING status: TESTED
Defect’s status: ASSIGNED / ACCEPTED

Step 1 Step 2 Step 3 Step 4

Logging defect Analyzing defect Correcting defect Retesting defect

The basic flow of the process of testing:

1. Development develops the Product and sends it to Tester
2. Tester tests the products and gathers the Defects.
3. The Defects found out is logged in DMS and assigned to the Developer.
4. The Defects are fixed by the developers and sent back for testing.
5. The above cycle is repeated from 2 to 4 until the defects have been
completely fixed

FPT-Soft Confidential TRAINING MATERIALS 23

  Project leader collects defect number in DMS
to Weekly Report to show-up the progress and
result of testing phase of the project.

Classify defects by:
 Defect types
 Severities
 Work products
 Quality control activities
 Defect origins

FPT-Soft Confidential TRAINING MATERIALS 24

 D. Test methods
 White Box Testing
 Is a test case design that uses the control structure of the
procedural design to derive test cases
• It guarantees that all independent paths within a module have
been exercised at least once
• Exercise all logical decisions on their true an false sides
• Execute all loops t their boundaries and within their operational
bounds
• Exercise internal data structures to assure their validity
 Black Box Testing
 Focuses on functional requirements of the software
 Derive sets of input conditions that will fully exercise all functional
requirements for a program
• Incorrect or mission functions
• Interface errors
• Errors in data structures or external db access
• Performance errors

FPT-Soft Confidential TRAINING MATERIALS 25

 • White box: Reveal problems with the internal structure of a program

• Black box: Assess how well a program meets its requirements

I/P O/P

FPT-Soft Confidential TRAINING MATERIALS 26

 E. Stages of Test – V Model

FPT-Soft Confidential TRAINING MATERIALS 27

 Project overview
Requirement
Design
•Functional
Specification •ADD

•SRS •DDD Code

•Code Review
•Unit Test

Functionality
System •Integration
•System Test test
Acceptance
•Performance Test
•Acceptance
•Stress Test
Test

FPT-Soft Confidential TRAINING MATERIALS 28

  2 categories:
 Document => review
 Application => code review, test

 Unit Test
 Integration Test
 System Test
 Acceptance Test

FPT-Soft Confidential TRAINING MATERIALS 29

 1. Unit Test

 Verifies that the component/module functions

properly
 Done in a controlled environment
 Normally white box oriented
 What to test:
• Data structure is examined
• Boundary conditions are tested to ensure that the
module operates properly for input and output data
• All independent paths are exercised
• Error handling paths are tested
• Use of stubs (modules to be called) and drivers
(modules that calls)

FPT-Soft Confidential TRAINING MATERIALS 30

 2. Integration Test

 Compare with ADD: Check that modules integrate with one

another correctly
 Problems may occur due to:
 Data lost across interface
 One module can have an adverse affect on another
 Sub functions may not produce the desired major functions
 Individual acceptable imprecision but unacceptable if
magnified
 Global data structures presents problems

 Integration test:
• Functionality
• Interface

FPT-Soft Confidential TRAINING MATERIALS 31

 3. System Test

 Compare with SRS: Check that the whole system works as

expected
 Sources of Software Faults
 not all faults are seen
 faults may occur in different phases
 programmers cannot be expected to spot design faults
 faults may be added when changes are made to correct other
faults
 maintenance faults
 documentation faults
 Process
 Performance Testing (Load test, Stress test, Volume test)
 Test Stability, Compatibility
 Installation Testing

FPT-Soft Confidential TRAINING MATERIALS 32

 4. Acceptance Test

 Results
 it is the customer’s way to verify that what was wanted is
what is built
 uncovers more than requirements discrepancies
 allows the customers to determine what they really want,
whether specified in the document or not.
 new problems may arise
 rapid prototyping
 changes may not only mean improper definition of
requirement, but also because customers may decide that
the problem is changed and a different solution is needed
 Goal:
No defect during acceptance test

FPT-Soft Confidential TRAINING MATERIALS 33

 F. Types of Testing

 Functional testing
 Interface testing
 Performance testing: Test performances requirement
specifications (response time)
 Data & Data integrity testing
 Security & Access control testing
 Load testing
 Stress testing
 Volume testing
 Regression testing
 Installation testing

FPT-Soft Confidential TRAINING MATERIALS 34

 G. Testing Infrastructure

1. Test environments
2. Test documentation
3. Configuration management and change control

FPT-Soft Confidential TRAINING MATERIALS 35

 1. Test Environment

 The test environment includes:

 the system or components under test
 test documentation, describing the tests to be
performed and recording results
 test cases,test data

 Test Environment Independence

It is very important to maintain independence between
the test environment and the production environment.

FPT-Soft Confidential TRAINING MATERIALS 36

 2. Test documents

Test plan:
Purpose: to organize testing activities
Defines:
 Scope of test: stages and types of test
 Risks may affect testing
 Requirements to test: what will be tested
 Testing strategy: how will be tested
 Test criteria (start, stop, suspension, completion)
 Tools used to test
 Resource and environment of test
 Deliverables of test

FPT-Soft Confidential TRAINING MATERIALS 37

 2. Test documents (cont.)

 Test case: build test cases from use cases

- Test case specifications:
 Basic flow
 Alternate flows
- Test procedures:
 Pre-condition for test this case
 test actions taken by the actor
 Test input: input data (valid and invalid data)
 Expected result
Input 1 => Expected Output 1
Input 2 => Expected Output 2
Input 3 => Expected Output 3
…

FPT-Soft Confidential TRAINING MATERIALS 38

 Test case:

• Normal cases
• Abnormal cases
• Boundary values

FPT-Soft Confidential TRAINING MATERIALS 39

 2. Test documents (cont.)

 Test data
 Test result: report test coverage
• Test case ID
• Output
• Result (pass, failed, …) by comparing with test
case expected output.
• Test Coverage/ Success Test rate
 Test status:
• Pass: test done successfully
• Failed: test done and result different from expected
• N/A: test cannot be done yet

FPT-Soft Confidential TRAINING MATERIALS 40

 3. CM and Change Control

 CM: tester knows where to get source codes for

test
 Change requirements:
 update TP: type of testing, time of test
 update TC: add or modify test cases
according to CRs
 track the change in Modification history
 re-test basing on updated TC
 regression test

FPT-Soft Confidential TRAINING MATERIALS 41

 H. Test Measurements

 Measurements:
Defects in DMS
 Defect rate
Weighted defects/ project size (in UCP)
 Test coverage: number of test cases executed
 Successful Test coverage: number of passed
TCs/ Total executed TCs
 Metrics to assess individual tester performance:
 Test effectiveness
Weighted defects/ Test effort
 Leakage
Weighted defects found after release/ project size

FPT-Soft Confidential TRAINING MATERIALS 42

 I. Test Web application

 What to test:
• All components of a Web application on both the client
and server side
• Validation or Functional Testing
• HTML Validation
• Link Testing
• Load and stress Testing
• Security Testing
• Regression Testing

FPT-Soft Confidential TRAINING MATERIALS 43

 I. Test Web application

 Problem:
• Impossible to test all possible dependencies and
everything that could go wrong with this site.
• Impossible to test varied target audience and platforms
that a web applications addresses
• Tools used for load test, volume test
 Solution:
• Determine where to focus testing efforts within budget
and schedule constraints
• Analyse risks and set priorities based on risk analysis

FPT-Soft Confidential TRAINING MATERIALS 44

 I. Test Web application

 Risk analysis:
• Which functionality is most critical to its purpose?
• Which function is most likely to be the most frequently
used or the sequence of steps to be most frequently
performed?
• What type of areas could possibly cause the most
complaints or bad publicity?
• What parts of the application pose the maximum security
risk?

FPT-Soft Confidential TRAINING MATERIALS 45

 Resources & references

Reference
 Practical Guide to Software System Testing

 Test Guideline

 Lists of common defects: Web, Java, C/C++.

VB, .Net
 TestTraining for beginer.doc

FPT-Soft Confidential TRAINING MATERIALS 46

Questions and Answers