Overview

Of
Software Testing
By
Mina Desai, Joint Director
Shampa Dey, Project Engineer
Contents

• Software Testing
• Need for Software Testing
• Role of Software Tester
• Qualities required for a good tester
• Testing Team Members
• Software Testing Misconceptions
• General facts about Software Testing
• Advantages of Software Testing
• Limitations of Software Testing

06/15/2020 www.cdac.in 2
Software Testing
• Software testing is a process

 to evaluate the system or its components

 to test the functionality of software application
 to find whether the developed software meets the specified requirements or not
 to find out the defects of the product
 to identify any gaps, errors or missing requirements

• According to ANSI/IEEE 1059 standard, Testing can be defined as

 A process of analyzing a software item to detect the differences between existing and required
conditions (that is defects/errors/bugs) and to evaluate the features of the software item.

06/15/2020 www.cdac.in 3
Who can Test ?
• A company depending on its requirements can have a team with responsibilities to
evaluate the developed software in context of the given requirements.
• Moreover, developers also conduct testing.
• The following professionals are involved in testing a system within their respective
capacities −
1. Software Tester
2. Software Developer
3. Project Lead/Manager
4. End User

• Different companies have different designations for people who test the software on the
basis of their experience and knowledge such as Software Tester, Software Quality
Assurance Engineer, QA Analyst, etc.

06/15/2020 www.cdac.in 4
Need for Software Testing
Ariane 5 crash June 4, 1996
On June 4, 1996 Ariane 5 rocket launched by European Space Agency crashed about 40 seconds after takeoff
Loss was about half a billion dollars
Explosion was the result of a software error
Module was re-used without proper testing from Ariane 4
Error was not supposed to happen with Ariane 4

Calculate 4195835 – 4195835 / 3145727 * 3145727 in ‘94 Pentium

it doesn’t return 0, but 256
Five entries in the quotient digit lookup table were erroneous.
Design Fix: 5 missing transistors, fixed with change to single mask
Cost of bug if detected at earliest spot: <$100
Actual Cost: $450 million

• Software Testing is necessary because

 We all make mistakes. Some of those mistakes are unimportant, but some of them are expensive or
dangerous.
 We need to check everything and anything we produce because human beings are prone to errors and
things can always go wrong.

06/15/2020 www.cdac.in 5
Need for Software Testing

A software product should only be released after it has gone through

a proper process of development, testing and bug fixing.

 Testing looks at areas such as performance, stability and error handling by setting up
test scenarios under controlled conditions and assessing the results.
 Software is mainly tested to see that it meets the customers’ needs and that it
conforms to the standards.
 Software is considered of good quality if it meets the user requirements.

06/15/2020 www.cdac.in 6
Role of software tester
• Apart from exposing faults (“bugs”) in a software product confirming that the program meets the
program specification, as a test engineer you need to create test cases, procedures, scripts and
generate data.
• You execute test procedures and scripts, analyze standards and evaluate results of
system/integration/regression testing.

You also...
• Speed up development process by identifying bugs at an early stage (e.g. specifications stage)
• Reduce the organization's risk of legal liability
• Maximize the value of the software
• Assure successful launch of the product, save money, time and reputation of the company by
discovering bugs and design flaws at an early stage before failures occur in production, or in the field
• Promote continual improvement

06/15/2020 www.cdac.in 7
Qualities required for a good tester
Knowledge of technology
 Technology in which the application is developed (added advantage) to design better and powerful test cases.

Perfectionist and a realist

 Being a perfectionist will help testers spot the problem
 Being a realist helps know at the end of the day which problems are really important problems.
 You will know which ones require a fix and which ones don’t.

Tactful, diplomatic and persuasive

 Tactful and know how to break the news to the developers.
 Diplomatic while convincing the developers of the bugs and persuade them when necessary and have their bug(s) fixed.
 It is important to be critical of the issue and not let the person who developed the application be taken aback of the
findings.

06/15/2020 www.cdac.in 8
Qualities required for a good tester
An explorer
 Creativity
 Attitude to take risk helps the testers venture into unknown situations and find bugs that otherwise will be looked
over.

Troubleshoot
 Troubleshooting and figuring out why something doesn’t work
 Confident and clear in communicating the defects to the developers.

Posses people skills and tenacity

 Testers can face a lot of resistance from programmers.
 Socially smart and diplomatic but not indecisive.
 Socially adept and tenacious where it matters.

06/15/2020 www.cdac.in 9
Qualities required for a good tester
Organized
 Best testers realize that they too can make mistakes
 They are very well organized
 Using checklists, facts and figures to support their findings to use as an evidence
 Double check their findings.

Objective and accurate

 Very objective and know what they report
 Convey impartial and meaningful information that keeps away politics and emotions
 Reporting inaccurate information is losing a little credibility.
 Good testers make sure their findings are accurate and reproducible.

06/15/2020 www.cdac.in 10
Qualities required for a good tester
Defects are valuable
 Good testers learn and improve from them.
 A defect found early substantially costs less when compared to the one found at a later stage.
 Defects can cause serious problems if not managed properly.
 Learning from defects helps – prevention of future problems, track improvements, improve prediction and
estimation.

06/15/2020 www.cdac.in 11
 Steps to Software Testing
1.
A Web Application
Software is developed

2.
Quality of the product
is not known
3.
How will be the
performance when
deployed 4.
Testing Team is required
to verify the quality
5.
Test Manager is required
to lead the team

06/15/2020 www.cdac.in 12
Software Testing Team
IT PROJECT TEAM

PROJECT MANAGER

BUSINESS TECHNICAL
SYSTEM ANALYST DEVELOPERS TESTERS
ANALYST ARCHITECT

TESTING TEAM

06/15/2020 www.cdac.in 13
Test Manager

The role of test manager is optional.

The test manager is hired when there are many testing groups.

The number of testers and testing groups depends on the software

testing workload.

Test Manager plays a central role in the Team.

06/15/2020 www.cdac.in 14
Role of Test Manager
 prepares test strategy
 prepare the test budget
 define test levels and test cycles
 develop strategy for estimating test effort
 develop strategy for test documentation,
metrics and reporting
 guides and controls the testing teams
 co ordinates with the test leaders and project
manger
 gathers the updates on testing progress
 identify the risk and priorities for testing
 design the testing approach, procedures  and
techniques
 identify, evaluate and choose the testing tools
 set the strategy on the automation testing
 co ordinate with other teams for software
releases

06/15/2020 www.cdac.in 15
 Test Leader
Each test group is led by a test leader.

The test leader performs the roles of test manager in absence of test
manager.

06/15/2020 www.cdac.in 16
 Role of Test Leader
 prepare the test plan at each test level based on
test strategy
 define the objectives, test items, approaches, risk,
contingencies in testing process
 assign roles and provide schedule to testers
 identify the test specifications test activities for
testers
 gather metrics and track the testing progress
 define entry and exit criteria
 define test environment and monitor the set up
process
 design test documents templates
06/15/2020 www.cdac.in
 set up the configuration management
 lead and conduct the meetings
 review the test documents
 co ordinate in the testing group
 monitor the testing and co ordinate the software
release
 resolve the issues in the test groups
 guide the testers and prepare the test summary
report
 co ordinate for automation testing
17
Tester
The testers group consists of

 entry level testers

 senior testers
 performance testers
 automation testers
 testers performing other specific tests

06/15/2020 www.cdac.in 18
Role of Tester
 gather the test requirements
 review the project documents to understand
the requirements and identify the errors
 assist the test leader to prepare the test plan
 create the test documents like traceability
matrix, test data and test cases
 set up and verify the test environment
 test the software at different levels and record
the results
 identify, report and track the defects
06/15/2020 www.cdac.in
 co ordinate with developers to ensure the
defect fixing process
 automate the tests and provide test support
 develop the test specifications, conduct the
test and record the results for the different
items of the software
 participate in the review meetings
 assist the users in UAT (User Acceptance
Testing)
 provide the test updates to test leader
 provide the learning experience during the
project and document it
19
Software Testing Misconceptions

REALITY

MISCONCEPTION

Just as every field has its misconceptions, so does the field of Software Testing.

There are many misconceptions surrounding software testing.

06/15/2020 www.cdac.in 20
Software Testing Misconceptions
Misconception 1
 100% defect- free product.
Reality 1
 Uncover as many defects as possible while ensuring that the software meets the requirements.

Misconception 2
 Easy
Reality 2
 Can be difficult and challenging

06/15/2020 www.cdac.in 21
Software Testing Misconceptions
Misconception 3
 Anyone can test
Reality 3
 Testers require specific kinds of skills

Misconception 4
 Too Expensive
Reality 4
 Early testing saves both time and cost

06/15/2020 www.cdac.in 22
Software Testing Misconceptions
Misconception 5
 Time consuming
Reality 5
 Testing while developing is never time consuming. Diagnosing and fixing the errors may take some time.

Misconception 6
 Complete testing is possible
Reality 6
 The domain of possible inputs of a program is too large to be completely used in testing a system.
 There are both valid inputs and invalid inputs
 There may be different workflows

06/15/2020 www.cdac.in 23
Software Testing Misconceptions
Misconception 7
 Automated testing eliminates the need for manual testing.
Reality 7
 100% test automation cannot be achieved. Manual Testing, to some level, is always necessary.

Misconception 8
 When a defect slips, it is the fault of the Testers.
Reality 8
 Test strategy may result in bugs being missed by the testing team. It is the responsibility of all members,
including developers of a project.

06/15/2020 www.cdac.in 24
Software Testing Misconceptions
Misconception 9
 Software Testing does not offer opportunities for career growth.
Reality 9
 With increase in competing software and demanding users, the need for software testers to ensure high
quality software will continuously grow.

06/15/2020 www.cdac.in 25
General Facts about Software Testing
Testing can’t show that bugs don’t exist.
• You can perform your tests, find and report bugs, but at no point can you guarantee that there are no bugs.

It is impossible to test a program completely.

• A simple program will have a large number of inputs, outputs, workflows and the specification is subjective
to frequent changes.

You can’t guarantee quality.

• As a software tester, you cannot test everything and are not responsible for the quality of the product.
• A tester can fail if he is unable to report accurately an observed defect.
• We have little control over quality.

06/15/2020 www.cdac.in 26
General Facts about Software Testing
Target environment and intended end user.
• Testing the application in the environment user is expected to use
• Single user / multi user system

No application is 100% bug free.

• Recognize the priorities
• Leave some less critical problems unsolved or unidentified.

Be the customer.
• Use the system as a lay user.
• Get a person who has no idea of the application to use it for a while.

06/15/2020 www.cdac.in 27
 General Facts about Software Testing

Build your credibility.

• Credibility is like quality that includes reliability, knowledge, consistency, reputation, trust, attitude and attention
to detail.
• It is not instant but should be built over time and gives voice to the testers in the organization.

Not all bugs you find will be fixed.

• Which bugs will be fixed and which won’t is a risk-based decision.

06/15/2020 www.cdac.in 28
General Facts about Software Testing

Review competitive products.

• Gaining a good insight into various products of the same kind and getting to know their functionality and
general behavior will help you design different test cases and to understand the strengths and weaknesses of
your application. This will also enable you to add value and suggest new features and enhancements to your
product.

Follow standards and processes.

• As a tester, you need to follow the standards and guidelines set by the organization. These standards pertain
to reporting hierarchy, coding, documentation, testing, reporting bugs, using automated tools etc.

06/15/2020 www.cdac.in 29
Testing - Advantages

Managers see testing as

 cost and not a profit source.
 They reduce the quality of testing or do not implement it at all.
 Profitable software and not best software
But
 Testing phase of every project is vital.
 Software testing is a great tool for business optimization.

06/15/2020 www.cdac.in 30
Testing - Advantages
Software Quality
Product must be of high quality
For quality, customers will surely pay more money.
Selling high-quality products build a strong reputation and brand image -
things that are important in the long-term.

06/15/2020 www.cdac.in 31
Testing - Advantages
Customer Satisfaction
Happy customer
Client can ask for a refund if the product does not suit him.
If the product is not reliable, you need to invest more money in fixing or
replacing it
It pays off to produce a higher quality product from the start.

06/15/2020 www.cdac.in 32
Testing - Advantages
Profit
Testing phase is not a cost, but a part of profit.
A good product needs less promotion - users will recommend.
Recommendation is the best and most valuable commercial you can get.
Help in retaining old customers and gaining new ones.
Cut down existing expenses – software does not need constant fixing.
Compromising on quality ends up with having to spend more money.
Avoid unnecessary expenses through customer support.

06/15/2020 www.cdac.in 33
Testing - Advantages
User Experience
User experience is a significant factor
Software needs to be simple, understandable, and easy to use.
Only testers can assure that the software is designed in a way that is logical
and intuitive.
For great user experience, software needs to be free from bugs and errors - a
source of dissatisfaction for users.

06/15/2020 www.cdac.in 34
Testing - Advantages
Business Optimization
Biggest benefit - software testing leads to business optimization.
More satisfied clients
Customer retention
Fewer costs of fixing a product
Fewer costs of a customer service
Better quality
Reliable products
Improved reputation and brand image

06/15/2020 www.cdac.in 35
Testing - Limitations

 There is no risk at all

 But if the testing is done badly or in an improper way, it will affect the quality of product.
 However, there are some limitations in software testing

06/15/2020 www.cdac.in 36
Limitations of Software Testing
Exhaustive Testing is Impossible
We cannot test a program completely
You cannot test every path
You cannot test every valid input
You cannot test every invalid input

Incorrect System Requirements

We can only test against system requirements
We may not detect errors in the requirements.
Incomplete or ambiguous requirements may lead to inadequate or incorrect testing.

06/15/2020 www.cdac.in 37
Limitations of Software Testing
Money
Budget constraints require very careful planning of the testing effort.

Time
Traditional software testing may simply take too long.
You will run out of time before you run out of test cases

Inadequate Knowledge of Tester

Tester may not have adequate knowledge about the product.

06/15/2020 www.cdac.in 38
 Thank you

06/15/2020 www.cdac.in 39