SOFTWARE QUALITY

ASSURANCE
Course Outline
Course Outline (Cont….)

Course Contents
Lecture 11 Cost of Quality
Lecture 12 Software Quality Metrics
Lecture 13 Introduction of Testing Tool (Guest Lecture)
Lecture 14 Course Recap
 LECTURE # 12

SOFTWARE TESTING
AUTOMATION
Today’s Plan
 Software Test Automation
 Objectives of Automated Testing
 Myths and Facts
 Pros and Cons of Automated Testing
 Examples of some testing tools
Definition of Automation

“Automation is the use of tools and strategies

that reduce human involvement or
interaction in unskilled, repetitive or
redundant tasks”
Introduction
 Automation of software testing is becoming more popular
these days both for client server and web applications.
 Software applications are becoming complex day by day,
new requirements are changing constantly and the time for
software testing is getting shorter.
 This situation calls a need to automate
the software testing process to
address the cost and time factors.
What is Software Test
Automation?
 Test automation is the use of software to control the
execution of test, the comparison of actual outcomes to
predicted outcomes, the setting up of test preconditions,
and other test control and test reporting functions.
 Commonly, test automation involves automating a
manual process already in place that uses a formalized
testing process.
Objectives of Test Automation
 To free engineers from tedious and redundant manual
testing operations.
 To speed up a software testing process, and to reduce
software testing cost and time during a software life
cycle.
 To increase the quality and effectiveness of a software
test process by achieving pre-defined adequate test
criteria in a limited schedule.
Key to success

“To reduce manual testing activities and

redundant test operations using a
systematic solution to achieve a better
testing coverage”
Test Automation and Testing
 Test Automation is not testing.
 Test Automation supports testing.
 Test Automation is a process to be managed.
 Industry surveys indicate that 75% of all functional
testing is still done manually.
Myths of Automated Testing
 Automated testing always results in improved software
quality.
 Every application development project or testing team
can use automated testing tools.
 Automated testing finds more bugs.
 It eliminates or reduce manual testers.
 Test Automation is Silver Bullet - If something can be
automated and you can automate - automate it.
Realities of Automated Testing
 Automated testing requires a higher initial investment but
can yield a higher ROI (Return On Investment).
 Skills and training are required to be successful with any
automated testing tool.
 Every company doing automated testing still does some
amount of manual testing.
 Improves testing productivity.
 Improves accuracy.
 Increases test coverage.
 Does what manual testing cannot do.
Simple [Black Box] Testing Model

System
Test Inputs Under Test Results
Test
Implications of the Simple Model
 We control the inputs
 We can verify results

But, we aren’t dealing with all the

factors
 Memory and data
 Program state
 System environment
Automated or Manual Testing?
Automated testing is a good idea when we have to do:
 Regression Testing: For re-testing preexisting application functions
that are being carried forward to new versions (usually the majority,
unless app is brand new)
 Smoke Testing: For getting a quick high-level assessment on the
quality of a build and making go / no-go decision on deeper testing.
 Data Driven Testing: For testing application functions where the
same functions needs to be validated with lots of different inputs &
large data sets (i.e. login, search)
 Load & Performance Testing: No feasible manual alternative exists.
Advantages of Automated Testing
Reliable Repeatable

Tests perform precisely the same You can test how the software reacts
operations each time they are run, under repeated execution of the
thereby eliminating human error. same operations.

Programmable Comprehensive

You can program sophisticated tests You can build a suite of tests that
that bring out hidden information covers every feature in your
from application.
the application.
Advantages of Automated Testing
Reusable Better Quality Software

You can reuse tests on different Because you can run more tests in
versions of an application, even if the less
users interface changes. time with fewer resources.

Fast Cost Reduction

Automated Tools run tests As the number of resources for

significantly faster than human users. regression test are reduced.
Disadvantages of Automated
Testing
Expertise:
Proficiency is required to write the automation test scripts.
Debugging:
Debugging the test script is major issue. If any error is present in
the test script, sometimes it may lead to deadly consequences.
Maintenance:
Test maintenance is costly in case of playback methods. Even
though a minor change occurs in the GUI, the test script has to be
rerecorded or replaced by a new test script.
Common types of automated testing
methods
 Monkey Testing
 Capture / playback
 Code-based (Unit) testing
Monkey testing
 Randomly selecting inputs from
a large range of values and
monitoring if exceptions are
thrown.
 For example, a monkey test can
enter random strings into text
boxes to ensure handling of all
possible user input.
 It applies not only for GUI or
web testing, but also for unit
testing.
Capture / playback
 It’s a set of software programs that capture user
inputs and stores it into a script for later replay.
+ Repeated testing can be performed quickly.
+ Does not require programming skills.
- When the GUI changes, input sequences previously
recorded may no longer be valid.
- Difficult to determine location of bugs.
Code-based (Unit) testing
 Individual units of source code are tested to
determine if they fit for use.
 Ideally, each test case is independent from the others:
+ Bugs can be found in early development stage
+ Easy to test boundary cases
- Not effective for the integrated system testing
Examples
 There is a wide variety of tools available in the market
for different testing types.
 Some of the tools are commercially available (requires a
license to use) while some of them are open source.
 Some tools provide the facility to test the client server
and web based applications.
 Some tools are specific to performance or load testing
only.
Examples

Tool Vendor
Quick Test Professional (QTP) HP formerly Mercury
Rational Functional Tester IBM formerly Rational
Visual Team Test Microsoft
SilkTest Borland formerly Segue
Selenium Open Source
Anteater Open Source
Choosing Automation Tools
 Ease of integration
 Compatibility
 Performance
 Types of tests
 Maintainability
 Affordability
You are awesome!

I have a surprise for you….