The Future of Testing / QA

Richard Bornet
rbornet@estsys.com
416-895-7176
www.scenariotester.com
Can we predict the future?

• Most people say no.

• We can predict patterns?
• Future is not just passive, we can affect
the future!
History of World in One Slide
• You can predict the future. But you have to look
at the long term big picture.

• Stone Age → Agriculture Age → Industrial Age

→ Information Age

• The one common element across the ages is

CHANGE and change is speeding-up.
• The reason for change is innovation.
How Innovation Changes Work
• Home workers
– Manual
– Cottage Industry

• Central Workplace
– Manual Work
– Tools
– Assembly line for component assembly
– Robots / Automation

• Unit Cost Decreased.

• Fewer individuals involved in creating the product.
So Where Are We in QA
• Manual with tools
– Word
– Excel
– Defect-tracking systems
– Requirements managers
– Test automation
• Change in QA will follow traditional growth
patterns.
• Future Goal
– Much higher productivity and efficiency.
– Significantly improved ROI
Return on Investment
• ROI will be the central driving force behind
IT budgets in next decade.

• QA will have to show a +ve ROI.

“Business units will control virtually all

discretionary IT spending.”
Gartner
Return on Investment
“Software defects are costing the U.S. economy 60
billion dollars.” National Institute of Standards and Technology

“It is much more cost effective to catch a bug in

test than it is in production. About half the cost.”

“Quality must be raised without incurring

additional costs.”
Moving to an improved ROI

• Organizations are employing strategies to

increase ROI.
• If catching the defect in QA is more cost-
effective than finding it in production
then:
– Finding it before QA will be even more cost
effective
– Not creating it in the 1st place will be
preferred.
So how is QA going to change?

• We Predict
– More testing by End user 5%
– More testing by Developers 5%
– Significantly shorter development cycles 7%
– More knowledgeable QA Resources 7%
– Increased appreciation for QA 8%
– Improved Processes 14%
– More Test Automation 31%
Move to Packaged Software

• Less in-house development.

• Buy packages which will execute the functions
– SAP, BPCS , PeopleSoft.
• Fewer and larger vendors.
• Customize for specific business.
What this means for QA

• Vendors provides a stable application.

• Vendors thoroughly test application.
• QA will need to do less testing.
• Test
– A good regression
– Custom adaptations
– Concentrate on whether business needs are
being met.
Pre-built components

• Common model in hardware.

• Not just objects.
• Whole functional modules.
• Storage facility for On Demand videos
– Database may come for IBM
– Web Services through .Net
– Logging and monitoring system through
another company.
What this means for QA
• Vendors provides stable modules.
• Less testing.
• Test
– A good regression
– Code that is built in-house
– Concentrate on whether business needs are being
met.
• Don’t test the video management system, it
should work.
Outsourcing

• Bring in a third party to develop and

maintain your systems.
• Outsourcers bring:
– Stable code
– Libraries of re-usable code
– Applications tested in various companies
– Costs shared between clients
– Faster turnaround.
What this means for QA

• Outsourcer provides stable code.

• Outsourcers can perform the QA functions.
• If testing in house test less testing.
– Outsourcer responsible for providing solid
code
– Concentrate on whether business needs are
being met.
Offshore development

• Same as outsourcing except theoretically

cheaper.
• Again, you can demand stable and defect
free code.
• What this means for QA
– Less testing
Move testing to the developers

• Some companies have moved testing

responsibilities onto their developers
• QA departments shut down or downsized
Move testing to the developers

• Problems
– Programmers over worked.
– Programmers may not know the business.
– Programmers hate to test.
– Programmers have time pressures.
– Person who build should not have sole responsibility
for testing.
– Programmers may not know how to create and
execute thorough testing.
Move testing to the developers

• Advantages
– Moves testing earlier in development cycle.
– Cost effective to catch bugs earlier.
– Catch bugs at development stage even more
cost effective.
– Programmers like to use tools.
Move testing to the developers

• Extreme programming
– Programmers design and build in tests
• Large use of Unit Testing tools
– Programmers build their own tools
– Many tools are available often for free.
Test Automation

• Many organizations try to automate.

• It has huge potential.
• Many fail (78%).
• Almost never is it complete testing.
• To often it is used for Smoke Tests.
Traditional Development Cycle
Courtesy of Borland
New Model
Efficiency Improvements -
Define / Requirements Stage
• Huge cost savings by catching problems early
– $65 per defect at Requirements stage
– $1000 to $3000 at System test
– $10,000 or more in Production
• Test the requirements
– Proof read the requirements
• There are technologies available now that will:
– Give you 100 % test coverage
– Automatically generate test cases
– Put testers and developers on same page
• Better tools for management of requirements.
• In the future
– Process will be enhanced, simplified and automated more.
Efficiency Improvements -
Design Stage

• Designers and Developers will build

software mindful of how we are going to
test it.
– Build tests into the application.
– Build hooks for testing into application.
Efficiency Improvements

• Development Stage
– Unit test automation
– Automation
• Deployment Stage
– Automation
• Production
– Automation
TEST AUTOMATION

• Single biggest change in QA, will be the

amount and type of test automation

• This will include:

– New types of testing
– New tools
– New strategies
Test Automation – Current State

• Stage 1: Record and Playback

• Stage 2: Write Code
• Stage 3: Modularity
• Stage 4: Data Driven Testing
Test Automation – Who Does What

• Stage 5: Teach the testers how to program.

• Stage 6: Make the automation tools simpler
• Tree view diagrams.
• “WinRunner Lite”.
• Stage 7: Teach programmers the business.
• Stage 8 : Bring in Automation experts.
– Hire a programmer.
– Bring in an external consultants.
– Create a test automation team.
Test Automation – Current State

• None of these stages fit the way testers

work.
– Having to code or handing off your work and
responsibilities to a 3rd party is not acceptable.
• Almost never is it complete testing.
• No wonder many automation efforts fail.
(78%).
Stage 9: Empower the TESTERS

• Give testers an interface to enter test

cases / test scripts.
• Press a button to execute them.
• Needs to be simple and powerful.
• Create test scripts before the application is
even built.
Web Services

• Endorsed by Microsoft, Borland, Sun and IBM.

• It is the ability to provide hooks to an application
which can be utilized by external applications.
• CIO like it because they can keep their legacy
systems.
• Users like it because they can get new and
useful interfaces.
Web Services – Testing

• Multiple Inputs
– GUI application, Palm Pilot, your phone,
– your TV, Voice, special devices
• Testing will be more interesting as new
forms of accessibility are developed.
• Component model of software.
Web Services – Testing

• Web Services creates a stable

environment.
• Testing components.
• Testing is split between
– Testing the web service
– Testing the GUI / application
– Testing the business rules.
• New simple tools.
New Model
FUTURE: Building the interface
between the tests and the app.
• Currently you Point and Click
• Building the scripts automatically
– Screen scrapers
– Web services test tools will pick up the WSDL
file.
FUTURE: Building the interface
between the tests and the app.
• Design Stage
– Developers when they design application,
design hooks into the application to allow for
data to be passed to and from the app.
– Design to expose the application to an outside
test tool.
– Generally straightforward.
FUTURE: Building hooks to test
automation into the application
• Application arrives automatically
automated.
• Test tool automatically connects test cases
and app.
• Build extra scripts in VB and Java and
compile code.
• Same cost no matter how many testers.
• Cheaper /more control.
FUTURE: Building testing right into
the application.
• Hardware manufacturers have been doing
this for ages.
• In Production we call it monitoring.
• eBay.
Future of QA

• Test in Production
– Automatic monitors for functionality and
performance.
– Programs, which can execute tests both
automatically and on request.
• Same set-up can be used at Deployment
and Development stage
Future of QA
• Get requirements right by thoroughly testing them.
• Automating the writing of the test cases and test scripts.
• Automated environment for testing
– Automatic monitors for functionality and performance.
– Programs, which can execute tests both automatically and on
request.
• Deployment testing is automated.
• Production is automatically monitored and tested
• Automated system allows for Troubleshooting and
Diagnostics.
Future QA – Solves problems

• Increased ROI.
– Decreased costs.
– Complete test coverage.
– Faster turnaround.
• We need to reinvent ourselves or we will
become obsolete.
• Huge opportunities for QA.
New Model
Contact Information
Richard Bornet
416-895-7176
rbornet@estsys.com

Web site
www.scenariotester.com

Presentation
www.scenariotester.com and go to Articles