FOR DUMMIES

Greg Phillips
Technical Sales Consultant

greg.phillips@qa-systems.com

Copyright 2012
Copyright 2012 –– QA
QA Systems
Systems Limited
Limited Test Test
Slide 0
Test
 Intelligent Testing For Dummies

Copyright 2013 – QA Systems Ltd Slide 1

 Why test?
Industry Regulation

Avionics: DO-178B/C
Nuclear: IEC 60880
Rail: EN 50128
Copyright 2013 – QA Systems Ltd Slide 2
 Why test?
Commercial risk
• Corporate and brand reputation
• Litigation

Cost of testing
• Test code early
• Increases overall software quality
• Reduces overall test costs

Copyright 2013 – QA Systems Ltd Slide 3

 Objectives

I want an easy life

Write tests in a language that I already know

Test harness that is easy to reuse across projects
Results
Copyright 2013 – QA Systems Ltd Slide 4
 Where do I start?

Dumb approach: start writing a system-level test

Why won’t this work? Coverage

Intelligent approach: break down my system

& test individual components

Copyright 2013 – QA Systems Ltd Slide 5

 Unit Testing
Imagine your system:

Unit A

Unit B Unit C Unit D

Unit E Unit F Unit G

Unit H Unit I Unit J

Copyright 2013 – QA Systems Ltd Slide 6

 Testing in isolation

Problem: dependencies

Smart solution: Stubs / mocks

Stub/Isolate
Source Called
for Called
Code Object
Object

Copyright 2013 – QA Systems Ltd Slide 7

 Unit Testing

Dumb approach: simply write a test for each

of my components

Fairly easy to do: but what does this prove?

Intelligent approach: Start from requirements

Highlight unwanted / unnecessary code

Copyright 2013 – QA Systems Ltd Slide 8

 When do I stop?

Dumb approach: when I’m bored?

Once I’ve written a test for every component?
At home time?

Copyright 2013 – QA Systems Ltd Slide 9

 Analyse Coverage

Unit A

Unit B Unit C Unit D

Unit E Unit F Unit G

Unit H Unit I Unit J

Copyright 2013 – QA Systems Ltd Slide 10

 Analyse Coverage

Rule sets
Drill-down
views

Copyright 2013 – QA Systems Ltd Slide 11

 Analyse Coverage

Intelligent
highlighting

Copyright 2013 – QA Systems Ltd Slide 12

 Increase Coverage

How do I improve my coverage?

Write more tests and hope for the best

Copyright 2013 – QA Systems Ltd Slide 13

 Inputs

Increase Coverage
Public

Intelligent approach: table-driven tests  Outputs

Vary input parameters

Work with large
data sets

Copyright 2013 – QA Systems Ltd Slide 14

 Other tools

Auto-generation of tests
Test 1 Test 2 Test 3 Test 4

Unit A

Test optimisation

Copyright 2013 – QA Systems Ltd Slide 15

 Are we there yet?

Analysed Coverage
Maximise Coverage
Optimised Tests

Copyright 2013 – QA Systems Ltd Slide 16

 White-box testing
“…is a method of testing software that tests internal structures
or workings of an application, as opposed to its functionality”
source: Wikipedia

Copyright 2013 – QA Systems Ltd Slide 17

 Inputs
 Internals
White-box testing
Public Private

 Outputs
Accessing static data and functions
Must be
unobtrusive

Copyright 2013 – QA Systems Ltd Slide 18

 Integration Testing
Looking at how the components interact

Unit A

Unit B Unit C Unit D

Unit E Unit F Unit G

Unit H Unit I Unit J

Copyright 2013 – QA Systems Ltd Slide 19

 Wrapping

What is wrapping?
Wrapper

Check Out Parameters BEFORE Modify Out Parameters

Check Call order Wrapper

Source Called
Code Object

Modify In Parameters AFTER Check In Parameters

and Return Wrapper and Return

Copyright 2013 – QA Systems Ltd Slide 20

 Results
Multiple report formats
• Console Summary
• Test Results Explorer
drill down to each check
links to test script
• XML Test Report
• ASCII text results with Outline

Detailed results
• Checks list actual against expected
• Summaries by
Test projects
Test cases
Individual checks
• Reports Filtered for pass/fail/warnings

Copyright 2013 – QA Systems Ltd Slide 21

 Summary

Easily analyse
coverage Check static data

Table-driven tests Wrap function calls

Auto-generate Results

Optimise

Copyright 2013 – QA Systems Ltd Slide 22

 Selection of Cantata Customers

Copyright 2012
Copyright 2012 –– QA
QA Systems
Systems Limited
Limited Test
Test SlideTest
Slide 23
Test
 Take Cantata for a Test Drive

> Free Evaluation

• Full featured product

• Full technical support
• See how leading companies do intelligent testing

Copyright 2012
Copyright 2012 –– QA
QA Systems
Systems Limited
Limited Test
Test SlideTest
Slide 24
Test
 Thank You

Thank you for

your attention!
Any Questions?

Copyright 2012 – QA Systems Limited Test SlideTest

25