FUNDAMENTALS OF SOFTWARE TESTING

Rahesh Dudani
Certified Quality Manager Certified Software Quality Professional Certified Software Test Manager Certified Lead Auditor - ISO 9001:2000 Certified Lead Auditor – BS 7799:2002 e mail: rahesh.dudani@gmail.com phone: 2569247 mobile: 9370148115
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

1 of 44

SOFTWARE TESTING
"Primary role of testing is not demonstration of correct performance, but the exposure of hidden defects." - G. J. Myers
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

2 of 44

SOFTWARE TESTING
• It is not sufficient to demonstrate that the software is doing what it is supposed to do • It is more important to demonstrate that the software is not doing what it is not supposed to do.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

3 of 44

TRADITIONAL DEFINITIONS OF TESTING
• Establishing confidence that a program does what it is supposed to do (Hetzel, 1973) • Confirming that a program performs its intended functions correctly • THESE DEFINITIONS ARE INCOMPLETE AND DESCRIBE ALMOST THE OPPOSITE OF WHAT TESTING SHOULD BE!!
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

4 of 44

WHAT TESTING ACTUALLY IS
• Identifying the differences between expected and actual results • Process of executing a program (or part of it) with the intention or goal of finding defects • Detecting specification errors and deviations from the specifications.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

5 of 44

ESSENTIALS OF SOFTWARE TESTING
• TESTING IS DESTRUCTIVE PROCESS : A CREATIVE DESTRUCTION • TESTING NEEDS A SADISTIC APPROACH • If a test does not detect a defect actually present, is it successful or unsuccessful? • Test that detects a defect is valuable investment: it has helped in improving the product..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

6 of 44

ESSENTIALS OF SOFTWARE TESTING

• It is risky to develop software and not test it • High pressures to deliver software as quickly as possible: test process must provide maximum value in shortest time.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

7 of 44

VERIFICATION
• Disciplined approach to evaluate whether a software product fulfils the requirements or conditions imposed on them (are we building the product right?) • Method: walkthrough, formal inspection and review of each software product • Also called static testing..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

8 of 44

VERIFICATION
• Done by systematically reading the contents of a software product with the intention of detecting defects • Helps in identifying not only presence of defects but also their location • A ‘filter’ applied at various points during the SDLC to ‘purify’ the product as it progresses through various phases.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

9 of 44

VALIDATION
• Disciplined approach to evaluate whether the final, as-built software product fulfils its specific intended use (are we building the right product?) • Method: testing each software product at each phase of life cycle using test plans, test cases for properly selected types of tests • Also called dynamic testing..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

10 of 44

VALIDATION
• Done by systematically testing a software product with the intention of finding defects • Helps in identifying the presence of defects, not their location • Necessary to demonstrate not just that the software is doing what it is supposed to do, but also is not doing what it is not supposed to do

FUNDAMENTALS OF SOFTWARE TESTING

SICSR

11 of 44

WALKTHROUGH
• An informal process, initiated by the author of a software product to a colleague for assistance in locating defects and for suggesting improvements
» » » » » normally not planned author explains the product colleague comes out with observations author provides clarification if required author notes down relevant points and takes corrective actions.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

12 of 44

INSPECTION
• A thorough, word-by-word checking of a software product (or part of a product) with the intention of:
» locating defects » confirming traceability of relevant requirements » checking for conformance to relevant standards and conventions.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

13 of 44

REVIEW
• Normally done at the end of a phase of SDLC, when the author/s feel/s that the product is error-free and can go to the next phase
» Usually conducted by a manager » Main purpose is to decide if the product can go the next phase » Includes to check if suitable inspections have been done » After review, the product is ‘base-lined’.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

14 of 44

GUIDELINES FOR INSPECTIONS
• Plan the inspection • Allocate trained resources • Moderator / inspection leader circulates the product to be inspected and checklist to inspection team in advance • Product is clear-compiled, checked for spellings, grammar, formatting, etc..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

15 of 44

GUIDELINES FOR INSPECTIONS
• Inspection team of 3 to 5 members: roles - moderator, author, scribe, domain expert, user • Maximum time: 1 to 1 1/2 hour per inspection; size of product to be selected accordingly • Inspect the product; not the person behind it..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

16 of 44

GUIDELINES FOR INSPECTIONS
• Author / moderator reads the product, line by line • Concentrate on locating defects; avoid discussions on possible solutions • Classify and record the defects • Review the inspection process • Products that can be inspected: SRS, Design, Code, User Manual, Installation Manual, Help Screen, Various plans, etc.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

17 of 44

ECONOMICS OF TESTING
Optimum testing
Number of defects Cost of testing

Duration

18 of 44

ESTABLISHING A TESTING POLICY

• Good testing does not just happen • It must be planned • Automation is not the only answer • Avoid ‘Anti-testing’ messages.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

19 of 44

TEST PROCESSES
• • • • • • • Establish test objectives Design test cases Write test cases Review test cases Execute the tests Examine test results Repeat the tests (regression).
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

20 of 44

PRINCIPLES OF SOFTWARE TESTING • Good testing MUST examine:
» users’ requirements » design objectives » user interface » internal structure » AND then execute the code..

FUNDAMENTALS OF SOFTWARE TESTING

SICSR

21 of 44

PRINCIPLES OF SOFTWARE TESTING
Probability of detecting more defects
The probability of detecting new defects is proportional to the number of defects already detected:

Number of defects detected

22 of 44

PRINCIPLES OF SOFTWARE TESTING
• Testing is not something done as a formality at the end of development; it is done throughout the SDLC to detect defects as early as possible and prevent defect migration to subsequent phases • SOFTWARE TESTING =
» Verification (Static Testing) »+ » Validation (Dynamic Testing).
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

23 of 44

TEST METHODS
• Black box testing (also called functional testing):
» Test to the specs (Do not look inside the code)

• White box testing (also called structural / glass box testing):
» Test to the code (Do not look inside the specs).
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

24 of 44

OTHER IMPORTANT TESTS
• • • • • • • • • • • Stress Testing Usability Testing Security Testing Storage Testing Recovery Testing Procedure Testing Compatibility Testing Installability Testing Documentation Testing Configuration Testing Parallel Testing.
SICSR 25 of 44

FUNDAMENTALS OF SOFTWARE TESTING

TEST PLAN
• Test plan identifier: unique • Introduction: objective, background, scope, references • Test items: product to be tested, with version number; document against which each is to be tested • Features to be tested: traced to clause / para no. of referenced document.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

26 of 44

TEST PLAN
• Suspension criteria • Resumption criteria • Test deliverables: test cases, test logs, test incident reports, test summary • Environmental needs: hardware, software, communication lines, special tools, stubs, drivers, etc. • Responsibilities: managing, designing, executing, witnessing, checking, approving, resolving..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

27 of 44

TEST PLAN
• • • • Skills required Training needs, providing training Risks and contingencies Completion criteria: cost, duration, defects found, defects remaining, coverage.

FUNDAMENTALS OF SOFTWARE TESTING

SICSR

28 of 44

• THE SECRETS OF GOOD TESTING:
» Proper test plans » Systematically prepared test cases

• Without these, you are doing ‘MONKEY TESTING’.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

29 of 44

CASE AND CAST TOOLS

FUNDAMENTALS OF SOFTWARE TESTING

SICSR

30 of 44

MANUAL SYSTEMS
• Difficult to obtain compliance
» » » » Inexperienced persons lack perspective/ motivation Procedures may be felt cumbersome/irritating Human error – forget/by – pass – “just this once” Some persons see it as a block to creativity and against their fundamental right to circumvent it.

• Require more “Training” on procedures • Suffer more in case of personnel turnover
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

31 of 44

AUTOMATED SYSTEMS
• Can be done at various levels (Organization, Project) • Beneficial for routine and cumbersome tasks • Automation is more difficult to circumvent, hence compliance is better, less non-compliances in audits • Less chances of error • Team members have more time for non-routine activities, can concentrate on work needing human involvement.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

32 of 44

CASE / CAST TOOLS
• Tools which automate various activities related with software development:
» Project Management, analysis, design, coding, testing, documentation, etc.

• Help to reduce effort required for development of a product..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

33 of 44

CASE / CAST TOOLS
• Help to build quality into a product • Not the ultimate solution: skilled people are required to understand, select and use proper tools effectively • Can be a single tool for supporting a specific activity or a combination of a number of components.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

34 of 44

• Business Process Engineering and modeling tools:
» Represent business data objects, their relationships, processes

• Project planning tools:
» Estimating size, effort, duration, resources » Network modeling, interdependencies, scheduling

• Risk analysis tools:
» Handling list of risks, their analysis

FUNDAMENTALS OF SOFTWARE TESTING

SICSR

35 of 44

• Project management tools:
» Monitoring of progress of project » Collecting metrics related to project performance

• Requirements tracing tools:
» Create database of all requirements » Identify requirements traced to other products

• Metrics management tools:
» Collect and update various metrics like KLOC, defects / KLOC, effort in person-days / person-months, etc.

FUNDAMENTALS OF SOFTWARE TESTING

SICSR

36 of 44

• Documentation tools:
» Contain templates for various documents » Provide facilities for updating documents only by authorized personnel » Handle version control of documents

• Database management tools:
» Handle project database

• Software Configuration Management tools:
» Configuration identification, change control, library control, version control, configuration status, configuration audit, etc.
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

37 of 44

• Analysis and design tools:
» Help create models of the system to be built (Data Flow Diagrams, Control Flow Charts, etc.)

• Prototyping / simulation tools:
» Develop mock-ups of the system to be built like screen and report layouts

• Interface design and development tools:
» Tool kit for components like menus, icons, buttons, scrolling mechanisms, etc.

FUNDAMENTALS OF SOFTWARE TESTING

SICSR

38 of 44

• Programming tools:
» Various compilers, editors, application generators, OO programming environments, query languages, graphical programming environments, etc.

• Web development tools:
» Generating elements of web pages: text, colours, fonts, graphics, etc.

• Static testing tools:
» Performing walkthroughs, inspections and reviews of products » Complexity analysis, code comprehension, syntax and semantic analysis..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

39 of 44

• Tools for test planning:
» Templates for test plans » Test effort and staffing estimates (based on complexity, size) » Test schedules, monitoring

• Tools for test design:
» Test data generators » Requirements based test design » Coverage analysis: statement coverage, condition coverage, decision coverage, multiple condition coverage, path coverage..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

40 of 44

• Tools for test execution and evaluation:
» » » » » » » » » Capture / playback Coverage analysis Test case management Collection of defect metrics Defect tracking Client / server testing tools: load tests, stress tests Web testing tools Testing communication applications, networks Simulate hardware configuration.
FUNDAMENTALS OF SOFTWARE TESTING

• Tools for simulation and performance:

SICSR

41 of 44

ISSUES TO CONSIDER
• • • • • • • Integration with other tools/development platform Scaling to support growth (no. of users) Security/protection features Different types of user interfaces Supports multiple site development Supports different operating systems Need for hardware resources (some tools require considerable disk space, RAM and CPU power)..
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

42 of 44

ISSUES TO CONSIDER
• Price and licensing terms • Delivery period • Caliber of development/support staff : Training – Conceptual and tool based • Vendor attributes • Opinion of customer base • Utility to the organisation • Stability • Customizing to suit your process: or customizing your process to suit the tool?
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

43 of 44

COSTS OF AUTOMATION
• Automation
» Cost of tool » Cost of training » Cost of changing “way of working”

• Non-automation
» Time spent doing the tasks manually » Efforts in correcting problems due to
• Non compliance • Human errors
FUNDAMENTALS OF SOFTWARE TESTING

SICSR

44 of 44