Test Design Techniques

Asking the right questions

Information objectives 1
!

Find defects Block premature product releases Help managers make ship / no-ship decisions Minimize technical support costs Assess conformance to specification

!

!

!

!

Copyright (c) Cem Kaner and James Bach, kaner@kaner.com

Information objectives 2
!

Conform to regulations Minimize safety-related lawsuit risk Find safe scenarios for use of the product Assess quality Verify correctness of the product Assure quality
Copyright (c) Cem Kaner and James Bach, kaner@kaner.com

!

!

!

!

!

To different degrees, good tests have these attributes:
!

Power. When a problem exists, the test will reveal it. Valid. When the test reveals a problem, it is a genuine problem. Value. It reveals things your clients want to know about the product or project. Credible. Your client will believe that people will do the things that are done in this test. Performable. It can be performed as designed. Maintainable. Easy to revise in the face of product changes. Repeatable. It is easy and inexpensive to reuse the test.
Modified Slide: Original by Cem Kaner and James Bach, kaner@kaner.com

!

!

!

!

!

!

Provides useful information for the debugging programmer. Supports troubleshooting. and prove you ran it. you can hit it with more complex tests and more closely simulate use by experienced users. Appropriately complex. As the program gets more stable. What IS a Good Test Case? .Test attributes ! Coverage. Accountable. as well as direct costs. justify. Easy to evaluate. includes time and kaner@kaner.com Copyright " ! ! ! ! ! See Kaner.effort. This(c) Cem Kaner and James Bach. You can explain. It exercises the product in a way that isn't already taken care of by other tests. Cost.

Requirements and How Testers can Help making them better ! Official requirements specification (in theory) # # # Whatever information we can find in the project! Plus functionality that is NOT written but needed Plus all possible/likely problems that we need to test ! Create a Test Design Model # Quality assurance of the requirements ! Test analysis and design # Is about asking the right questions # Good Test Design often finds more defects than test execution! .

Unified Process (in Theory) ! System Requirements # # # # Use Cases Supplementary Specification System Vision Additional Document describing Business Rules? ! Business Requirements # # Information Model Processes ! Technical # # System Architecture Document Navigation Map .

Finally do advanced testing using some creative thinking . Add test data for low level test cases iv. Analyse test basis and create a model ii. Cover the model with high level test cases iii.Test Design – The Four-Step Workflow i.

Skottland.-.line Mata ut kort .Examples of Models Stoppa in kort H Bokbeställning Beskrivning Parameter Antal – Giltiga grupper Ogiltiga grupper Verifiera giltighet A1 Felaktigt kort Be om kod Mata in kod A2 Kontrollera kod Felaktig kod Testfall Prio Id 1 1 X A3 Be om transaktion Ät kortet Felaktig kod tredje gången - Värde Heltal 1-99 Prio 1 2 2 3 3 3 3 2 2 2 2 3 2 4 X 3 5 X 3 6 3 7 Mata in transaktion Heltal <1 Heltal >99 Decimaltal 1.01-99. Nordirland.* räkneoperatorer Övriga ickenumeriska tecken Antal – 1 giltiga gränsvärden 99 Antal – 0 Ogiltiga 100 gränsvärden Land – grupper England. Wales Övriga länder i Europa Övriga världen Land – Första landet på listan gränsvärden Sista landet i listan Betalningssätt – Faktura grupper VISA. pengar & kvitto X X 2 3 1 1 1 3 3 1 1 1 X X X X X X X X X X X X A5 Kunden tar ej pengarna Sug in pengar & skriv till fellogg . MasterCard Postförskott A4 X Kontrollera transaktion Felaktig transaktion Mata ut kort A6 Annan bank ej on .99 Bokstäver +.

All Science is Based on Models All models are wrong but some are useful. .Gerald Weinberg ! A good model is easy to understand A good model is a true simplification of reality Too little information makes the model incomplete Too much information makes the model hard to understand Quality Assurance of the model is very important ! ! ! ! .

user profiles. scenarios.Test Design Models ! ! All techniques use some kind of model Problem Areas # # # # # # Data – tables. syntax . diagrams Flows – flow graphs Logic – tables and trees Combinatorics – free tools. formulas Event-based systems – graphs. tables. trees. tables Overall tests – graphs.

Test Design Techniques 1 ! Data # # # # Equivalence Partitioning Boundary Value Analysis Domain Testing Classification Trees ! Logic # # # Decision Tables Decision Trees Modified Condition Decision Coverage ! Event-based # # State graph State table .

Test Design Techniques 2 ! Combinatorics # All-Pairs or more using tools like PICT ! Overall Testing # # # # # Scenarios Charters Data Cycles Time Cycles Syntax ! Flows # # # # General thoughts on coverage Control and data flows for unit testing Business Process Testing Use Case Testing .

Test Data A Test Design Technique But Also Step Three in the Test Design Process .

groups have boundaries Defects are often found here. >. ! ! ! Confusion <.g.. ". !.Test Data ! Equivalence Partitions # How do you group values so that it is enough (in theory) to test only one value in each group? ! Boundary Values # # Some. e. = Start at 1 instead of 0 Number of characters/values as input/output . but not all.

Number of books per ordered title is 1-99.Example: Numerical Partition ! Order books on the Internet. Which are the partitions? Parameter Number of books Valid Number of books Invalid Invalid Partition Integer 1 -99 Integer <1 Integer >99 Valid Invalid 1 99 -3 17 121 .

plus another one in the middle . max. typical.Is the Partitioning Correct? ! ! Possible factors that play a role: # Ordering of more than one book with the same title may give a discount # Ordering for a certain sum may give you free postage # Odering over a certain amount might require pre-paid delivery # Ordering many books may require delivery in several packages # Default values may exist Solution: test several values for each partition # Min. default.

1 .-.What else is invalid? Parameter Number of books Valid Number of books Invalid Partition Integer 1 -99 Integer <1 Integer >99 Decimals 1.01 -99.99 Letters +.$ Empty field Ask for a robust design! That will make it easier to test AND build.* operators Other non -numericals like @.

…and the boundary values ! For number of books 1-99 Parameter Number of books Valid boundaries Number of books Unvalid boundaries Boundary Value 1 99 0 100 Invalid Valid Invalid 01 99 100 .

The Complete List ! For number of books 1-99 Invalid Valid Invalid -3 01 17 99 100 121 ! Additional test data # # Empty field Any non-integers .

Parameter Country Valid Country Invalid Partition England. North ern Ireland. dearer. Wales Rest of Europe Outside Europe Empty field ”all other values”. Scotland. May not be possible but try anyhow! F:Postage GBR Count ries SWE NOR Etc. . •For the rest of the world outside Europe there is another.Postage •Adress in Great Britain: free postage •For the rest of Europe there is a special postage list. list.

working-days etc # Are usually seen not to have boundaries ”Free format input field”: both values and number of characters # Has valid and invalid partitions # Has boundaries Logical Values: Yes/No/Not selected # No boundaries ! ! ! .Variations on partitioning I ! Interval: continuum of numerical values # Has valid and invalid boundaries Sets: countries.

withdrawal amount vs account status # Valid and invalid boundaries exist Sequence of values: Airport Code.Variations on partitions II ! Time dependent: databases. printers. Currency Code # Usually no boundaries Number of values: Too many. entering data slowly or fast # Can have valid/invalid boundaries Relative partitions: value in one field depends on value in another: e. too few # Valid and invalid boundaries exist ! ! ! .g.

All partitions Parameter Number of books Valid Number of books Invalid Partition Integer 1 -99 Integer <1 Integer >99 Decimals 1.Scotland Wales Rest of Europe Rest of world Empty field Other values First Country on list Last country on list Valid boundaries Invalid boundaries Country valid Invalid Country .* operators Other non -numericals like @.01 -99.99 Letters +. Nor thern Irelan d.$ 1 99 0 100 England.-.

Data 1: EP&BV: Use a Table Employee test data Description Parameter Name Partition Regular name Double-name Short Name Long Name Changed when married Foreign Name Age Below 18 18-65 Above 65 Regular employee Temporary Monthly pay Hourly pay X X X X X X X X Employees Id 1 X 2 X 3 4 X 5 X 6 7 X X X X X X Classification Salary Amount More partitions=better coverage .

Data 2: Classification Trees Card Deeper tree=better coverage Unreadable Readable ATM Other Valid Invalid Petrol Library Our bank Other Expired Blocked Demo CTree Editor .

Data 3: Domain testing ! ! Partitions created by several parameters working together In principle this is a set of rules More boundary tests = better coverage .

Exercise ! Define equivalence partitions and boundary values for the following: $ Last % Name(20 char) &%Age (of a person) '%Fulltime worker Y/N ! Any questions regarding the parameters # Believe nothing. question everything .

2.9.7} {15.4.2.15} {3.2} {Q.3} {2.2. {5. 4.5.15. 2. 7.4.4.6.4. 0. 0. 5. 11.2. 6. 2.1.5. 0. 8.4. 7.3} {1.0. 7.3.2. 9.4.0.2.Examples of Myers' categories 1.0. 4.6. 2.6.0.5.8.6.3.7} {0.3.2. 3.1.0.0.8.6} (examples courtesy of Doug Hoffman) . 5. 8.0} {3.-6} {1.0. 3.

Testing Logic Decision Tables Decision Trees .

<=65 >45. <=65 >65 Sex Male Male Female Female Car Type Family. <=45 >45. <=45 >25.Logic ! Variations # # # Set of Rules Conditions Formulas Explicit variations 1 2 3 4 5 6 7 8 9 10 11 Age <=25 <=25 <=25 <=25 >25. <=45 >25. <=65 >45. Regular Sport Family Sport Regular Family Sport Regular Sport Fee 2000 3000 500 1500 300 1500 500 200 1400 400 1500 ! Design techniques # # Decision Table Decision Tree . Regular Sport Family.

1/0 or X or the value itself Rule 1 Rule 2 Y 1 N 0 Y N 0 1 Rule 3 X X Rule 4 <5 True 300 No Condition 1 Condition 2 Result 1 Result 2 X .Format used ! ! ! Either put the rules in rows or in columns One rule per column – one test case per row makes is easy to create test data Enter Y/N.

Business Rule 3: Everyone over 25 with a family type are subject to an excess reduced by 200 kronor. Business Rule 2: Men under 25 years of age are subject to an excess increased by 1500 kronor.Business Rules: Car Insurance ! Insurance price depends on a number of factors $ % &% '% ( % )% *% + % . Business Rule 4: All persons between 45 and 65 are subject to an excess reduced Clients above 25 with car type family have a reduced fee of 100 SEK Every one between 45 and 65 have an extra reduced fee of 100 SEK Car type sport gives an extra fee of 1000 SEK Car type regular does not effect the fee Everyone above 65 has no reductions whatsoever ! How do you test this? . % % Business Rule 1: As a starting point.the excess for all policy holders is 500 kronor.

25-45. >65 Car Type: family. car type. 45-65. gender Partitions: Age: <25. female . sport Gender: male.Step 1: Identify Parameters? Parameters: age. regular.

<=45 >45. <=65 >45.List Rules in a Table – Are They Correct? Explicit variations 1 2 3 4 5 6 7 8 9 10 11 Age <=25 <=25 <=25 <=25 >25. Regular Sport Family Sport Regular Family Sport Regular Sport Fee 2000 3000 500 1500 300 1500 500 200 1400 400 1500 4*2*3 = 24 possible combinations in total . Regular Sport Family. <=45 >25. <=65 >65 Gender Car Type Male Male Female Female Family. <=45 >25. <=65 >45.

Draw a tree by selecting one rule and one parameter at a time # # Complete? Not overlapping? ! Create # # # One high level test case for each leaf Low level test cases depending on boundary values Low level level test cases depending on logic More combinations (rules)=better coverage .Decision trees ! Start with the table you created.

Create a Decision Tree Missing .

Regular Sport Family. <=65 >45. Regular Fee 2000 3000 500 1500 300 1500 500 200 1400 400 1500 500 4*2*3 = 24 possible combinations in total . <=45 >25. <=65 >45. <=45 >25. Regular Sport Family Sport Regular Family Sport Regular Sport Family.Add missing rule Explicit variations 1 2 3 4 5 6 7 8 9 10 11 12 Age <=25 <=25 <=25 <=25 >25. <=65 >65 >65 Gender Car Type Male Male Female Female Family. <=45 >45.

# Which model is the best for testing purposes? adapted from Clive Bates.Exercise: Decision Table and Tree ! ! On Saturday Peter has a lot to do. Part A: # # # ! Create a decision table for the rules above. If he awakes early and the weather is bad and he still has any money left then he will go shopping. But if he sleeps late he will instead hang the laundry. Is anything unclear? How many test cases do you suggest using? Why? ! Part B: Create a decision tree from the information above. What he will do depends on the circumstances. If he wakes early and it is sunny outside he will cut the grass. Grove Consultants #Exercise: .

<55 >25 . <55 >25 .Exercise: Decision Tree Rule 9 10 11 12 13 14 16 17 18 Property N N N N N N N N N Age >=55 >=55 >25 . <=25 <=120 <=120 >120 <=120 <=120 Nej Ja NonPmt Loss Income Inc b tax >1000 <=1000 Nej Nej Own C Raise TIP: Is the table complete? Excel for solution Original specification . <55 >25 . <55 >25 . <55 >25 . <55 <=25 No No No No No Yes >1.2 >25 <=5 >5.

<=75 <=75 TIP: Use domain analysis and create a graph to understand the last crux . <=426 >426 >120 >75 >120 >120 >50.Exercise: Advanced Decision Tree Rule 1 2 3 Property Y Y Y <=42 <=42 Age NonPmt N N N Loss <=100 >100 >100. <=426 <=120 >120 <=50 Income Income/Age 4 5 6 7 8 Y Y Y Y Y <=42 >42 <=42 <=42 N N N N Y >100 >100 >100.

Sign up to vote on this title
UsefulNot useful