You are on page 1of 23

Minimization and Prioritization of Test Cases

Regression Testing
Software maintenance is expensive Development may take 2 to 4 years

Same may have to be maintained for 10 to 15 years

Regression Testing
Reason for software change
1. Errors during actual use 2. Additional functionality 3. Changes in external world 4. Restructuring work 5. Change in technologies 6. Obsolete capabilities to be deleted.

Regression Testing
Development testing When we modify software, we typically retest it.

This retesting is called regression testing. Regression testing is the process of retesting the modified parts of the software and ensuring that no new errors have been introduced into previously tested code.

5 .Regression Testing Purposes Increase confidence in the correctness of the modified program. Regression Testing S. Locate errors in the modified program. Preserve the quality & reliability of software. 1 Development Testing Regression Testing We can make use of existing test suites and test plans We retest modified components or affected by modifications No time We create test suites and test plan 2 We test all software components Budget give time for testing 3 4 Once on software product Performed under the pressure of release date Many times Performed in crisis situations. under greater time constraint.N o. We may perform regression testing during latter stage of software development. Ensure the software’s continued operation.

These test case minimization techniques attempts to find redundant test cases Reducing the Number of Test Cases Prioritization of Test Cases A test case with highest rank has the highest priority and second highest rank test case has second highest priority and as so on.Reducing the Number of Test Cases MINIMIZATION OF TEST CASES Select all those test cases that traverse the modified portion of the program and the portion that is affected by the modification . .

Reducing the Number of Test Cases Prioritization of Test Cases Test case prioritization General test case prioritization Version specific test case prioritization Reducing the Number of Test Cases Prioritization of Test Cases In general test case prioritization. for a given program with its test suite. In version specific test case prioritization. when the original program is changed to the modified program. with the knowledge of the changes that have been made in the original program. . we prioritize the test cases that will be useful over a succession of subsequent modified version of original program without any knowledge of modification(s). we prioritize the test cases.

if time permits Not important test case Redundant test case . Most important is the “impact of failure” which may range from “no impact” to “loss of human life” and must be studied very carefully. All risky functions should be tested on higher priority. Reducing the Number of Test Cases Priority Category Scheme Priority code 1 Priority code 2 Priority code 3 Priority code 4 Priority code 5 : : : : : Essential test case Important test case Execute.Reducing the Number of Test Cases Prioritization Guidelines Any guidelines should address two fundamental issues like: What functions of the software must be tested? What are the consequences if some functions are not tested? All prioritization guidelines should be designed on the basis of risk analysis.

e. Risks may be ranked on the basis of its risk exposure.Reducing the Number of Test Cases Priority Category Scheme Priority code 1 : Important for the customer Priority code 2 : Required to increase customer satisfaction Priority code 3 : Help to increase market share of the product Risk Analysis What is risk? Probability of occurrence of a problem (i. an event) Impact of that problem Risk analysis is a process of identifying the potential problems and then assigning a ‘probability of occurrence of problem’ value and ‘impact of that problem value for each identified problem. Both of these values are assigned on a scale of 1 (low) to 10 (high). . A factor ‘risk exposure’ is calculated for every problem which is the product of ‘probability of occurrence of problem’ value and ‘impact of that problem’ value.

No. Potential Problems Probability of Impact of Risk occurrence of that Exposure Problem problem 2 6 3 2 1 2 8 3 3 2 3 2 10 9 1 2 1 4 6 12 9 4 10 18 8 6 2 8 1 2 3 4 5. No. 6 7 8 9 10 Issued password not available Wrong entry in students detail form Wrong entry in scheme detail form Printing mistake in registration card Unauthorised access Database corrupted Ambiguous documentations Lists not in proper format Issued login-id is not in specified 2 format School not available in the data base 2 .Risk Analysis Risk Analysis Table S. Potential Problem Probability occurrence problem 1 2 3 4 of Impact of that Risk of Problem Exposure Risk Analysis S.

1. 8. 7. 5. 4 and 9.Risk Analysis • The potential problems ranked by risk exposure are 6. 3. We may use the risk matrix to assign thresholds that group the potential problems into priority categories. 10. estimate their probability of occurrence with impact and rank the risks based on this information. . 2. Risk Analysis Risk Matrix Risk matrix is used to capture identified problems.

. but PC-1 and PC-4 will remain the same. PC-2 and PC-3 will swap. Hence.Risk Analysis Threshold by quadrant 5 10 9 8 7 6 PC-3 PC-1 Impact of the problem 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 10 1 0 1 4 9 PC-4 3 8 2 7 PC-2 Probability of occurrence of the problem Risk Analysis The priority category in defined as: Priority category 1 (PC-1) Priority category 2 (PC-2) Priority category 3 (PC-3) Priority category 4 (PC-4) = High probability value and high impact value = High probability value and low impact value = Low probability value and high impact value = Low probability value and low impact value We may decide to give more importance to high impact value over the high probability value.

Risk Analysis Alternate Threshold by quadrant 5 1 0 9 8 7 Impact of the problem 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1 0 1 0 1 4 9 PC-4 3 8 2 7 PC-3 PC-2 PC-1 6 Probability of occurrence of the problem Risk Analysis Threshold by diagonal quadrant 10 9 8 Impact of the problem 7 6 5 4 3 2 1 PC-4 PC-3 PC-2 PC-1 0 1 2 3 4 5 6 7 8 9 10 Probability of occurrence of the problem .

Risk Analysis Threshold based on high “Impact of Problem” value 1 0 9 8 7 Impact of the problem 6 5 4 3 2 1 0 1 PC-5 PC-4 PC-1 PC-3 PC-2 2 3 4 5 6 7 8 9 Probability of occurrence of the problem 1 0 Risk Analysis Threshold based on high “probability of occurrence of problem” 1 0 9 8 Impact of the problem 7 6 5 4 3 2 1 PC-5 PC-3 PC-1 PC-4 PC-2 0 1 2 3 4 5 6 7 8 9 Probability of occurrence of the problem 1 0 .

The impact of the problem may be critical. 11. 28. 21. 17. 25. the high priority risks are identified. 35. 15. 58. 10. 23. 41. 32. 31. 46 46. 47. 55 55. 2. 50. serious. 38. 51 5. 34. 4. 59 3. 14. 20. 19. 13. 29. 37. 41. 48. moderate. 52 6. 40. 36 26. 60 Test Case Array . 9. 3. These risks should be discussed in a team and proper action should be recommended to manage these risks. 12. 7. 8. 27. 57. All that matter when using risk matrix is the relative order of the probability estimates (which risks are more likely to occur) on the scale of 1 to 10. 50 1. 9.Risk Analysis After the risks are ranked. A risk matrix has become a powerful tool for designing prioritization schemes.42. minor or negligible. 45. 40. These risks are required to be managed first and then other priority risks and so on. 23. 16. 39 15. 20. 54 5. 24. 33. 30. Regression Testing Execution history is given below T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 1. 53. 22. 24. 18. 49. . 4. 43. 44. 56. 6.

45 and 55 are modified.Regression Testing Suppose 1. T8 and T9 Total : 8 Regression Testing Test cases T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 Modified lines 1. T3.2. 15 15. T7.5. Simplest is to execute all test cases that have any of the modified lines of code. T6. T2. of matches 2 1 1 0 2 2 1 1 1 0 . Selected test cases : T1. 2 1 5 -5.15. 35 45 55 55 -- No.35. T5.

of matches (nfound) 2 2 2 1 1 1 1 1 Matches found 1. 15 15. Regression Testing Test cases T1 T5 T6 T2 T3 T7 T8 T9 No. 35 1 5 45 55 55 Candidate 1 0 0 0 0 0 0 0 Test cases in descending order . 2 5. This test case will be executed first & is shown on next slide. The test case that has the maximum number of matches is selected by making its candidate value = 1.Regression Testing Number of matches found (NFOUND) is stored in an array and elements are stored in descending order.

1 and 2 LOCs that are still to be executed: [ 1. 35 5 45 55 55 Candidate 1 0 0 0 0 0 .Regression Testing T1 is selected for execution and it covers line nos. 15. 55 ] – [ 1. of matches (nfound) 2 2 1 1 1 1 Matches found 5. 45. 55] Again. 15 15. we check for the number of modified lines of code covered by each test case and sort them in descending order & select the one with maximum number of matches. 15. 35. 35. 45. Regression Testing Test cases T5 T6 T3 T7 T8 T9 No. 2. 5. 2 ] = [5.

15. LOCs that are still to be executed: [ 5. of matches (nfound) 1 1 1 1 Matches found 35 45 55 55 Candidate 1 0 0 0 . 15 ] = [35. 55 ] – [ 5. 45. 35. Regression Testing Test cases T6 T7 T8 T9 No. we check for the number of modified lines of code covered by each test case and sort them in descending order & select the test case with maximum number of matches found. 5 and 15.Regression Testing T5 is selected for execution and it covers line nos. 55] Again. 45.

45. T5. T8 . LOCs still to be executed : [ 45. T7. 35.Regression Testing T6 is selected and covers line no. T6. 55 ] Test cases T7 T8 T9 No. of matches (nfound) 1 1 Matches found 55 55 Candidate 1 0 T8 is selected for execution Order : T1. of matches (nfound) 1 1 1 Matches found 45 55 55 Candidate 1 0 0 Regression Testing T7 is selected for execution covering line no. LOCs still to be executed : [ 55 ] Test cases T8 T9 No.

3. 17. 17. Regression Testing Test Cases T1 T2 T3 T4 T5 1. 8. 13 Execution History Modified lines : 6. 5. 15. 14. 5. 15 . 7. 2. 8. Example has 20 lines of code.10. 18 1. 20 Deleted lines : 4. 6. 13. 9. 19 1.Regression Testing Out of 10 test cases. 12. 5. 11. we need to run only 5 test cases for 100% code coverage of modified lines of code. 13. 5 Test Cases as given on next slide. 4. 2. 8. 7. 20 2. 8. 50% saving of test case Second Case Lines of code have been modified and also deleted. 20 6. 17. 9. 16.

We come to know that T1 & T5 are redundant test cases. 12. 13 Regression Testing Now. 17. 8. 5. 13. 9. 5. 20 2.7. This situation has become important due to deletion of few lines. 3. 9. .Regression Testing From the test history information. 2. 14. 8. 8. Test case array will be: Test Cases T1 T2 T3 T4 T5 1. 20 Execution History 6. 6. If we see execution history carefully.15. 16. 8. 18 1. remove the deleted lines say : 4. 17. we want to find redundant test case. 19 1. 5.10. 2. 11..

8. 20 Execution History 6. 11. 16. 19 . 2. 17. 5.Regression Testing Test cases T1 Line numbers 1 5 20 Found in test case T4 T2 T2 T4 T2 T3 T3 T3 T3 Redundant Y/N Y Y Y Y Y Y Y Y Y T5 1 2 6 8 9 13 Regression Testing The remaining test cases are = T2. 17. T3. 18 1. 3. 9. 5. 8. 10. 12. 8. 13. 14. T4 Test case array is: Test Cases T2 T3 T4 2.

17 20 17 Candidate 1 0 0 . 13. of matches found T3 T2 T4 3 1 1 Line numbers 6. of matches found (nfound) 1 3 1 Regression Testing Test cases are sorted on the basis of matches found (descending order) Test cases No. 13. 17.Regression Testing Number of matches for modified lines are Modified lines are : 6. 13. 17 17 No. 20 Test cases T2 T3 T4 Matches found 20 6.

Remaining modified line is : 20 Hence T3 and T2 are sufficient to execute modified lines. Saving : 60% .Regression Testing T3 is selected for execution. T1 & T5 should be deleted from the test suite.