Test Estimation Using Test Point & Use Case Point Slide 1 Agenda ? Why Test Estimation?

? Test Estimation Methods Test Point Analysis Use Case Point Analysis Case Study Slide 2 Quotation

In many organizations, software testing accounts for 30 to 50 percent of software development costs. Yet most people believe that software is not well tested before it is delivered. That contradiction is rooted in two clear facts: First, testing software is a very difficult proposition; and second, testing is typically done without a clear methodology & without appropriate Estimations
By Jim Heumann Requirements Management Evangelist Rational Software Slide 3 Why Test Estimation? Why Good Estimations are important: Schedule and Delivery of the project Resources planning Early Risk assessment Testing is often blamed for late delivery Testing time is squeezed Minimize the deviations between estimates planned vs. actual

Slide 4 Definition of Test Estimation

Test Estimation is the estimation of the testing size, testing effort, testing cost and testing schedule for a specified software testing project in a specified environment using defined methods, tools and techniques. Slide 5 Definition of Test Estimation Testing Size the amount (quantity) of testing that needs to be carried out. Some times this may not be estimated especially in Embedded Testing (that is, testing is embedded in the software development activity itself) Testing Effort the amount of effort in either person days or person hours necessary for conducting the tests Testing Cost the expenses necessary for testing, including the expense towards human effort Testing Schedule the duration in calendar days or months that is necessary for conducting the tests Slide 6 Test Estimation Methods Informal: Ad-hoc methods Percentage of development time Formal: Test Point Analysis Use Case Point Analysis

Slide 7 Test Point Analysis An extension of Function Point Analysis to high-level (System & Acceptance) testing TPA, the philosophy

Size

Strategy

Test Point Analysis

Estimated Test Effort

Productivity Slide 8 TPA, the model Function Points

Dynamic Test Points (function dependent) Test Strategy Static Test Points (function in-dependent)

Total# Test Points

Environmental Factor

Productivity Factor

Primary Test Hours Control Factor (management overhead) Total# Test Hours

Slide 9 Dynamic Test Points To calculate the test points for the individual functions, the influential variable and factors are divided into two categories: 1. Dynamic Quality Characteristics (Qd) sum of Qde - explicit characteristics & Qdi - implicit characteristics

2. Function dependent (Df) Slide 10 Dynamic Quality Characteristics (Qd) Four dynamic, explicitly [Qde] measurable quality characteristics are: - Functionality (Qf) - Security (Qs) - Usability (Qu ) - Efficiency (Qe ) Slide 11 Dynamic Quality Characteristics (Qd) Give a rating for each characteristics on a scale: 0 -quality requirements not important 3 -quality requirements relatively unimportant but for purpose of testing to be considered 4 -quality requirements of normal importance 5 -quality requirements very important 6 -quality requirements extremely important Depending on the rating of the characteristics suitable degree of severity is introduced into testing: Functionality - weightage 0.75 Security - weightage 0.05 Usability - weightage 0.10 Efficiency - weightage 0.10

Slide12 Dynamic Quality Characteristics (Qd) Cont Based on the rating & weightage, Qde obtained by the following formula: Qde = ( 0.75Qf + 0.05Qs + 0.1Qu + 0.1Qe )/4 Slide 13 Dynamic Quality Characteristics (Qd) Cont Dynamic, implicitly [Qdi] measurable quality characteristics are: - User friendly - performance - maintainability

These are not standard and vary from project to project, assign the value 0.02 (for each characteristic) which ever are important Qd obtained by adding of Qde and Qdi values: Qd = Qde + Qdi Slide 14 Function Dependent Factors (Df) Various function dependent factors and the associated ratings are described with the following conditions: - One of the given rating must be selected - If insufficient information is available to enable rating of a given factor, nominal rating (bold) should be assigned 1. User Importance (Ue) with following weights: User-importance is assigned to the functionality identified by the user, this means assigning user-importance to the user function. 3 Low importance of function 6 Normal importance of function 12 High importance of function Slide 15 Function Dependent Factors (Df) Cont 2. Usage Intensity (Ui) with the following weights: The Usage-intensity has been defined as the frequency with which a certain function is processed by the user and the size of the user group that uses the function. 2 Low used a few times a week 4 Normal used a number of times a day 12 High used continuously throughout the day Slide 16 Function Dependent Factors (Df) Cont 3. Interfacing (I) with following weights: Interfacing is an expression of the extent to which a modification in a given function affects other parts of the system, this is determines by logical data sets (LDS). 2 Low degree of interfacing with the function 4 Normal degree of interfacing with the function

 8 High degree of interfacing with the function Slide 17 Function Dependent Factors (Df) Cont 4. Complexity (C) with the following weights: The Complexity of a function is determined on the basis of its algorithm. 3 Function has no more than 5 conditions 6 Function contains between 6 and 11 conditions 12 Function contains more than 11 conditions Slide 18 Function Dependent Factors (Df) Cont 5. Uniformity (U) with following weights: Under the following circumstances, only 60% of the test points assigned to the function under analysis count towards the system total: - In the case of a reusable or clone function: the test specifications can be reused for reusable or clone functions. - In the case of a dummy function (provided that reusable test specifications have already been drawn up for the dummy) 0.6 for cases where test specifications can be largely reused because of similar or clone functions 1.0 if function is unique in terms of processing or a new data set Slide 19 Function Dependent Factors (Df) Cont Formula for Df Calculation: Df = (( Ue + Ui + I + C ) / 20 ) * U Df = weighting factor for the function-dependent factor Ue = user-importance Ui = usage-intensity I = interfacing C = complexity U = uniformity

Slide 20 Function Dependent Factors (Df) Cont Standard Functions: For error reporting function, help screen function and/or menu structure function Standard number of test points can be assigned, and are shown below Function Error Message Help Screens Menus FPs 4 4 4 Ue 6 6 6 Ui 8 8 8 I 4 4 4 C 3 3 3 U 1 1 1 Df 1.05 1.05 1.05

Slide 21 Dynamic Test Points (TPf) The number of direct test points is obtained by the following formula TPf = FPf * Qd * Df TPf FPf Qd Df - Number of test points assigned to function - Number of function points assigned to function - Quality characteristics factor - Function dependent factors

Slide 22 Static Test Point (Qs) Indirect test point count depends mainly on the FP count for the system as a whole. This is also influenced by the requirements regarding the static quality characteristics to be tested (the Qs factor) One has to determine whether the static measurable quality characteristics are relevant for the test purpose. A Static test can be carried out using a checklist. Slide 23 Static Test Point (Qs) cont Method of Calculation of Qs factor: Quality characteristic is tested using the checklist available, the factor Qi get the value six.

 For each subsequent quality to be included in the static test, another six is added to the Qi factor rating. Qs = Qi where i = 16 Slide 24 Total # of Test Point (TP) The total number of test points assigned to the system as a whole is calculated by the following formula: TP = TPf + (FP * Qs) / 500 TP = total number of test points assigned to the system as a whole TPf = sum of the test points assigned to the each individual functions (dynamic test points) FP = total number of function points assigned to the system as a whole (minimum value 500) Qs = weighting factor for the indirectly measurable quality characteristics Slide 25 Primary Test Hours The primary test hour count is the number of hours required for carrying out the test activities involved in the test life cycle phases Preparation, Specification, Execution and Completion. Slide 26 Productivity factor The productivity factor indicates the number of test hours required per test point The productivity factor is a measure of experience, knowledge and skill sets of test team The productivity factor can vary from one organization to the next In practice the Productivity factor has shown to have a value between 0.7 to 2.0 Slide 27 Environmental factors Number of environmental variables & associated ratings are defined for calculation of the environmental factor One of the given rating must be selected

Slide 28 Environmental factors cont 1. Test Tools (Tt) Test tools variables reflects the extent to which testing is automated. Ratings are 1 Use of a SQL, record & play-back tools is used 2 Use of a SQL, but no record & play-back tools is used 4 No test tools are available Slide 29 Environmental factors cont 2. Development Testing (Dt) This reflects the quality of earlier testing, if the estimate under preparation is for an acceptance test, the earlier testing will be system testing and so on. Ratings are 2 Test plan is available and the test team is familiar with the actual test cases and test results. 4 Test plan is available but team is not familiar with this. 8 Test plan is not available. Slide 30 Environmental factors cont 3. Development Environment (De) This reflects the nature of the environment used to developed and released the system. Ratings are 2 Development in dot-Net, Java, 4-GL programming language with an integrated DBMS. 4 Development in dot-Net, Java, 4-GL programming language, and possibly combination of 3-GL programming language. 8 Development only in 3-GL programming language such as COBOL, PASCAL etc. Slide 31 Environmental factors cont 4. Test Basis (Tb)

 The test basis variables reflects the quality of the (system) documentation upon which the test under consideration and tested. Ratings are 3 During the system development documentation standards are being used and a template, in addition the inspections are organized. 6 During the system development documentation standards are being used and a template. 12 The system documentation was not developed using a specific standards and a template. Slide 32 Environmental factors cont 5. Test Environment (Te) The test environment variables reflects the extent to which test infrastructure used for testing. Ratings are 1 The environment has been used for testing several times in the past. 2 Test execution conducted on a newly equipped environment similar to other well-used environment within the organization. 4 Test execution conducted on a newly equipped environment which may be considered experimental within the organization. Slide 33 Environmental factors cont 6. Test ware (Tw) The test ware variables reflects the extent to which the tests can be conducted using existing test ware. Ratings are 1 A usable general initial data set (tables, etc.) and specified test cases are available for the test. 2 A usable general initial data set (tables, etc.) are available for the test. 4 No usable test ware available for the test. Slide 34 Environmental factors cont Calculation method for Environmental factor The environmental factor (E) is calculated by adding together the ratings for the various environmental variables, then dividing the sum by 21 (the sum of the nominal ratings).

E = (Tt+Dt+Tb+De+Te+Tw)/21 Slide 35 Primary test (PT) hours formula The number of primary test hours is obtained by the following formula: PT = TP * P * E PT = the total number of primary test hours TP = the total number of test points assigned to the system as a whole P = the productivity factor E = the environmental factor Slide 36 Management overhead factor 1. Team Size (Ts) This reflects the number of people are working in the team. Ratings are 3 The team consists of no more than four people. 6 The team consists of between five and ten people. 12 The team consists of more than ten people. Slide 37 Management overhead factor cont 2. Management tools (Mt) This reflects the extent to which automated tools are used for planning and controlling. Ratings are 2 Both an automated time registration system and an automated defect tracing system are available. 4 Either an automated time registration system or an automated defect tracing system are available. 8 No automated (management) system are available. Slide 38 Management overhead factor cont Calculation method for Management Overhead factor

 The planning and management percentage is obtained by adding together the ratings for the influential factors. The allowance in hours is calculated by multiplying the primary test (PT) hours count by this percentage MO = PT * (Ts + Mt ) / 100 Slide 39 Total# of Test Hours TH = PT + MO Slide 40 Distribution of the Test Hours Based on the experience of the TPA technique suggests that the following test hours breakdown between the phases:

10%
Preparation

40%
Specification

45%
Execution

P&C

Planning & Control

Overhead: an additional 5-10%

Slide 41 Summary of Total Test Points Calculation

Productivity factor P Based on historical data, 0.7 2.0

Environmental factor E = (Tt+Dt+Tb+De+Te+Tw)/21

Primary Test Hours PT = P * E * TP Total test points from previous step Management overhead factor MO = PT * (Ts + Mt ) / 100 Test basis 3, 6, or 12 Dev Testing 2, 4, Test or 8 1, 2, tools or 4

Total Test Hours

TH = PT + MO

Test team size, 3, 6, or 12 Total test hours Breakdown: 10% preparation (plan, design), 40% specification (test dev), 45% execution (run, report), 5% completion (final report)

Management tools, 2, 4, or 8

Slide 42 Use Case Point Analysis Test Estimation using Use Case Use Case Method is an effort estimation algorithm proposed by Gustav Karner in 1993. Alistair in 1999 proposed test effort estimation using Use Case. Slide 43 What are Use Case? Popular method of capturing and describing the functional requirements for a system based on actors and scenarios Is now part of UML (early 2000s) Many References and texts Start at www.uml.org for more info Slide 44 Use Case A use case model describes a system's intended functions and its environment. It has two parts:

 A diagram that provides an overview of actors and use cases, and their interactions. An actor represents a role that the user can play with regard to the system. A use case represents an interaction between an actor and the system. The use case descriptions detail the requirements by documenting the flow of events between the actors and the system. Slide 45 Use Case for Estimation Since use cases are available early in the development process, and since they describe the requirements of the system Easy to use them to drive the estimation. Slide 46 Use Case for Estimation Use Case Points Methodology Methodology introduced in 1993 by Karner Alistair in 1999 proposed test effort estimation using Use Case Requires all transactions be counted Straight forward and simple methodology Slide 47 UCP Estimation method in detail Each actor and use case is categorized according to complexity and assigned a weight. The complexity of a use case is measured in number of transactions. The unadjusted use case points are calculated by adding the weights for each actor and use case. The unadjusted use case points are adjusted based on the values of 9 technical & environmental factors for testing. Finally the adjusted use case points are multiplied with a productivity factor. Slide 48 Calculate Unadjusted Actor Weight (UWA)

 Count the Actors, classified them as simple, average, or complex, and then calculate the UAW Simple defined API Average interface with TCP/IP type protocol, HTTP, FTP, external data store Complex potential a person interacting with system thru GUI or Web page Weighting of 1,2,3 respectively Total UAW (unadjusted actor weight) determined by summing the products of the actors * their weights Slide 49 Calculate Unadjusted Actor Weight (UAW)

Actor Type Simple Average Complex

Description GUI

Multiplier # of Actors Total 1

Reasons

Interactive or Protocol-driver 2 Interface API Low Level Interactions 3

Total Unadjusted Actor Weights (UAW) Slide 50 Calculate Unadjusted Use Case Weight (UUCW) Categorize Use Cases as simple, average, complex: Simple: <4 transactions Average: 4-7 transactions Complex: > 7 transactions then count the # in the each category, and using the weighting factor of: Simple: 5 Average: 10 Complex: 15 Multiply the factor with the number in each category, and then sum to get the UUCW. Slide 51 Calculate Unadjusted Use Case Weight (UUCW)

Use Case Type Description Simple Average Complex 1-3 Transactions 4-7 Transactions > 7 Transactions

Multiplier # of Use Total (Given) Cases 5 10 15

Reasons

Total Unadjusted Use Case Weights (UUCW) Slide 52 Calculate Unadjusted Use Case Point (UUCP) The calculation of the unadjusted UCP is done by adding the unadjusted actor weight and the unadjusted use case weights: Total Unadjusted UCP, UUCP = UAW + UUCW Slide 53 Determine Adjusted Use Case Point (UCP) Adjust based upon technical factors and environmental factors - this is to account for factors not related to size of task TE factor = Sum (Individual technical & environmental factors * their weight) Slide 54 Technical Complexity & Environmental factor Factors Description T1 T2 T3 T4 T5 T6 T7 Weight Complexity Assigned Values (0-5) Total Reasons

Test Tools 5 Documented Inputs 5 Development 2 Environment Test Environment 3 Test-ware Reuse 3 Distributed System 4 Performance Objectives 2

T8 Security Features T9 Complex Interfacing TOTAL T&E FACTOR (TEF) TCF = 0.65 + (0.01 * TEF) Slide 55 Compute Adjusted Use Case Point

4 5

Compute adjusted UCP Unadjusted Use Case Point multiplied by Technical Complexity to obtained the Adjusted UCP (or simply say UCP). AUCP =UUCP * TCF Slide 56 Compute Efforts Arrive at final effort Effort in Person-Hours = AUCP * Productivity Factor E.g. Effort = AUCP * 20 Where 20 man-hours are required to plan, write and execute tests on one UCP when using EJB (industry standard). Slide 57 Use Case Point Summary The method is simple, quick, and transparent to use A tuned, calibrated UCP method works extremely well. The calibration appears to be relatively easy and quick. Need to standardize size and level of Use Case within an organization The complexity classification rules for Use Case may need to be adjusted For large projects, an additional overhead factor may need to be added to handle the diseconomies of scale. (Or incorporate it into the productivity factor decreasing it by 50% or 100%). There are commercially available tools, such as Duvessa (http://www.duvessa.com/), which support these methods, and give additional flexibility with local sizing parameters. Slide 58 Case Study / Examples where TCF = [0.65+(0.01*TEF)]

Home Security Use Case Diagram

System Configuration Alarm Controller Home Security Central Processor
Light Activator/Deactivator Controller

Wireless Dial-out Device Controller

Key Device Controller

Slide 59 Determine Unadjusted Actor Weight (UWA) The Actors are: Human, Alarm Controller, Callout Controller, Key Device Controller, Light Controller Actor Human Alarm Controller Callout Controller Light Controller Actor # of Multiplier Type Actors Complex 3 1 Simple Simple Simple 1 1 1 1 1 1 1 1 Totals Reasons 3 1 1 1 1 7

Key Device Controller Simple

Total Unadjusted Actor Weights (UAW)

Slide 60 Determine Unadjusted Use Case Weight (UUCW) The Use Cases are: Initialize/Reconfigure System and System Configurations Alarm: Receive Alarm, Recognize it, Callout and Flash Lights Activator/Deactivator System

Use Case Description Complexity

Multiplier

# of Use Total Reasons Cases

Simple Average

System Configuration 5 1 Alarm Callout 10 1 Activator/Deactivator Simple 15 1 System Total Unadjusted Use Case Weights (UUCW) Calculate total unadjusted UCP UUCP = 7 + 20 = 27 Slide 61 Determine Technical Complexity & Environmental factor Factors Description T1 T2 Test Tools Documented Inputs Development T3 Environment T4 Test Environment T5 Test-ware Reuse T6 Distributed System Performance T7 Objectives T8 Security Features T9 Complex Interfacing TOTAL T&E FACTOR (TEF) TCF = 0.65 + (0.01 * 40) TCF = 1.05 Slide 62 Determine Adjusted Use Case Point (AUCP) Calculate total adjusted UCP AUCP = UUCP * TCF AUCP = 27 * 1.05 AUCP = 28.35 Slide 63 Determine Effort Use 20 Hours per UCP [ Effort = AUCP * Productivity Hours ] Effort = 28.35 * 20 = 567 Hours Complexity Weight Assigned Values (0-5) 5 1 5 1 2 3 3 4 2 4 5 1 1 1 1 0 2 2

5 10 5 20

Total 5 5 2 3 3 4 0 8 10 40

Reasons

 Assuming 40 hours per week = 567 / 40 = 14.18 person week Person Days = 567 / 8 = 70.88 person days Slide 64 References [1] Test point analysis: a method for test estimation - Drs. Erik P.W.M. van Veenendall CISA and Ton Dekkers [2] Test Effort Estimation Using Use Case Points - Suresh Nageswaran