You are on page 1of 5

World Congress on Software Engineering

A Software Project Risk Analysis Model Based on Evidential Reasoning Approach
Nan Feng, Minqiang Li, Hong Gao School of Management, Tianjin University, Tianjin, China E-mail: Abstract
In this paper, a software project risk analysis model based on evidential reasoning approach is presented. The modeling process consists of four phases: specification of the model structure, assessment of evidence strength, determination of software project risks, and risk monitoring and analysis. Using the changes of strength of evidences obtained from the process of software project development, the model can continually estimate the state of risk, and identify the sources of risk. The model provides a rigorous, structured manner to incorporate relevant software project risk factors and their interrelationships when estimating software project risks. the impact of each risk or avoid it completely. The final step is risk monitoring, which ensures that the riskreducing methods are implemented effectively and also determines whether or not the risk-reducing tactics are in fact reducing risks [5]. The crucial step in any risk management process is the analysis of identified risks in order to determine the degree of attention each risk need be paid, since this will ultimately affect the success of a project. In this paper, we develop a software project risk analysis model based on evidential reasoning approach. And the paper elaborates the theoretical concepts and provides operational guidance for implementing the model. In the next section, we discuss the main limitations with existing approaches for software project risk analysis. After that, the process of developing a software project risk analysis model is presented. Finally, we demonstrate and validate the model via a case study.

1. Introduction
There exist many uncertainties in software development processes and products; for instance, the uncertainties in estimating project size, schedule, quality, and in determining resource allocation. Current software engineering techniques cannot eliminate such uncertainties. Thus, risk management is critical [1]. According to reference [2], studies indicate that about 85% of all projects end in failure. Furthermore, it is estimated that 31.1% of projects will be cancelled before they are ever completed [3]. Software development, given its diverse and abstract nature, offers unique challenges and risks. A formal risk management program takes a structured way to evaluate risks in the software development process. A typical risk management framework involves identifying and analyzing the risks to a project and then implementing and monitoring measures to reduce them [4]. The first step in a typical risk management program is the identification of risk, usually involving checklists, questionnaires or brainstorming sessions. The next step is the analysis of the risks identified in such a way that they can be ranked in a meaningful manner. This is followed by the development of contingency measures to decrease

2. Background Research
Software project risk analysis has been given serious consideration by both academics and practitioners for quite some time. Because of space limitations, we do not present a complete review of software risk analysis literature. Instead, we discuss the major limitations of the existing approaches and how our approach mitigates some of these limitations. Reference [6] and [7] presented comprehensive reviews of this literature. Before we discuss the weaknesses of the existing approaches, we discuss the risk exposure estimating. As risk implies a potential loss, there are two elements at issue here: firstly the probability of an unsatisfactory outcome and secondly the consequences of such an outcome [8]. These two measurements are estimated using risk analysis techniques, the multiplicity of which forms the risk exposure metric that is defined by the formula below: Risk Exposure = Prob (UO) × Loss (UO) (1) Prob (UO) is the probability of an unsatisfactory outcome. Loss (UO) is the loss to the parties involved if

978-0-7695-3570-8/09 $25.00 © 2009 IEEE DOI 10.1109/WCSE.2009.228

Authorized licensed use limited to: RMK Engineering College. Downloaded on August 03,2010 at 03:16:06 UTC from IEEE Xplore. Restrictions apply.

Restrictions apply. Unfortunately. The Dempster–Shafer theory of belief functions is a generalization of Bayesian theory of subjective probability [10]. That is. Second. Mathematically. such as software project risk. that is. the estimation can be obtained through discussions among relevant parties. For example.2010 at 03:16:06 UTC from IEEE Xplore. (3) 3. there are mainly three limitations with existing approaches.e. a2. probability associated with the answers to a related question. a2}). existing methods only provide general suggestions. And the probabilistic estimation of risks is the key element of risk analysis. all these m-values add to one: A⊆Θ ∑ m( A) = 1. the answer to the question of interest is represented in terms of probability. under Bayesian theory. under Dempster–Shafer theory that are relevant to the formulations developed in this paper.. Evidential Reasoning Approach The evidential reasoning approach under the Dempster–Shafer theory of belief functions has been widely used in a broad range of disciplines. the level of uncertainty concerning whether the outcome will be positive or negative). the plausibility in a statement is the degree to which the statement is plausible in light of the evidence—the degree to which we do not disbelieve the statement. we associate objective or subjective probabilities to the possible answers to the question of interest. belief in a statement represents the total belief that the statement is true. and plausibilities. In the belief function formalism. These uncertainties are called m-values. an}. The proposed approach consists of both the quantitative computation of risks and the graphical representation of relevant constructs through an evidential diagram. the belief function for a subset of elements A of a frame Θ.the outcome is unsatisfactory. such discussions are quite limited for guiding users to properly estimate software project risk. This representation of risk covers only one subcomponent of software project risk and leaves out another important subcomponent (i. (2) where A represents all the subsets of the frame Θ. Whenever the changes of strength of evidences are obtained. we provide the mathematical definitions of three important functions. . . Mathematically. is defined as the sum of all the m-values for the individual elements in the subset A. mvalues. The m-value pertaining to a statement measures the degree of belief directly assigned to the statement based on the evidence. Intuitively. the degree of belief that can be associated with the possible answers to a question of interest may depend on the where B is any subset of A. . This study utilizes evidential reasoning approach to provide objective support for the estimation of risks. (5) 225 Authorized licensed use limited to: RMK Engineering College. a2} is Bel({a1. The current form of belief function theory is based on the work of Dempster and Shafer. in software project. This study defines software project risk as the plausibility of a negative outcome under the Dempster–Shafer theory of belief functions[9]. belief in the subset {a1. or emphasize the quantitative computation of risk probabilities.. First. the m-value for the empty set is 0. the existing methods use the probability of a negative outcome to represent risk. (4) The plausibility function can also be defined in terms of belief function as Pl ( A) = 1 − Bel ( A). Suppose we have a decision problem with n possible elements or states of nature forming a mutually exclusive and collectively exhaustive set represented by {a1. Under Bayesian theory. regarding how to estimate the probability of a risk’s occurrence. inform the analyst about a variable of interest. Basically. For the probabilistic estimation of risks. and m( ∅ ) = 0. the plausibility function for a subset of elements A of a frame Θ. beliefs. a2}) = m(a1) + m(a2) + m({a1. uncertainty is not only assigned to the single elements of the frame but also to all other proper subsets of the frame and to the entire frame Θ. when combined together. In terms of symbols: Bel ( A) = A⊇ B ∑ m( B). is defined to be the maximum possible belief that could be assigned to A if all future evidence were in support of A. . The notion of risk using plausibility is a more conservative measure of risk than the probability measure of a negative outcome. Under Dempster–Shafer theory. but not both. Third. For example. An evidential reasoning approach to risk analysis is a process where several variables (assertions). a3. Similar to probabilities. Downloaded on August 03. the existing methods either focus on the graphical relationships among software project risk factors using flow charts or diagrams. plausibility is defined as Pl ( A) = A∩ B≠∅ ∑ m(B). they are plugged into the model to update estimates. Call this entire set a frame represented by the symbol Θ. In the following paragraphs. the basic belief assignment function. and the m-values for any subsets contained in the subset A. In terms of symbols.

numbers in parentheses represent m-values or belief inputs provided by the software project risk assessor.Assuming A is an assertion that software project risk is low. the rounded boxes represent assertion nodes. (6) where K = 1 − ∑ {m1 ( B1 )m2 ( B2 ) B1 ∩ B2 = ∅} . Pl ( A) . Restrictions apply.2010 at 03:16:06 UTC from IEEE Xplore.” And evidence represents the information that supports or negates assertions. The first m-value represents the level of support the evidence provides to the assertion it pertains to. then the combined m-values (basic belief assignment function) for a subset A of frame Θ using Dempster’s rule is given by m( A) = K −1 ∑ {m1 ( B1 )m2 ( B2 ) B1 ∩ B2 = A. users assess strength of evidence.978.011) E1. Specification of Model Structure An evidential diagram consists of assertions. (0.2. which indicates the level of support that a piece of evidence provides in favor of or against the assertion it pertains to. Although costly. A1. For instance.3. Evidence nodes are connected to the corresponding assertion(s) that they directly pertain to. Downloaded on August 03.” directly pertains to assertion “A1. Model procedure 4.8. and between higher-level subassertions and lower-level subassertions) need to be defined using logical relationships such as “and” and “or. Assertions include the main assertion and subassertions. The software project risk assessor can use existing qualitative methodologies. methods such as Delphi techniques can be used to refine the completeness and accuracy of an evidential diagram. Hypothetical evidential diagram for the 2 Assessment of evidence strength assertion “Specification quality is good” 4. the plausibility of A . 4. In Figure 2. The complexity of software is low. Software size is small.1 Software size is small. represents maximum possible risk that software project risk is high. beliefs on assertions are computed by combining the strength of all the items of evidence (mvalues). . Group meetings or Delphi techniques can be used to help achieve consensus among experts.1. 4. and risk monitoring and analysis.9. A ≠ ∅} . Determination of Software Project Risks In this step. 226 Authorized licensed use limited to: RMK Engineering College. evidence strength can be elicited from multiple experts. and questionnaires to develop the evidential diagram. assessment of evidence strength. Specification quality is good” and thus it is connected to that assertion. which represents the renormalization constant. the procedure of the model is given in Figure 1 (Risk treatment is not involved in the model discussed in this paper).2. The second term in K represents the conflict. based upon the model’s structure. And evidence nodes are represented by rectangular boxes in the evidential diagrams. such as scenarios [11]. Specification quality is good. And. Relationships between assertions (e. To avoid being heavily affected by individual subjective judgment.. The main assertion is the highest-level assertion. 0. (0. determination of software project risks. between the main assertion and subassertions.1. the evidence In this step. and their interrelationships. 0) E1. In Figure 2.1) Figure 2. If m1 and m2 are the m-values representing two independent items of evidence pertaining to a frame Θ.g. (0. evidence. Strength of evidence is represented by m-values. in Figure 1. Evidential Reasoning Model for Software Project Risk Analysis The process of developing an evidential reasoning model consists of four phases: specification of the model structure. 1 Specification of model structure “E1. 0. Assessment of Evidence Strength New Evidence 3 Determination of software project risks 4 Risk Monitoring and Analysis Risk Database Risk Treatment Figure 1. and the second m-value represents the level of support for the negation of the assertion it pertains to. the subassertions are lower-level assertions.

880. 0) A1. A Case study The above model was verified by a financial management system developed by UFIDA Software Company. with 0. i. In the process of risk monitoring.1)(0. (0.1)(0. After the risk treatment. (0. Schedule pressure is small. new evidences are plugged into the model to update previous estimates.1. A1. For risk monitoring. m1 ( a )=0. a})m2 (a )]/ K = [(0. 5.978.8. We first cross multiply the two sets of m-values and collect all the resulting m-values and assign them to the common element of the cross product.1 Evidence 2: m2 (a)=0. 0) 4. Risk monitoring and analysis “Defects rate is low” 227 Authorized licensed use limited to: RMK Engineering College.(0. Suppose that we obtain the following beliefs in terms of m-values from the two items of evidence: Evidence 1: m1 (a)=0.e. Downloaded on August 03.1.910. Productivity is high.1) + (0.8)] = 0.820. 0) No A1.1. a })=0.0)(0. The complexity of software is low.2.1) + (0. Requirements change is not frequent. m2 ( a )=0. This suggests that the risk involved in specification quality is 0.8.6. a }) = [m1 ({a. a}) + m1({a.011. let us consider a hypothetical example where we have two independent items of evidence pertaining to a binary variable A with values a that the variable is true and a that it is not true.As figure 2.022—that is.2. a })=0. a continuously monitoring loop starts.1. 0) E1.1. Risk Monitoring and Analysis 2 Assessment of evidence strength E1.9. “A1.011 So. m1 ({a. (0. Software size is small.7. Expertise level is high. m2 ({a.91 = 0.2.1)] / 0. AVG (Bel(Ai))N < threshold (Ai).1)(0.91 m(a) = [m1 (a)m2 (a) + m1 (a)m2 ({a.. the belief negating the assertion is 0. Defects rate is low.1. we renormalize these m-values by dividing by renormalization constant. the risk involved in Ai should be treated with.Defects rate is low” was the assertion observed. Next. 0) E1.022.1.8) + (0. the observed Ai can be traced to its descendant interactively with the user to identify the sources of the risk. a}) + m1 ({a. they are plugged into the model to update estimates.91 = 0. In this case.2. Efforts estimation is accurate. A chronological record of such inputs and estimates are kept as a risk profile saved in the software project risk database. 0) E1. 0) Figure 4.926. (0. And then the risk sources will be traced. the plausibility that specification quality is not good is 0.0)(0. if the average belief of the previous N time units of the profile records for Ai is under the threshold for it.9)(0. a })]/ K = [(0. The cross multiplication and renormalization yield the following m-values and K: K = 1 − [m1 (a)m 2 (a ) + m1 (a )m 2 (a)] = 1 − [(0. The procedure of this phase is given in Figure 3. (0. (0. (0.91 = 0.2.1. the belief supporting the assertion that “Specification quality is good” is 0. Evidential diagram for the assertion Figure 3.1.1) + (0. Ai represents an assertion in the evidential diagram.9. 0) 3 Determination of software project risks Risk Database AVG (Bel(Ai))N< threshold (Ai) Yes Tracing the Sources of A New Evidence Risk Treatment E1. Specification quality is good.9)(0.1. (0.1) + (0.2010 at 03:16:06 UTC from IEEE Xplore. Restrictions apply.9)(0. Whenever the changes of strength of evidences are obtained.4.978 m(a ) = [m1(a )m2 (a ) + m1(a )m2 ({a. Once the software project starts.8)] / 0. a })m2 ({a.1. 0) & A1. a})m2 (a)] / K = [(0. .1.011 m({a. Since an evidential diagram can visually model cause consequence relations.0)(0.1.011 remaining as ambiguity. the threshold of A1 was set as 75% by expert experience.1)] / 0.9.

pp. field research is needed to empirically compare our approach with other methods.Based upon the process suggested previously. Shafer. After a period of time. China. 193-203 [2] G. and H. 22. And then. the model kept monitoring the assertion A1. “Perspectives on the theory and practice of belief functions”. “Risk Management for Software Projects”. Journal of Management Information Systems. pp. and T. the research has its limitations. Control and Decision. Conclusions This paper proposed a software project risk analysis model based on evidential reasoning approach. .820. R. Thus. Huang. References [1] C. It consists of one main assertion. Zarley. The model provides a rigorous. Requirements change is not frequent”. pp.3).” This means that project manager has 82 percent confidence that the overall assertion “Defects rate is low” is true and has zero belief that the assertion is not true since no negative evidence was obtained with respect to this assertion. 80. Padayachee. 109-142 [10] G. With the changes of strength of evidences. which is the plausibility that defects rate is high. 195-202 [3] B. pp. S. Han. the overall belief that the “Defects rate is low” is calculated to be “0. Hsia. We then presented the process of developing a software project risk analysis model. Y. Proc. high strength = [0. New York. Restrictions apply. Elsevier Inc. P. very high strength = [0. New York. 1990. “Evidential reasoning using DELIEF”. Belief inputs (m-values) on strength of evidence were elicited from project experts.1).9—1. Elsevier Inc. 2002 Annual Research Conf. but readily adaptable model of information system risk”. pp. Proc.. 205-209 6. 57-67 [9] L. New York. Based upon our evidential reasoning model with the “and” relationship and strength of evidence assessments provided by expert experience. T. “Project termination doesn’t equal project failure”. Mock. Mykytyn. Elsevier Inc. the modeling approach should be tested in other practice situations in addition to the financial management system discussed in this paper. Jiang. 0. S. 94-96 [4] K. the South African Institute of Computer Scientists and Information Technologists. Sherer. Communications of the Association for Information Systems. “A general. 11. September 2000. Pan. USA.2. 33. we are not able to provide much empirical evidence of the advantages of this approach over others. vol. an evidential diagram (see Figure 4) for the “Defects rate is low” assertion was developed. Elsevier Inc. 4.7). pp. Spring 2006. C. pp. Based on the evidential diagram. “An empirical analysis of risk components and performance on software projects”. vol. May 2007. P. New York. Journal of Systems and Software.. pp. MCB University Press. March 2001. Mykytyn.7—0. structured manner to incorporate relevant software project risk factors. Y. “A framework for integrated risk management in information technology”. pp. clearly. median strength = [0. IEEE Computer Society. Elsevier Inc. Klein. Journal of Systems and Software. and K. 37. [8] R. “An interpretive study of software risk management perspectives”. the project manager traced the potential source of this problem to be the evidence “E1. W. pp. UK. January 2004. vol. DELIEF software [12] was used to aggregate m-values form all the items of evidence and to compute the risk of defects rate. IEEE Computer. 1–28.1. vol. 1999. 22. vol.P. Yu. vol. J. 481-486+493 [7] S. 2002. AAAI Press. Association for Information Systems. 7th national conference of artificial intelligence. vol. Wang. F.. Fairley. Chen.. two second-level assertions (subassertions). risk treatment was performed to reduce the defects rate. Boehm. 228 Authorized licensed use limited to: RMK Engineering College. Of course. Management Decision. South Africa. Finally. G. 323-362 [11] W.2010 at 03:16:06 UTC from IEEE Xplore. Fan. M. it was found that AVG (Bel(A1))N < threshold (A1). W. Bandyopadhyay. vol.0]. Alter. L. pp. First. Downloaded on August 03. Journal of Systems and Software. Sun. Shafer.1—0. 118-127 [5] K. J. MN. low strength = [0. 56. 1988. International Journal of Approximate Reasoning. IEEE Software. 2004. and G. January 2007. IEEE Computer Society. Second. “Overview of the study on theories and methods of software project risk management”. J. New York. Atlanta. 14. and their interrelationships when estimating software project risks. The main limitations with existing approaches for software project risk analysis were first discussed. 437-444 [6] C. Srivastava. “BBN-based software project risk management”. May 1994. 42-50 [12] D. vol. This case shows that it is possible to disclose potential software project risks using the evidential reasoning model. “An information systems security risk assessment model under the Dempster-Shafer theory of belief functions”. Northeastern University. The following fuzzy metric was also provided to facilitate the input of the strength of evidence: very low strength = [0—0.1. pp.9). vol. “Seeking consonance in information systems”.3—0.. 73. Y. we demonstrated and validated the model via a case study. J.