This action might not be possible to undo. Are you sure you want to continue?
TOM GILB Independent Consultant Email: Tom@Gilb.com
Copyright © 2012 by Author Name. Published and used by INCOSE with permission.
Abstract. Accurate estimation is impossible for complex technical projects, but keeping to agreed budgets, and deadlines is achievable by using feedback and change. In other words, rather than trying to improve the initial project estimates, the budgets and deadlines must be set, based on the value of delivery (not the cost), and then iterative re-engineering of product and process must be used to stay within acceptable resource bounds. Or at least iteration must be used to get most of the expected value delivered, within the acceptable budgets and deadlines. This paper explains the background to this approach and discusses its use providing several examples.
Accurate estimation, of time and costs, for complex systems and software projects is seemingly impossible to guarantee (Collins and Bicknell 1998; Jørgensen 2005; Craig and Brooks 2006; Grimstad et al. 2006). There are many unavoidable reasons for this. Even when estimation seems to work, this might just be a case of ‘stopping the effort’ when the budget runs out. However that method is likely to result in delivering systems of unacceptably low quality. The main idea of this paper is that there is a constructive alternative to such unsatisfactory estimation processes: -‐ You use ‘process control’ (do a little, learn quickly and change quickly) to rapidly and continuously tune anything and everything about your project, so that prioritized resource budgets (like time to market, money and human resources) can be met -‐ You consciously sacrifice ‘less-holy’ things for your ‘more-holy’ objectives. We are better off stipulating reasonable resource constraints (deadlines and cost budgets) and then learning to live within them. This is acceptable as long as we have our highest performance and quality priorities already satisfied, when resources run out. The rest of the requirements are, by definition, marginal. There are several reasons why we cannot estimate project costs accurately, it is because: -‐ We do not define requirements (particularly multiple quality requirements) well enough, to estimate their costs, with useful accuracy -‐ We do not specify our designs (also known as strategies or architecture), which are powerful enough to satisfy our hopefully clear-and-complete requirements, well enough to know the design cost consequences -‐ We do not collect, or have access to, experience data that would allow us to estimate costs, for our well-specified designs, even if we did have clear requirements -‐ Even if we did have the experience data, for the design´s costs of meeting the clear requirements, it would probably not help much; because our new project (critical cost-driving variables) would be different (from past experience) in some decisive way.
with regard to cost and risk. make adjustments. early. DRIVERS: If you have not specified all critical performance and quality levels numerically – you cannot estimate project resources for those vague requirements. 2. However. Make an outsourcing contract. or changes mid-project – you cannot expect to estimate the costs for so many human variables. Then stipulate the value to you of reaching each major (top 10) requirement. intended to prevent overruns. only when that requirement is successfully delivered. 7. about the resources needed for your technical solutions. learn well. 5. designs or constraints – then the estimate might need to be changed radically. prevent overruns or assure value. APPLY NOW: Learn early. at a reasonable cost. LEARN SMALL: Carry out projects in small increments of delivering requirements – so you can measure results and costs. learn often. STAFF: If a complex and large professional project staff is an unknown set of people. Summary of the principles of resource control THE SHORT VERSION: Some constructive suggestions to people who think they want estimates Stipulate one or more useful deadlines from your point of view: -‐ Be specific about what has to be done by each deadline -‐ Ask if these deadlines seem reasonable for the tasks prioritized -‐ If necessary. 9. 4. and change anything about the next increments that you believe might get you back on track. and pay part of that value. ARCHITECTURE: If you implement your project solutions all at once. And – you probably will not have the information necessary to do it. This paper describes an alternative way to achieve these management needs. to any of the requirements. PRIORITIZE CRITICAL: You will have to prioritize your most critical requirements and constraints: there is no guarantee you can achieve them all. against (short term) estimates. we do not really need time and cost estimates! Overall long-term estimates are an old custom. They also need to be sure that the project will be profitable. EXPERIENCE: If you do not have experience data. Deliver ‘high-value for resourcesused’ first. then you cannot estimate the project resources. nor the insight that you need to do it. SENSITIVITY: If even the slightest change is made. RISK FAST: You should probably implement the design ideas with the highest value. and apply the learning to your current project. and to give management some feeling that the job will get done in time. LEARN ROOT: If incremental costs for a given requirement level (and its designs) deviate negatively from estimates – analyze the root cause. 8.In fact. The Control Principles 6. estimates do not. Summary of the Principles of Resource Control The Risk Principles 1. and will not embarrass them with unexpected losses. 10. after an ‘accurate’ estimation. Ensure you only do business with suppliers who consistently deliver value for . 3. What management needs is delivery of some critical system requirements in a predictable timeframe. Figure 1. in fact. without learning their costs and interactions incrementally – you cannot expect to be able to understand the results of many interactions.
that there are a large number (dozens) of software cost drivers. big enough to put their company out of business. In particular the cost-driving designs relate directly to the required levels of performance and quality requirements. One organization (Personal Communication.000 to 3. 170) assuming this is correct and realistic – even roughly so – we would need the fourth digit (xx. 2000). It has long been understood.999% availability.money. Costs are not quite so sensitive to functions or size – although there is a linear relationship there too.x8%) to signal that our project might cost 24. COCOMO for example does not try to operate at this level of precision.999% without ever having achieved such a result in the entire history of the Corporation. the 5ESS project (Lucent 1980) had a major objective of improving the availability of a previous generation system from 99. and some convincing arguments for yourself or others.08% in the level of one single quality. accurate. For those of you who like short papers. that meant a change of 00.90% to 99. management needed to make sure that they knew what they were contracting for. reliable. However that represents 80% of the way to perfection. then you can read on. Now Figure 2.x0 to xx. or true 1. They contractually promised 99. for example COCOMO (Boehm et al. DRIVERS: If you have not specified all critical performance and quality levels numerically – you cannot estimate project resources for those vague requirements Costs are a result of the designs you deploy.98% (The 5ESS actually replaced the 1AESS. for 100 developers. using 2. more detail. Part 1: The Risk Principles – Why you really cannot expect estimates to be correct. I’m done! Use the summary in Figure 1. In other words. fixed price.000 work-years (Personal Communication from David Long 2010). However some of you might like more explanation. such as availability. and without consulting their Technical Director. no pay’ (Gilb 2006) is one way to motivate suppliers to give you value for money – otherwise their motivation is to keep billing you forever (Craig and Brooks 2006). The high cost of perfection (Gilb 1988. more references. in order to meet your requirements. . Perfection costs infinity. ‘No cure. Don’t waste your money on suppliers who make excuses. The project took 8 years. The CEO acknowledged that they had taken an unwarranted risk. a Client) contracted for one year of effort. which was AT&T's first electronic (program controlled) switching system (Personal Communication from David Long 2010)). and promised to deliver airplane telephone switching software with 99. In one case. instead of value. This was of course an organizational problem.000 people. If so.
for example. it takes only one single other variable.2. and you do not have relevant historical data about the cost factor relationships! Does this begin to feel hopeless? Large-scale estimation is hopeless! That is what you need to appreciate. 3. as a function of investment in requirements. NASA Software project overrun. then you cannot estimate the project resources So what does it cost to develop software with 99.999% availability? There are few documented instances (for example. nobody can know. Source: Gruehl (NASA) in (Hooks 1994) Most of you do not have such data. We could take steps to change unexpected and bad costs or qualities. and perhaps no instances are relevant to our current system. or of other single factors. or one process. 40 years! Nobody knows. Once you are there. to change the real costs by at least a factor of 2 or 3. This is only one cost driver variable among many. one at a time in small increments. and nobody can be certain. before it was too late to change. Most of you don’t even know about the many factors like this that can influence costs. But with no valid and useful historical experience data. What we can do is to avoid ‘promising’ specific costs. such as ´our new investment in the requirements process´. If we implemented the critical variables. Figure 3. ARCHITECTURE: If you implement your project solutions all at once. One of the problems is that by the time you get the experience data. and what they will become. then we could see the effects more clearly. without learning their costs and interactions incrementally – you cannot expect to be able to understand the results of many interactions You all are well aware that Waterfall or ‘Big Bang’ projects have a strong tendency to fail (MacCormack et al. We might even have a real problem of convincingly proving that the system we have developed is actually at that level. on the requirements or the costs. We could better argue . See the chart below for an example of that. It is therefore difficult to see the effects of any one design. and will remain at that level for. we can turn to solving the problem of controlling large-scale costs in a serious way. But why do they fail? One of the many causes of failures is that waterfall method projects are committed to doing too many things at once. such as the design or development process techniques. Even if we knew a sample of costs for the required availability levels. it is likely out of date with respect to the technology you will deploy yourself. Lucent 1980). when we really do not know what they are. 2001). EXPERIENCE: If you do not have experience data about the resources needed for your technical solutions.
More below. does this very well. unfortunately. that multiple causes combine to give multiple effects. An example of a learning and change process (MacCormack et al. if you want to begin to understand what is going on. which help them achieve competitive advantage (Johansen and Gilb 2005). One cause. or 95% of the way to the target level (Johansen and Gilb 2005) . Notice the definitions for the requirements and costs.keep all else constant. Single cause versus single effect.2] for Evo Step 9. for a varied set of quality measures. 2001) – and of doing too much too early without feedback about cost and quality relationships the need for the changes with our management. They measure the effect of a design on a target quality requirement. It requires some discipline (Gilb 2010b) to break things up into these small ‘experiments’. The Planguage keyed icon “<->" means “from baseline to target value”. Even though the unsimplified reality is. for facts) cumulative quantified quality leaps. “Recoding" of the ‘Market Information Recoding’ design.Figure 4. But this step-by-step approach is the price we have to pay to get some real control over costs and qualities. and decide if the design is meeting its expectations. One of our clients. Confirmit. is a fundamental scientific analysis principle. one effect . Step 9 alone moved the Productivity value to 27 minutes. Their results are astounding (see ref. They do this in a weekly cycle. say ‘Intuitiveness’. and consequently manage to build predictably realistic measurable integrated complex systems. Table 1. Here is a simplified version of the Impact Estimation (IE) table [3.
designs or constraints – then the estimate might need to be changed radically. The real costs might be surprisingly higher costs initially. From Honeywell case (Berntsen 2007) where an offshore swap-out of qualified developers was caught by sensing short-term changes in the quality of work done . legal constraints (national sensitivity to personal data. Costs can be sensitive to other drivers than performance requirements. and 00. And we cannot expect that past team performance is a correct guide to future performance. They can be sensitive to resource constraints – people. So. and finding the root cause for worse performance (Berntsen 2007). and they do not seem to know how to be clear on the critical few Figure 5. policy constraints (‘do no evil’) and a large number of factors.08% more than required. But we normally cannot be sure who will staff our projects. SENSITIVITY: If even the slightest change is made. Honeywell discovered that top offshore staff had been swapped with less-competent staff. They realized this only by monitoring short-term numeric feedback.4.08% less than required might cost a bundle later. time. The unpleasant fact is that even the best of organizations are embarrassingly bad at clear and complete specification of requirements (Personal measured experience). or even if planned staffing can or will be carried out. due to unforeseen factors later – the true long-term costs. This gets even worse with offshore and outsourced projects. or they may come as a ‘big surprise’. For example. money (for development and for maintenance). The point being made then was that system qualities are major cost drivers. compared to expectations. and before payments is acceptable. unknowns and unpredictable variations in people. If these cost driving factors are not clearly specified. 00. nor even the insight that you need to do it You already have the example above (Principle 1) where a 00. 5. STAFF: If a complex and large professional project staff is an unknown set of people. as a result. you would still have significant problems using that information to understand their productivity.08% increase in availability for the software of the AT&T 5ESS project cost 8 years and took the efforts of thousands of developers. and are not really followed up in practice. They do not try very hard to be complete. then the real costs might vary widely. are yet another reason why it is difficult to make project cost predictions. after an ‘accurate’ estimation. might cost a bundle now. And – you probably will not have the information necessary to do it. for example). or to slight numeric changes in actually delivered quality. and quality requirements. and consequent time and money costs. But there is an additional point that these cost drivers can be very sensitive to apparently small numeric changes in requirements. to any of the requirements. or if staff changes mid-project – you cannot expect to estimate the costs for so many human variables Even if you had a track record for a defined and constant team.
-‐ We could use the estimates as a framework budget. then they have given management fair warning. Will provide a richer set of functionality for supporting next-generation logging tools and applications 7. The complete lack of measurable precision in these primary project objectives is. They ought to give fair warning. their budget -‐ They can decide. Real case study example (Gilb 2008a) of the primary objectives for a project that took 10 years before delivery of any of these objectives. and cost over $160. are. the risk principles. that is. and even on vastly more clear and complete requirements practices. and real designs. A primary goal is to provide a much more productive systems development environment than was previously the case 6. Managers should always regard any such estimates as highly suspicious. Figure 6. If the staff did this. Central to the Corporation’s business strategy is to be the world’s premier integrated <domain> service provider 2. testably. In fact. There are too many unknown and unknowable factors that can significantly affect the results. if the staff providing the estimates are themselves not explicitly aware of this fact. how much they can spend on the project. and the real requirements. are doomed to failure. splice. discussed above. and learn from experience what the real costs. Will provide a much more efficient user experience 3. like this for example: CAVEAT: The estimates cannot be trusted. based on the projected value of the system. Gilb 2008a). depth correct. are one way of saying that any attempt to estimate costs and timing. Notice that all objectives refer to qualities (of an existing system). recompute and/or do whatever else is needed to generate the desired products 4. and measurably . let alone on details that might be significant enough to affect costs significantly (Gilb 2005. -‐ But we would have to evolve the system in small steps. based on the market situation.000. -‐ The only way to be reasonably sure of the (money) costs and the (effort) deadlines would be to apply redesign-to-cost adjustments as the project progresses (Mills 1980). Robustness is an essential system requirement (see rewrite in example below) 8. according to my decades long worldwide experience. to deliver the value. when they want the stakeholder value to be delivered. Dramatically scale back the time frequently needed after the last data is acquired to time align. Most managers allow such things to happen on most projects. that is the deadlines -‐ They can then design the project organization. They lose control of costs immediately when primary critical project objectives are unclear. and also a prescription for ‘success’. even regarding their order of magnitude. Primary Objectives for a Project 1. merge. their competence is dubious.000. Major improvements in data quality over current practices. They have also understood the control principles in the second half of this paper! The estimation reality then becomes: -‐ Management can decide. Make the system much easier to understand and use than has been the case for the previous system 5. See the reference slides (Gilb 2008a) for examples on how to write these more clearly. based on current requirements practices. in my view. when they want it. To summarise so far. the primary reason for the time delay and costs.top-level requirements.
and deliver something. You cannot hide your ignorance from yourself any longer 2. You might have to do something not taught at school. your team and your own management). or an impossible project: and they should take that as a warning to stop or change. So you are destined to see the true costs of delivering value – not just the code costs 5.at a cost they find affordable -‐ If even the simplest and smallest (weeks or months) attempts to deliver value within satisfactory time and cost fails. You cannot continue to hide your lack of ability to produce results. if necessary Repeat until you no longer can find value for money See also (Denning 2010. assuming you are still interested in what to do about the estimation problem. Advantages with this method: 1. Stopping project effort. Radical Management) for further excellent discussion on this. So. You will learn a general method that you can apply for the rest of your career. without being recognized as a clear failure. or an ineffective process. who is daring enough to improve project management. Craig and Brooks 2006). Usually you can do that (deliver something useful). well enough for practical purposes. or destroy your management reputation – they are not useful. Disadvantages with this method: 1. If your real final project (or product delivery and service costs) costs destroy profit margins. and keep people happy 3. Measure values and costs 3. and distinguish yourself! The beauty of the control principles (see later) is that it does not take a long time to prove they work in practice (to yourself. and they are prepared to improve qualities over time to reach satisfactory levels -‐ If staff/contractors are highly motivated to NOT exceed budgets and deadlines. with no effort or cost whatsoever. We can simplify them as follows: 1. When Estimates Might Work: There are conditions where making estimates might work. or not taught in textbooks 3. There will always be people who criticize anything different or new 4. You can deliver value early. the five control principles below offer some practical solutions. does not prove the estimates were ever correct. by using a previous or old system. any effort spent improving the older system will often be appreciated – especially if some improvements have been made visible early. You are forced to think about the whole system. when deadlines and budgets are used up -‐ If the qualities and performance of the system are not yet at necessary required levels. including people (not just code) 4. but people have no particular better expectations. However. It just proves that you can stop. But we have also learned many ways of covering this truth up (Collins and Bicknell 1998. Our estimating capability usually fails this simple test. Do something of value in a short time 2. Hopefully you are a manager reading this. You cannot waste much time or money before you realize that you have false ideas 2. when dubiously-estimated resources are used up. Part II: The Control Principles – Some Practical Solutions The previous discussion has tried to establish that there is no reasonable way to get useful (credible) estimates for non-trivial software projects. inside a multi- . or might seem to work: -‐ If effort on the project ceases. This is of course a big change to the way most managers manage IT or software projects. or destroy your planned return on investments. then management has an incompetent team. Adjust what you do next.
but more importantly – increments of delivering value to your stakeholders (Gilb 2010b). LEARN SMALL: Carry out projects in small increments of delivering requirements – so you can measure results and costs against (short term) estimates All software projects can be broken into early. 6. -‐ You can select the highest available deliverable value. start delivering some value to some stakeholder. be about one week at a time. and monetary costs. and don’t even try to learn how. each step -‐ And deliver it to the most critical stakeholder -‐ And prove that your designs actually work -‐ And get some sense of the time needed. Almost all projects can. week after week – surprising things then happen: -‐ People cease to care about the conventional deadlines (you beat them by far) -‐ People cease to ask for estimates of the monetary budget (you have high return on money spent) -‐ You are strongly encouraged to keep on going. Now to explain the control principles in greater detail. From HP. Assuming you can deliver reasonable value for effort spent. Leaving aside ‘how to decompose’ for a moment (Gilb 2008b. The steps should usually. until the value to be delivered is less than the costs (you are making investors look good) -‐ You end up delivering far more real value than other projects do. next week and every week thereafter. step-by-step. You get a stream of high value. You prove that it is really value. Not merely increments of building.year delayed project. And. we believe. Concepts of small delivery cycles with stakeholder feedback. But most project managers do not even try. a client who uses the Evolutionary project management (Evo) method (Cotton 1996) . if they really want to. Gilb 2010a & b) – if you can deliver early stakeholder value. you have full control of the costs. small increments. well before the end of the project (without this approach a conventional project deadline would have been Figure 7. then you get control over ‘value for money’.
and choose the one that gives best impact on quality. then the taximeter (real costs) is still ticking.1 method or the Unity Method (Gilb 2010b) is as follows: Plan. to reach the required quality levels. we need to measure both the actual impact. No exceptions. and change anything about the coming increments. It is amazing the practical power of this simple idea of Unity. The 1. to the level some cost-optimist has estimated. One instance of this was reported by David Long (Personal Communication 2010) referring to a very large software project. how can you factor such processes into an initial cost calculation? You cannot. unfortunately. A simple strategy for root cause thinking. to make sure it is roughly as estimated. in 1 week To deliver at least 1% Of at least 1 requirement To at least 1 real stakeholder Using at least 1 design idea. identifying and getting approval for the small high-value delivery steps (Gilb 2002).1. and note the real initial development cost of getting to the required quality levels.1.1. for each design increment. for example Portability (Gilb 2005b. Then on deployment. You have to do it as you work incrementally. Keep track of the real costs. It is essential to learn as much as possible. Managers are very frequently at the wrong level of ‘Why?’ in understanding a problem: root cause analysis can help. The main leverage is in solving the critical few problems that are frequent and damaging. and implement them.set. If you really try. include operational and other costs. with the highest estimated impact first. CA.” You can conclude that there is potentially considerable cost reduction leverage to be had from root cause analysis. 25 aircraft projects at Boeing (El Segundo. AT&T Switching 5ESS. However. and note the actual costs. the greatest improvements came from a small amount of work that was done as the result of specific root cause analysis performed on specific outages. though most experience of its use is in that area. 7. This will not. and management persists with encouragement and support. If we have to continue tuning the design. from implementation and deployment – you are likely to get some surprises about the actual costs.1. We should also estimate the expected development cost. of course.1. On at least 1 function of the system. But they can be considered as separate qualities. which works well in practice is asking ‘Why?’ The Japanese ‘Five Whys’ method (Wikipedia 2010) hints at the need for continuing to ask this question of the resulting answer until you hit an answer that you can deal with effectively. and therefore try something to avoid it. and would also have been overrun) -‐ Management shifts focus from the budget and the costs to return on investment (ROI). If we can find the root cause. then McDonnell Douglas. We should also make an estimate of the expected impacts for the highest-contribution-to-requirement-levels design options. by redesign. In one ‘outside-the-box’ example. . that you believe might get you back on track The set of designs needed to deliver a demanding quality level can usually be implemented one design at a time. Client experience) used this 1.1. in relation to its estimated costs (the best ROI). it almost always works. LEARN ROOT: If incremental costs for a given requirement level (and its proposed designs) deviate negatively from estimates – analyze the deviation´s root cause. we have to ask.1. Chapter 5).1. It is not a principle limited to software.1 method in practice. we can reduce costs and improve quality levels (and move towards our estimates). as early as possible. from what I saw. One process for learning is to seek the root cause (Goldratt 2008) of the deviation from our estimates. He wrote that: “But. and maybe reduce costs.
the deadlines and the budgets is not so important any more. once it becomes a deadline or a budget respectively. provides a recent example. and value delivery (in terms of meeting requirements). We implement designs incrementally so that we can get acceptable feedback on costs. Instead of implementing all your designs in a ‘big bang’ manner. developing and integrating over a hundred million bytes of . PRIORITIZE CRITICAL: You will have to prioritize your most critical requirements and constraints: there is no guarantee you can achieve them all. and integrating over seven million words of program and data for eight different processors distributed between a helicopter and a ship in 45 incremental deliveries [Ed. -‐ Where in the past ten years. or if the budget runs out (possibly because of a deadline being moved up earlier or a budget cut!). One of the smartest ways to deal with limited resources is intelligent prioritization (Gilb and Maier 2005). from 1996 a part of Lockheed Martin Marietta) “some ten years ago [Ed. and what they each actually cost. LAMPS software was a four-year project of over 200 person-years of effort. developed by IBM’s Harlan Mills (1980) they reported: “Software Engineering began to emerge in FSD” (IBM Federal Systems Division. Planguage’s impact estimation (IE) method (Gilb 1988. In such situations. If a deadline is hit. With a bit of luck. Every one of those deliveries was on time and under budget -‐ A more extended example can be found in the NASA space program. about 1970] in a continuing evolution that is still underway: -‐ Ten years ago general management expected the worst from software projects – cost overruns. called LAMPS.Figure 8. and what works and what doesn’t is learned in practice. fortnightly or monthly cycles). is a ‘limited resource’. A Navy helicopter ship system. Simple example of applying the ‘5 Whys’ (Velaction 2010) 8. and see how each of the designs actually works. Note 2%!]. and hopefully about 2% of the project financial budget. Deliver ‘high-value for resources-used’ first A time or cost estimate. 1980!]. and hoping to meet all the requirements and resource (time and cost) estimates. unreliable and incomplete software -‐ Today [Ed. developing over three million. within budget. Gilb 2005) is helpful in identifying the designs that give the estimated best stakeholder value for their costs. stakeholders then receive interesting value very early. FSD has managed some 7. In the Cleanroom Method. management has learned to expect on-time. late deliveries. we suggest you deliver the project value a little bit at a time. then at least there is still a complete live real system with delivered high-value. deliveries of high-quality software.000 person-years of software development. the whole concept of the project-level estimates. ‘A little bit at a time’ should be interpreted as delivering evolutionary steps of approximately 2% of the overall project timescales (say weekly.
So. the system requirements and design ideas can be updated to reflect feedback. The right-hand of the diagram shows the main sub-processes of Requirement Specification and the Design Process (Gilb 2005). Lets us say they had a 40% profit margin. If the project ran over. and they could be wrong by 40% to 200% (The NASA range.” Nobody does it better! Harlan Mills told me that they had to solve the persistent problem of cost overruns. IBM was using intelligent feedback and learning. That means 2% delivery cycles. they could just as well leave the business (Personal Communication). But this did not give them the accuracy they needed to avoid losing money. Note Impact Estimation and Priority Determination are within the Design Process . Notice the “45 deliveries” Mills cites. They actually had very sophisticated estimation technology based on a thorough collection of experiences (Walston and Felix 1977). Figure 9. If IBM did not find a better way. and necessary change. Within the Delivery Cycle. Quinnan describes their process of intelligent dynamic prioritization in detail in (Mills 1980 in IBM SJ 4-1980). actual delivery of the Evo step occurs. Priority management as an iterative process (Gilb and Maier 2005): The lefthand of the diagram shows the Strategic Management Cycle and the Delivery Cycle within Evolutionary Project Management (Evo) (The Development Cycle and Production Cycle are not shown). and none at all in the past four years. and any changes that have occurred. Mills’ colleague. -‐ There were few late or overrun deliveries in that decade. Within the Strategic Management Cycle. to come in ‘on time and under budget every time’. They would still lose money on most contracts. and using fixed-price contracts (as opposed to cost plus). because the government was getting smarter. IBM lost its own profit. they had to compensate for their estimation inaccuracy by incremental feedback. in relation to the contracted estimates.program and data for ground and space processors in over a dozen projects. see earlier Figure 3).
and the numerically specified requirement was to reduce the learning time from 60 minutes to ten minutes. Gilb 2010c). See Figure 11. but this level is OK for our purposes here). and some smaller ones. too many design specifications are at this level.A thorough explanation of systems development processes helping achieve intelligent prioritization is found in (Gilb 2005). Nothing less will suffice for large and complex systems (that is. the value prioritization is ambiguous and too subjective (Gilb 2010c. years of effort. 267) If we look at the Design Ideas in Figure 10 (from real case at Ericsson). several organizations have made good simple practical adaptations (Upadhyayula 2001 (Study of Evo at HP in 1990s). The agile community has adopted small iterative cycles (Gilb 2010c). and no estimates of the certainty of the estimates. Chapter 9) to identify the designs with highest value with regard to cost and risk. to better understand risk. To illustrate how IE achieves this. See also Figure 9. Scale Impact: The first set of estimates are our best guess as to what the result would be if we implemented each design individually and incrementally. for project involving hundreds of staff. Which of these designs is high risk? Well. Scale Uncertainty: The scale impacts are just estimates. 129. So there is no perceivable difference. but they have failed to adopt the notion of measurement of value and quality – which is essential for larger projects. ‘On-line Help’ is estimated to get us to the required 10 minutes. For smaller systems (small teams over a few months). of costs or results. and we can try them out first. and uncertainty. Impact Estimation (IE) documents error margins. no estimates for expected impacts on requirements. Johansen and Gilb 2005) of the essential ‘Evo’ development process ideas (Evo itself predating later ‘Agile’ methods) (Denning 2010. here is a real example: The main requirement we shall focus on is called ‘Learning’. Figure 10. 9. In this case. Unfortunately. as to what our experience would lead us to expect. the ± 5 minutes for ‘Online Help’ means that we think the worst we could get is a result of 15 minutes (10 + 5). so we need to assess the likely best-case and worst-case range of estimates. Brief description of a simple real example of some design ideas to improve learning time (Gilb 2005. No information about interesting quality and cost differences. And . there is no documentation. Gilb and Brodie 2010). This is one means of expressing that there is a risk that the real result will NOT be our initial estimate. Without explicit absolute quality requirement and design impact metrics. A few descriptive words and we are done: no estimates for costs. and millions of dollars in cost). RISK FAST: You should probably implement the design ideas with the ‘highest value with regard to cost and risk’ early We can use the Impact Estimation (IE) method (Gilb 2005. here. defined only at the ‘Gist’ (summary) level (there was in the real example more detail to the definition.
In this case.0 perfect) (Gilb 2005). Figure 11. So the IE table provides a warning that the ‘Picture Handbook’ design has a high risk of NOT giving us the result we want. or cost more than what we can tolerate. that does not settle the question of the costs and their uncertainty! Development Cost: We can make an estimate of the development cost (and/or any costs . we assign a Credibility score (0.0 worthless to 1. you would not need the other design ideas. like Learning. ask yourself which one of the four designs is the ‘smartest’ one to try out first? The design ‘On-line Help’ looks good to me. and ‘Picture Handbook’ looks like a loser. If ‘On-line Help’ succeeds.5). But. Further. This is not important for our purposes here. based on that ‘evidence & source’ information. as a ‘common currency’ to make comparisons requirements that we might well be looking at simultaneously to make a risk decision. on what basis? It is rare to find designers for software systems documenting the ‘why’ and ‘who’ of estimates.8) so we can stick with our belief that it will give us the best result. let alone the estimate of impact on a numeric quality requirement. An Impact Estimation table showing the impact of the design ideas in Figure 10 on the Learning objective (Gilb 2005.the best is 5 minutes (10 . So we have to take also consider some additional information. We use the Percentage Impact. 267) Evidence/Source: We document the evidence (on what basis. How good is the estimate of the impact? Who made it. the ‘who’). The Percentage Impact is just a way of expressing these same estimates in direct relation to the ‘Learning’ requirement level (where 10 minutes = 100% of the objective). as best we can. to get the result. We are primarily interested in the risk that it will cost us more than we have estimated. ‘On-Line Help’ comes out best (0. the ‘why’) and the source (person or document for the evidence. But not in this simplified example! Now based on the Scale Impact and the Scale Uncertainty.
then you risk losing 3 weeks of time and four times as much resource. though we considered it above. APPLY NOW: Learn early. You cannot play all your bets at once. We have one last trick up our sleeve to make a decision. which may seem contradictory – that given several designs you need to implement – you should not do the potentially high-value to cost design early. This assumes that there is high value at low cost estimated. There is a high value in knowing the reality. should it be falsely overoptimistic! 10. We could make a ± estimate for the costs. learn well. it becomes even more convincing that we should stick with ‘On-line Help’ as the design to try first (3. but the ‘high risk’ designs instead. You need to start learning the true costs of your designs for meeting your project requirements as early as possible. If you consistently choose the safest bets. and implementing unevaluated designs. in good time. credibility adjusted). but others). then your costs will be under better control. it is a lot cheaper than leaping before you look.1. In this case. of course you should not spend your employer’s money unnecessarily because of your own arrogance and incompetence – and lack of ethics. as software people so often seem to do. You should start learning for real. and being able to retreat. and discussed in connection with. Would you bet your life that you are right in your cost estimates? Would you deduct the amount you are in fact wrong from your pay? Of course not! So. this doesn’t apply to you the reader. It is in fact a reasonable fact-based reasoning or logical process that you should try to do. But it is a lot cheaper than just diving in.000 monetary units. ‘On-line Help’ is estimated to cost 25. the stakeholder is the system architect. Meetings and opinions just can’t beat reality. you have to learn to control costs during the project.1. then you have to get to the truth of the cost of meeting your project requirements. such as development duration). You have to use feedback and also learn to estimate better.1. (Of course. You have to humbly recognize that neither you nor anyone else is certain of the costs. that is the iterative implementation options that promise. or until you run out of resources. to deliver best requirement-value for costs. for example 80%±70% for a quality or a cost budget. and apply the learning to your current project The tenth control principle is implied by. Remember the 1.1 principle of decomposition discussed above? If you use a month to measure some cost and effect.aspects that interest us. Then the pace of learning cycles should be weekly until all requirements are delivered. or not.1. At least. as the week it would take you to find out that you do not yet know what you are doing! . and then failing. how risky). If you want to get control over costs and budgets. and you have to provide better evidence and better sources. the second week of any project. the other principles. In other words. but that the ± uncertainty is very wide. Do you feel lucky today punk? (Abridged from The Internet Movies Database 2010): This might all seem like a lot of ‘bureaucracy’ to some of you. but maybe you have something to learn about decomposing projects (Gilb 2008b) if you can’t identify weekly cycles.2 being the largest value for money. Credibility-adjusted Performance to Cost Ratio: When we modify the costs with the Credibility factor – a rough measure of how well we know the Design Impacts and Costs based on history (Evidence and Source). and consider how wide the range of cost estimates is (that is. But remember you have to do this in an iterative sequence. and the value is the value to the architect of validating the true impact and costs of their architecture. In that case there is an argument that we should find out whether the design is as promising as we optimistically would like it to be. Sometimes doing high risk first is useful: There is a related principle. learn often. and what our worst case equates to (estimate plus the highest cost border). based on good evidence. Longer two weekly or even monthly cycles are also practical.
Competitive Engineering: A Handbook for Systems Engineering. 2000. to deliver competitive levels of performance and quality. Elsevier Butterworth-Heinemann. Cost estimation requires far more than understanding code. and that projects learn what is real very quickly. Available from: http://www. Something more than the suggested principles apply to this problem. Horowitz.” Hewlett-Packard Journal. and Software Engineering using Planguage. DAC Case Studies: Management Objectives. 2007.php?fileId=254 [Accessed December 2010]. Available from: http://www. Jossey-Bass/Wiley.htm [Accessed December 2010]. Evolutionary Project Management is . Long term and total costs: Understanding initial development costs is tricky enough. 1996. Gilb.” Proceedings of ICSPI 2007. QC and a little about 25 Evo projects for aircraft engineering. “A Case Study in Globally-Distributed Lean Software Development. K. B. 2006). A.hpl.gilb. E. ISBN 9780470548684. Cotton. References Berntsen. Brooks. -‐ Management needs to also decide the time value of delivering specific system value levels (such as 99. However. T. Let us call this the high value delivery timing (HVDT). This is keeping your eye on the financial value.. Addison-Wesley. Slides are at http://www. W. Let us call this the maximum profitable cost (MPC).Chulani. “Evolutionary fusion: a customer-orientated incremental life cycle for fusion.hp. 25-38. markets and politics. function points and use cases. D.php?fileId=432 Boehm. ———. R. Constable. So understanding the real costs for developing initial solutions. S.gilb.com/hpjournal/96aug/aug96a3. But management would be foolish to believe these estimates are sufficiently reliable to bet their own career on them. The Leaders Guide to Radical Management. ISBN 0-684-81687-3. Requirements Engineering. initially and gradually (Grimstad et al. people. software is always part of a larger system of machines. or profitability.com/tikidownload_file. and with some luck might get the order of magnitude right. laws. Simon and Schuster Ltd.com/tiki-download_file. D. ———. Craig. 2006. We are for example talking about how to engineer long-term cost-drivers into the system. Software Cost Estimation with Cocomo II. Bicknell. B. 4. 2005. and R. J. Prentice Hall. Management needs to make sure that projects are organized to deliver highest possible value as early as possible. Madachy. with D. Reifer and B. 2002. 2010. But understanding long term operational and maintenance costs – which usually dwarf initial development costs – is even more difficult – as it is dealing with a more un-measurable and unpredictable future. 1998. is near impossible up front. Denning. stories. W. T. Abts. Plundering the Public Sector: How New Labour are letting consultants run off with £70 billion of our money. So management’s strategic planning needs to take into account these principles: -‐ Management needs to base their planning on the cost they are prepared to pay to deliver specified value (as specified in quantified requirements).Summary The world of software is complex enough in itself – some say software projects are the most complex projects we humans undertake. Collins. Steece. 1988.99% availability within 12 months). C. People will make estimates. Principles of Software Engineering Management. CRASH: Learning from the World’s Worst Computer Disasters. Reinventing the Workplace for the 21st Century. S. Clark. J. Brown. T. -‐ Management then needs to make sure that their projects are done one step at a time (2% steps of the MPC and/or the HVDT). Download of Chapter 10.
May/June 2005. 2008. See also his 2008 Slides. “Rapid and flexible product development: an analysis of software projects at Hewlett Packard and Agilent.no/research/se/publications/Simula.php?fileId=26 [Accessed January 2011]. “Value-Driven Development Principles and Values – Agility is the Tool. 2006.gilb. A. 4. Oslo.php?fileId=350 ———. Jørgensen and K.gilb. Johansen. Available from: http://www. 2005. Top level Critical Project Objectives: How to quantify and control key objectives at all levels of the project. Available from: http://www. Maier. 414-420. Moløkken-Østvold. Inc.com/resume_m_david_long. Part 2: Values for Value. I. E. 2010b. See also related slides at http://www.php?fileId=180 ———.gilb. Iansiti and R.2006. Mills.gilb. S. Also available from: http://www.” Masters thesis.no/research/se/publications/Grimstad. and T.com/tiki-download_file. Not the Master.gilb.com/tiki-download_file.php?fileId=41 [Accessed December 2010]. Guide for Managing & Writing Requirements. Available at: http://simula.1/simula_pdf_file Hooks. Agile Record. No Cure No Pay.” IEEE Software. “Practical Guidelines for Expert-Judgment-Based Software Effort Estimation.gilb.SE. North River Press. more user-friendly.” MIT Sloan Management Review.” IBM Systems Journal. Also available from http://www. T.com/tiki-download_file. The Choice. July 2010. “The management of software engineering: part 1: principles of software engineering. M. Also available from http://www. 1980). Gilb. Agile Record. 2006.gilb.php?fileId=65 [Accessed . 19. Available from: http://www. 2001.. Software Development Effort Estimation: Why it fails and how to improve it. “Product-Development Practices That Work: How Internet Companies Build Software. October 2010. “From Waterfall to Evolutionary Development (Evo): How we rapidly created faster.com/tikidownload_file. 2.com/tikidownload_file. 1994. ———. 2008b. Reprinted IBM Systems Journal 38 (1999).gilb. ———.php?fileId=32 Jørgensen. Massachusetts Institute of Technology. Compliance Automation. T.gilb. 2001. Software Effort Estimation Terminology: The Tower of Babel. M.com/tikidownload_file. and M. “Decomposition of Projects: How to Design Small Incremental Steps.imdb. Available from: http://simula. Also available from: http://www. 289-295.php?fileId=431 Gilb. 2008a. Available from: http://www. S. Proceedings of INCOSE 2005.com/tikidownload_file. Verganti.375 Lucent AT&T 5ESS Case. W. Grimstad. M. Brodie. Available from: http://www. 2010a. 2010c.gilb. T. Principles and Values – Agility is the Tool.php?fileId=77 and Chapter 5 Scales of measure from http://www.php?fileId=448 Gilb.” Proceedings of INCOSE 2008.pdf MacCormack. 1980. 2005.com/tiki-download_file. Available from: http://www.com/tikidownload_file. Upadhyayula. 3.. The 111111 or Unity Method for Decomposition.php?fileId=85 ———. Managing Priorities: A Key to Systematic Decision-Making. and more productive software products for a competitive multi-national market. Value Planning Slides for Scrum Product Owners. 2010. Available from: http://www.com/tiki-download_file. 2005.available from http://www. and L.com/title/tt0066999/quotes [Accessed December 2010].php?fileId=353 [Accessed December 2010]. H. Not the Master”. M.gilb.com/tiki-download_file.com/tikidownload_file.” Proceedings of INCOSE 2005. 42.php?fileId=60 Goldratt. ———. Winter 2001.php?fileId=38 [Accessed December 2010]. 4 (Dec. Available from: http://www.mdavidlong. Presented at the 2010 Smidig (Agile) Conference.gilb.gilb. The Internet Movies Database (2010) Memorable quotes for Dirty Harry.com/tiki-download_file.
com Twitter @ImTomGilb .com URL: http://www.velaction. His other books include ‘Software Inspection’ co-authored with Dorothy Graham (1993). “A method of programming measurement and estimation. 2010. 2010. C. 1/1977. P. Tom’s key interests include business metrics. Available from: http://www. His latest book is ‘Competitive Engineering: A Handbook For Systems Engineering. and further development of his planning language. since 1960. 1977.December 2010]. Requirements Engineering.” IBM Systems Journal. 5 Whys. Email: Tom@Gilb. 54-73. E.org/wiki/5_Whys [Accessed December 2010]. Wikipedia. teacher and author. Lean term: 5 Whys. evolutionary delivery. Out of Print) has been cited as the initial foundation of what is now CMMI Level 4.com/5-whys/ [Accessed December 2010]. Available from: http://en. and ‘Principles of Software Engineering Management’ (1988). Velaction.Gilb. and Software Engineering Using Planguage’ (2005). Walston.wikipedia. His ‘Software Metrics’ book (1976. and C. Felix. Biography Tom Gilb has been an independent consultant. Planguage.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.