You are on page 1of 11











Vol. 1, No. 3 February 2006

Identifying Software Development Project Risks:

What You Don’t Know Will Hurt You

1441 Branding Lane Suite 240 Downers Grove, IL 60515 V: 630.434.8200 F: 630.434.8391

©2006 Geneca LLC

Identifying Software Development Project Risks: What You Don’t Know Will Hurt You
Table of Contents: Introduction........................................................................1

Typical Software Risks ....................................................2 Technical Risks ................................................................3 Process Risks ..................................................................4 Engagement Risks ..........................................................5 Organizational Risks........................................................6 Identifying Software Development Project Risks ....8 Conclusion ........................................................................9 About the Author..............................................................9



ccording to the Airlie Software Council — a group composed of the nation's most distinguished Software Engineering experts — risk management is the most important best practice for software developers to master.

While most software practitioners have a healthy paranoia about the importance of early identification and management of project risks, identifying risks in the development process might not be as clear cut as you think. Although the majority of technology people believe that most projects fail because of either technology or process, we have found just the opposite to be true: Technology and process risks are, in fact, easy to mitigate. The risks capable of killing your project are the organizational and engagement risks. Why? Because these risks have more to do with ingrained behavioral and culture issues rather than mechanical ones. Identifying project risks is the first step in the development of a healthy risk management program – or the systematic identification and elimination of risks that could cause a project's ROI to be compromised. This paper represents battle-tested thinking on risk identification and includes a comprehensive framework to check against.


“If You’re Not Managing Risk, You’re Not Managing”
oftware engineers are known to be eternal optimists: We either assume everything will go according to plan or we maintain that the creative nature of software development makes predictability too difficult to achieve. Both of these attitudes lead to unexpected events that have the potential to throw your project off track.


Risk management is becoming recognized as a best practice in the software industry for reducing the surprise factor. While we can never predict the future with certainty, we can apply structured processes to contain risk by managing concerns before they become crises. Symptoms that an organization is not successfully practicing risk management include a persistent state of project instability, frequent fire-fighting, multiple schedule slippages because of recurring surprise factors, and continuously operating in crisis management mode. Formal risk management greatly improves the likelihood of successful project completion while reducing the negative consequences of the risks that cannot be avoided.

Typical Software Risks
Typical Software Risks The list of things that can befall a software project is miserably long. The enlightened project manager will begin to build extensive lists of potential risks from the get go to help the team deal with as many concerns as early as possible in the planning process. Typically, we look for four categories of risks. Most projects include risks from each of these categories: 1. Technical risks involve the technology being used on the project. These risks include working on unproven software, difficult technical objectives (performance goals), and solutions to technical design problems that are not known. 2. Process risks involve the processes used to accomplish project tasks. These include processes not being followed, inconsistent processes, lack of process for a particular task, and processes that may have been omitted (example: testing, project management, risk analysis) 3. Engagement risks cover general engagement issues such as client and vendor behavior, aggressive time frames, and insufficient funding. 4. Organizational risks cover the structural organization of the development team. Risks include unproductive resources, staff turnover, ineffective coordination between multiple teams and companies, project personalities, and running remote projects. The key to managing risks is to understand where to look for potential risks, develop contingency plans for these risks, and build enough time into your project schedule to mitigate risks that you do not know about. 2

Technical Risks
Type of Risk Difficulty of Identification Difficulty of Mitigation

Technical Process Engagement Organizational

Easy Easy Moderate Difficult

Easy Moderate Moderate Difficult

Our checklists are living things and not meant to be the definitive list of all risks. We revise it constantly, adding new risks as they crop up on our projects. Maintain your own inventory of risks and expand it over time. Revisit often. One of the biggest traps you can fall into is believing in static lists.

Technical Risks Technical risks involve the technology being used on the project.This is the category that usually first comes to mind when people think of risks. Technical risks include working on unproven software, difficult technical objectives (performance goals), and solutions to technical design problems that are not known. These risks also involve the processes used to accomplish these tasks and include processes not being followed, inconsistent processes, lack of process for a particular task, and processes that may have been omitted (Examples: Testing, project management, risk analysis). Like process risks, technology risks are usually not fatal to a project for a couple of reasons. First, technical risks are for the most easy to identify. You pretty much know when you are working with hardware, software, or networking components that are risky. Second, they are usually easy to mitigate. You can either build prototypes to resolve the risks or there are usually substitute products or solutions that you can use.

Technical Risk Checklist: Technical risks cover hardware, software, and networking. Unproven components Complicated logic Complex architecture Features not feasible Third party software Re-usable components available Technology match Hardware Constraints Performance factors Unproven hardware Access (for remote teams) Networking Constraints Capacity


Process Risks
Process Risks Process risks are associated with the methods and procedures that we use when we conduct a project. Process risks are slightly more deadly than technical risks so we are moving up the deadliness scale. The most famous risk here is scope creep and has to do with requirements. Many projects face uncertainty and turmoil around the product’s requirements. While some of this uncertainty is tolerable in the early stages, the threat to success increases if such issues are not resolved as the project progresses. If we don’t control requirements-related risk factors, we might either build the wrong product, or build the right product badly. Either situation results in unpleasant surprises and unhappy customers. Almost as common though, especially among small to medium size businesses, is the lack of established and well defined QA procedures. Identifying risks here is pretty straightforward. Just think of the major processes that need to be present for the project to be a success. Then ask yourself if the processes are present and how well defined they are. Become familiar with established requirements gathering and project management practices, and watch out for these risk factors: Process Risk Checklist: • Requirements Lack of clear product vision Lack of agreement on product requirements Unprioritized requirements New market with uncertain needs New applications with uncertain requirements Rapidly changing requirements Ineffective requirements change management process Inadequate impact analysis of requirements changes Unclear project ownership and decision making Inadequate planning and task identification Inadequate visibility into actual project status • Development Feasibility Suitability Reliability Deliverability

Scope and Feature Creep (So what else is new?) Your client agrees to a requirement for a Logon page: The client enters their user id/password; the password is validated and allows entry upon successful validation. Simple enough. However, during a meeting immediately prior to coding, the client tells your PM that they now want a daily report that shows how many people log in each day. The client figures that since we already have the login information, it will only take a few minutes to automate a report. Although this sounds simple to the client, it requires many different things to happen. First, the PM needs to amend the requirement document. The programmer needs to understand the new requirement. The testing team must design new test scenarios. The documentation team must now include this report in the documentation. And, finally, the user acceptance team must plan to test this.

• Project management approach Micro management Too hands-off Lack of experience Lack of organization skills Lack of leadership Lack of authority • Design Functionality Difficulty Interfaces • Performance • Change Requests • QA • Deployment • No-risk management process


Engagement Risks
Engagement Risks The risks that can really kill you are the organizational and engagement risks. Why? Because these have to do more with behavioral and cultural issues rather than mechanical issues that are straightforward to deal with. Unfortunately, although most technology folks would rather not have to deal with organizational and engagement issues, these are the ones with the highest impact and the most difficult to mitigate. This makes for a deadly combination. Engagement risks are associated with how the project is conducted. Examples of an engagement risk include projects that are done under an extremely short timeframe, projects under budget constraints, and the risk that probably gets the most people riled up – expectations. Engagement risks can also arise due to dependencies from outside firms or factors – such as client-supplied items or subcontractor relationships. Because we cannot usually control these external dependencies, mitigation strategies may involve contingency plans to acquire a necessary component from an alternative source or working closely with the source of the dependency to manage looming problems. Engagement risks typically cause the most tension on a project because often times they are the most visible of risks, especially to executives. As with the other risks, go through the checklist and identify those most likely to occur: Engagement Risk Checklist: • Timelines Aggressive timelines – "Death marches" Relaxed timelines – "No sense of urgency" Unrealistic commitments made – often for the wrong reasons Unrealistic expectations from managers and clients • Cost Total project budget Contractual arrangements Mid-project course changes • Scope Are we solving the right problem? What specifically will we build? What are the overall project goals? Is everyone behind them? Project necessity and importance Clarity of objectives Clarity of requirements Conflicting goals • Expectations Expectations set inappropriately Failing to manage expectations 5

Organizational Risks
Organizational Risks So what are organizational risks? Think people. Anything that has to do with people: Skill sets, personalities, location, and environment go into the organizational risks section. In thinking about organizational risks, we are reminded of an old Tootise Pop commercial where a boy is trying to find out how many licks it takes to get to the center of a tootsie pop. He goes to the wise old owl who licks the tootsie pop three times and tells the boy its takes three licks. In our version, we ask how many high maintenance people can a project stand? The answer, from the wise old owl, is three (or less if you have a small project). We recommend that you spend a lot of time thinking about organizational risks since they can be deadly. They tend to be deadly because these risks tend to go unseen until the last minute. There are two reasons for this: • First, it can be hard to read interactions between people or the impact an environment has on a team. • Second, most technologists do not have expertise in organizational development and tend to underestimate the impact these sort of issues have on a project. There is one risk here that we want to call special attention to because we see it so often. We call this a "Fault Line" risk and it particularly occurs when you have multivendor teams. Faultline is a term from geology that refers to a crack in the earth's crust. At a faultline, you usually have pressure and friction caused by the two sides rubbing against each other. Teams are the same way. Sometimes teams fracture into cliques that often rub against each other and build up pressure. Another organizational risk is lack of knowledge. The rapid rate of change of software technologies, and the increasing shortage of skilled staff, mean that our project teams may not have the skills we need to be successful. The key is to recognize the risk areas early enough so that appropriate preventive actions — such as obtaining training, hiring consultants, and bringing the right people together on the project team – can be taken in a timely matter. Although organizational shortcomings impede the success of many projects, don’t be surprised if your risk management plan doesn’t list very many of these. After all, the project manager is usually the person who is writing the risk management plan, and most people don’t wish to air their own weaknesses (assuming they even recognize them) in public.


Organizational Risks -con’t.
The organizational risks listed below are, in our experience, among the deadliest of all risks. Organizational Risk Checklist: • Risks emanating from individuals Technical skills and talents Inadequate training Poor understanding of methods, tools, and techniques Inadequate application domain experience Unfamiliar technologies or development methods Staff personality conflicts Experience Lack of team spirit Personnel issues: Personality conflicts; cooperation and communications; skill mix and experience; resource availability Focus issues: Alignment; is everyone pulling on the oars at the same time? Morale: Bataan Death March; rats leaving the ship; getting on the bandwagon • Leadership abilities Poor communication Inability to compromise • Productivity Lack of/inconsistent productivity People on the critical path are not the most productive members • Application area Risks arising from team dynamics Fault lines and factions • Cultural Work ethic Attitudes

• Contractual and compensation Fixed price Time and materials None • Departmental Networking Database Application development • Ideology Split along ideological lines. Examples include: Microsoft vs. Open Source; Mainframes vs. PCs; Java vs. Visual Basic • Environmental risks Work Environment: Distributed locations; office space; support infrastructure Business Environment: Organizational stability; is the organization stable or constantly undergoing reorganizations? • Executive involvement What sort of commitment and involvement come from the executives? Too little (this is the classic lack of executive support) or too much (this can lead to scope creep or analysis paralysis)? Nature: Supportive, antagonistic, or neutral Triggers: From project inception or when things get closer to crunch time • Organizational politics Vendors Management End Users • Overall support 7

Identifying Software Development Project Risks
Risk Technical Multi-branded multi-lingual website Application must support the ability to be multi-branded. Low – being mitigated • Expertise with CSS 2.0 • Define extent of multi-branding capabilities • Finish current phase and use as model of future development along with prototype • Mitigate by extensive testing • Test transactions in prototype Description Impact Mitigation Plan

UI-middleware interface Feasibility/complexity of backend transactions Environmental issues

Testing between the web application and the middleware layers has taken place with the existing UI. There may be issues with expanding some of the backend transactions through the software interface or directly. Issues with the IT infrastructure may continue to arise from time to time. These issues arose with the Citrix development environment and the test environment.

Medium – being mitigated High


• Monitor and resolve issues as they arise during phase I • Development — Move development off of Citrix — Move development offsite • Testing — Set up separate test environment — Setup remote access to web server • Test communication paths using prototype developed in phase I • Set up a contact with each one of the vendors

Communication with external services could be an issue Process Regression testing

Several questions remain regarding communication to imaging, bill payment, e-statements, and credit card systems.


Currently no automated testing infrastructure is in place. Automated testing tools like Quick Test Pro should be investigated and brought in. Expanding scope on a project of this complexity coupled with an aggressive timeframe must be limited at all costs. Issues are increasing at a rapid rate. Not all team members are aware of the issues or their ongoing resolution and status. Project communication and coordination must be improved immediately.

Low for short term

• Evaluate, select, and install automated test tools • Create automated test scripts to perform regression testing • Freeze requirements quickly • Establish strict change control procedures to eliminate scope creep • Create, maintain, and distribute central issues list • Publish weekly status reports

Scope creep


Project coordination


As illustrated in this example above, the key to successful risk management is to identify all risks you know about and build time in for risks you do not know about. The mechanics of doing this is to simply list the risks, and figure out the loss (in hours) you expect would happen if the risk occurs, and then list the probability of the risk happening. Then you can factor that risk into your project schedule by multiplying the loss by the probability.


Conclusion Risk management is becoming recognized as a best practice in the software industry for reducing the surprise factor. While we can never predict the future with certainty, we can apply structured processes to contain risk by managing concerns before they become crises. Risks must not be passively monitored. Rather, a given risk and its associated exposure must be identified for the sole purpose of increasing the likelihood of delivering a successful project by decreasing the probability and impact of the risk.

It is not a law of nature that software projects will run late or over budget. A careful program of risk analysis and abatement can reduce the probability of major problems.

About the Author: Kevin Mason, Geneca Client Partner Kevin is a seasoned technology professional with over two decades of experience in software development and architecture, risk management and mitigation, business analysis, IT strategic planning, systems thinking, and project management. Kevin created the role of Client Partner at Geneca to insure that all Geneca clients experience the support, certainty and peace of mind they are promised throughout the software development process. Prior to Geneca, Kevin worked for groundbreaking firms such as SHL Systemhouse and AGENCY.COM. His former clients include ABN Amro, First Chicago (now Bank One), State Farm, U.S Bank, Motorola, McDonalds, Purina, and AT&T. Universal Card, Purina, 3M, Medtronic, and Compaq. Kevin is an industry speaker and has presented at Internet Commerce expos and management seminars on topics that include IT strategy, project solutions, technology perspectives and Rational Unified Process. Additionally, he has written numerous articles on risk management and web technologies.