You are on page 1of 148

SOFTWARE ACQUISITION

MANAGEMENT GUIDELINES

by

Daniel Svennberg

A Thesis
submitted for the Degree of
Master of Science

Linköping, Sweden, 2001

The Department of Computer and Information Science


Linköping University

LiTH-IDA-Ex-01/32

Lars Ljungberg, Mats Ran Kristian Sandahl


Advisors, Q-Labs AB Examiner, Linköping University
Software Acquisition Management Guidelines

Riktlinjer för ledning av programvaruupphandling

LiTH-IDA-Ex-01/32

Linköpings Universitet,
Institutionen för Datavetenskap
Linköping University

Q-Labs, Linköping and Maryland


Q-Labs AB

Fraunhofer, Maryland,
Fraunhofer, Inc.
Center for Experimental Software Engineering

Copyright 2001 Daniel Svennberg

All rights reserved. No part of this publication may be reproduced,


stored in a retrieval system, or transmitted, in any form or by any
means including, without limitation, photocopying, recording, or oth-
erwise without the prior written permission of the author. Written per-
mission is not needed if this publication is distributed for non-
commercial purposes.
Abstract

The thesis provides guidelines from the viewpoint of the cus-


tomer for managing an acquisition project concerning an out-
sourced development of a software product customized for the
specific needs of the customer. Current frameworks for soft-
ware acquisition management such as SA-CMM, IEEE 1062,
ISO 12207, etc. as well as literature, articles, and interviews has
been used as a basis for formulating the guidelines.

The resulting guidelines include acquisition team roles descrip-


tions, checklists for artifacts, customizable processes, and cus-
tomization factors. The processes contain guidelines for the fol-
lowing issues: project steering, project management, planning
and organizing, acquisition training, requirements manage-
ment, risk management, contractor selection, contract manage-
ment, product evaluation, transition to support, and post par-
tum.

The guidelines can be used as a basis for planning an acquisi-


tion project or be read as an overview over the many issues that
concern software acquisitions.

i
Acknowledgements

Many thanks to (in alphabetical order)…

Fraunhofer Institute, Jaak Urmi, John Marciniak, Kristian San-


dahl, Lars Ljungberg, Linköping University, Mats Ran, Mikael
Lindvall, Q-Labs, Victor R. Basili, Winifred Menezes, Östen Os-
karsson, and everybody else

… for your help making this thesis possible.

D.S.

ii
Table of Contents

Abstract.................................................................................i

Acknowledgements ............................................................ii

1 Introduction ...............................................................1
1.1 Background.............................................................................1
1.2 Target audience......................................................................2
1.3 Objective .................................................................................2
1.4 Problems .................................................................................2
1.5 Scope ........................................................................................3
1.6 Approach.................................................................................3
1.7 Outline.....................................................................................4

2 Software Acquisition Environment .........................6


2.1 The nature of software .........................................................6
2.2 Software acquisition.............................................................7
2.3 Software development .........................................................8
2.4 Outsourcing............................................................................9
2.5 Common problems .............................................................13
2.6 Frameworks ..........................................................................16

3 Customization .........................................................20
3.1 Customization levels ..........................................................20
3.2 Customization factors.........................................................21
3.2.1 Product characteristics............................................................22
3.2.2 Effort........................................................................................24
3.2.3 Contracting environment .......................................................26
3.2.4 Other factors ...........................................................................28
iii
Table of Contents

4 Roles ........................................................................29
4.1 Sponsor roles........................................................................29
4.1.1 Sponsor....................................................................................29
4.1.2 Collaborating customer...........................................................30
4.1.3 Product manager.....................................................................30
4.2 Managing roles ....................................................................30
4.2.1 Acquisition manager...............................................................30
4.2.2 Technical manager ..................................................................32
4.2.3 Contract manager ...................................................................32
4.2.4 Administrator .........................................................................32
4.3 Assisting roles......................................................................33
4.3.1 Acquisition expert...................................................................33
4.3.2 Technical expert ......................................................................33
4.3.3 Domain expert.........................................................................33
4.3.4 Legal expert .............................................................................34
4.3.5 Translator................................................................................34
4.3.6 End user ..................................................................................34
4.3.7 Maintenance and support staff ...............................................34
4.3.8 Independent verification and validation .................................35
4.3.9 System engineering and technical assistance .........................35
4.4 Executing roles.....................................................................36
4.4.1 Contractor ...............................................................................36
4.4.2 Collaborating contractor .........................................................37
4.4.3 Subcontractor..........................................................................37
4.4.4 Supplier ...................................................................................37

5 Artifacts ...................................................................38
5.1 Inception ...............................................................................39
5.1.1 Acquisition proposal ...............................................................39
5.1.2 Project specification ................................................................40
5.1.3 Acquisition plan......................................................................41
5.1.4 Risk list....................................................................................43
5.1.5 Requirements specification .....................................................44
5.1.6 Maintenance plan ...................................................................46
5.2 Solicitation............................................................................47
5.2.1 Request for proposal ................................................................47
5.2.2 Contractor evaluation criteria ................................................49
5.2.3 Proposal...................................................................................50
5.2.4 Contract...................................................................................51
5.3 Monitoring............................................................................53

iv Software Acquisition Management Guidelines


Table of Contents

5.3.1 Artifacts from the contractor ..................................................53


5.3.2 Product evaluation plan..........................................................54
5.3.3 Product evaluation report .......................................................55
5.3.4 Change request........................................................................55
5.3.5 Project status report................................................................56
5.4 Finalization...........................................................................57
5.4.1 Deliverables.............................................................................57
5.4.2 Post partum plan.....................................................................57
5.4.3 Post partum.............................................................................58

6 Processes................................................................60
6.1 Overview...............................................................................60
6.2 Project steering ....................................................................66
6.3 Project management ...........................................................69
6.4 Planning and organizing ...................................................72
6.5 Acquisition training ...........................................................74
6.6 Requirements management ..............................................76
6.7 Risk management................................................................78
6.7.1 Risk identification ...................................................................80
6.8 Contractor selection ............................................................85
6.8.1 Contract types .........................................................................88
6.9 Contract management.........................................................96
6.9.1 Project status metrics............................................................100
6.9.2 Earned value..........................................................................102
6.10 Product evaluation ............................................................106
6.10.1 Product evaluation approaches .............................................109
6.11 Transition to support........................................................110
6.12 Post partum.........................................................................113

7 Results and summary ..........................................116


7.1 Guidelines summary ........................................................116
7.2 Outlook ...............................................................................122

8 Bibliography..........................................................124

A Frameworks coverage..........................................129
A.1 SA-CMM .............................................................................129
A.2 FAA-iCMM.........................................................................130
A.3 IEEE 1062.............................................................................131
A.4 ISO 9000-3 ...........................................................................132

© Daniel Svennberg 2001 v


Table of Contents

A.5 ISO 12207 ............................................................................133


A.6 ISO 15504 ............................................................................134
A.7 Euromethod ........................................................................135

Index.......................................................................137

vi Software Acquisition Management Guidelines


1 Introduction

“Much of present-day software acquisition procedures rests


upon the assumption that one can specify a satisfactory
system in advance, get bids for its construction, have it
built, and install it. I think this assumption is
fundamentally wrong, and that many software acquisition
problems spring from that fallacy.”
– Frederick P. Brooks, Jr.

This chapter states the background of the thesis, its objective,


the approach, and outlines the contents of the report.

1.1 Background
Many organizations have a need to acquire customized soft-
ware products that meet special demands that no commercial
off-the-shelf product can meet. Many times the most efficient
and economically viable solution to obtain the software is to
outsource the development to a contractor instead of develop-
ing the software in-house.

In order to manage such a software acquisition project success-


fully, many different non-trivial issues such as requirements
elicitation and management, risk analysis and management,
contractor selection, contract management, product evaluation,
etc. need to be addressed. Several frameworks and standards
have been developed for assisting and assessing such software
acquisition efforts.
1
1 Introduction

The purpose of this thesis is to gather practices from these


frameworks and other sources and to present them as guide-
lines for software acquisition project managers. The thesis along
with the diploma thesis (Assmann, 2000) of a German student,
Danilo Assmann, are parts of an international project on soft-
ware acquisition between Linköping University in Sweden,
Q-Labs in Sweden and USA, and the Fraunhofer Institute in
Germany and USA. The thesis by Assmann deals mainly with
the evaluation and selection of a contractor, while this thesis deals
mainly with the management of an acquisition project.

It isn’t possible for a single master’s thesis to cover this subject


in all its intricacies, therefore the guidelines should be consid-
ered as an introduction to software acquisition management,
and hopefully as an inspiration on further studies on how to
manage software acquisition projects.

1.2 Target audience


The primary audience group for this master’s thesis consists of
those who are assigned the task of managing a software acqui-
sition project. Naturally, all groups that are involved in soft-
ware acquisition projects may find this thesis interesting.

1.3 Objective
The objective of this master’s thesis is to provide guidelines
from the viewpoint of the customer for managing an acquisition
project concerning an outsourced development of a software
product customized for the specific needs of the customer.

1.4 Problems
The objective stated above can be broken down into seeking an-
swers to the following problems:

• What problems are common?

2 Software Acquisition Management Guidelines


1 Introduction

• What activities should be performed?

• What roles are conceivable?

• What artifacts are conceivable?

• How can the activities, roles, and artifacts be customized


for different kinds of projects?

1.5 Scope
The management of custom-made software product acquisi-
tions will be examined – not services. Fully or partially devel-
oped products are the main concern – not commercial off-the-
shelf products. The acquirer’s point of view is mainly ad-
dressed – not the contractor’s. Legal aspects will not be exam-
ined further than contract contents. The issues will be examined
from a business perspective instead of a governmental or mili-
tary perspective.

1.6 Approach
An introductory study of the subject was performed by reading
the thesis of the German student (Assmann, 2000), the different
frameworks listed in section 2.6, Frameworks, on page 16, web
pages, and articles on software acquisition, and performing in-
terviews with Jaak Urmi, Östen Oskarsson, and Mats Ran. This
gave an understanding of the concepts of software acquisition
management and a number of ideas on how to formulate the
guidelines and provide answers to the problems stated above.

I found that the different sources gave quite a coherent picture


of software acquisition management without many contradic-
tions. The sources contained mainly different descriptions of
the same issues. Like Assmann (2000) does in his thesis, I as-
sume that the frameworks are useful in their own context and
therefore are viable as input to the guidelines. Furthermore my
goal was to extract and assemble the lion’s share of the advice

© Daniel Svennberg 2001 3


1 Introduction

given in the main frameworks SW-CMM, SA-CMM, FAA-


iCMM, IEEE 1062, ISO 9000, ISO 12207, ISO 15504, and Eu-
romethod, with as little filtering as possible, and then augment
with advice from the other sources studied. Naturally, I had to
integrate and arrange the advice into a suitable and coherent
form for the purpose of the guidelines.

The ideas on how to describe the guidelines were presented


and discussed with several people, including Kristian Sandahl,
Lars Ljungberg, Danilo Assmann, John Marciniak, Jon Valett,
Winifred Menezes, Mikael Lindvall, and others. These discus-
sions motivated me to write the guidelines in their present
form.

The guidelines described in this thesis are not a new model for
software acquisition management, but an attempt to extract, as-
semble, and integrate advice from the existing models and
frameworks and present them as management processes, check-
lists for artifacts, descriptions of roles, and customization fac-
tors. The option would have been to describe the guidelines in a
more prosaic fashion, but that would have made the sugges-
tions for customization less logical and to the point.

1.7 Outline
The following is a description of the contents of each chapter in
the report:

Chapter 1, Introduction, on page 1 describes the background of


the thesis, its objective, and approach.

Chapter 2, Software Acquisition Environment, on page 6 in-


vestigates the nature of software, different types of software ac-
quisitions, software development, outsourcing issues, problems
with software projects, and software acquisition frameworks.

Chapter 3, Customization, on page 20 describes a number of


factors that can be used as a guideline on what level of formal-
ity is appropriate for the actual software acquisition project.

4 Software Acquisition Management Guidelines


1 Introduction

Chapter 4, Roles, on page 29 describes the different roles that


are conceivable for a software acquisition project.

Chapter 5, Artifacts, on page 38 contains checklists for the dif-


ferent types of artifacts that are conceivable for a software ac-
quisition project.

Chapter 6, Processes, on page 60 contains suggestions on cus-


tomized software acquisition management processes.

Chapter 7, Results and summary, on page 116 summarizes the


guidelines and contains an outlook on continued work.

© Daniel Svennberg 2001 5


2 Software Acquisition
Environment

“A typical software project can present more opportunities


to learn from mistakes than some people get in a lifetime.”
– Steve McConnell

This chapter introduces the concept of software acquisition, its


nature, environment, and problems. It also lists some frame-
works that deal with software acquisition management.

2.1 The nature of software


There are a number of inherent characteristics of software that
affect the acquisition and development of customized software
in a profound way. The most salient of these characteristics, in
my opinion, are the following:

Software is an almost infinitely malleable conceptual con-


™
struct. As a consequence this makes it difficult to settle
the requirements and makes the scope of the software
prone to change.

Software has no physical features and is therefore difficult


™
to visualize. Brooks (1995) tells us that this makes it hard
to design the software and to communicate the design of
the software, which in turn makes it difficult to estimate
how long it will take to develop the software. It also

6
2 Software Acquisition Environment

makes it difficult to see progress in the development of


the software.

Software is extremely cheap to manufacture, as Reeves


™
(1992) argues. All it takes is to transform the source code
into executable code and clone as many copies as you
wish. The actual costs for software lie in development and
maintenance.

Software is extremely complex. This is, according to


™
Brooks (1995), due to the fact that software has a very
large number of possible states, which increases exponen-
tially with size. This makes it very hard to get an over-
view of the software, which in turn makes it hard to de-
sign the software to fully meet all quality requirements
and to have full control over the possible errors.

2.2 Software acquisition


There are several different ways to acquire software. According
to IEEE 1062 (1998), software products can be classified accord-
ing to the degree to which the acquirer can specify the features
of the software.

Commercial-off-the-shelf software, or COTS, is commercially


available software. It is usually well defined and documented
and a wide range of commercial users has demonstrated its fit-
ness for use. The supplier isn’t normally willing to modify the
software for the needs of a specific customer, and also controls
the maintenance of the software. The cost to acquire the soft-
ware is low to medium and the delivery time is immediate. The
customer has no control of the quality characteristics of the
software.

Modified-off-the-shelf software, or MOTS, is similar to COTS


software with the difference that the supplier is willing to make
modifications to the software product according to the cus-
tomer’s requirements. The software product’s fitness for use
has been demonstrated in similar applications. The customer

© Daniel Svennberg 2001 7


2 Software Acquisition Environment

has some control of the maintenance of the product and its


quality characteristics, i.e. the customized part. Delivery time is
short to long and the cost to acquire the software is medium to
high.

Fully developed software are unique, one-of-a-kind or low-


volume applications fully customized for the specific needs of
the customer. The software product’s fitness for use is unprece-
dented, but the customer has full control over its maintenance
and quality characteristics. The cost to acquire fully developed
software is high and the delivery time is long.

The characteristics of these product classes are summarized in


the table below:

COTS MOTS FD*


Partially Fully
Scope Fix
customizable customizable
Demonstrated in
Fitness-for-use Demonstrated Unprecedented
similar applications
Maintenance No control Some control Full control

Delivery time Immediate Short-Long Long


Acquisition cost
Low-Medium Medium-High High
(Not usage/maint. cost)
Quality (ISO 9126) Not controllable Partially controllable Mostly controllable

*) Partially to fully outsourced


Table 1. Software acquisition types. FD stands for fully developed
software.

This thesis applies to MOTS and especially fully developed


software products.

2.3 Software development


To be brief, software development essentially consists of four
activities: analysis, design, coding, and testing. There are a
number of methods that describe how and when these activities
should be performed, ranging from code and fix, which is the
software development version of trial and error, to perhaps the

8 Software Acquisition Management Guidelines


2 Software Acquisition Environment

most well known method, the waterfall method, which in its


purest form is to perform the activities in the following sequen-
tial manner: analyze, design, code, test, and never go back to a
previous step.

More recently, iterative, evolutionary, or incremental develop-


ment methods such as eXtreme Programming (Beck, 1999) have
become popular. Following such a method means that the pro-
gram is developed in a series of iterations or increments with a
subset of the requirements implemented in each iteration; this
has the benefit that you can more easily change your mind
about the requirements as the program evolves. See the figure
below for a comparison between the waterfall model, the spiral
model and extreme programming.

Different software development methods suit different types of


projects, but that is not for this report to examine. What is im-
portant is that the contractor can produce good results with the
method they use.

Waterfall Spiral XP

Analysis

Design

Coding

Testing

Scope

Figure 1. The development cycles of the waterfall model, the spiral


model and extreme programming (Beck, 1999).

2.4 Outsourcing
There are many different reasons for outsourcing a software
development project, such as the following examples:

© Daniel Svennberg 2001 9


2 Software Acquisition Environment

• It’s difficult to hire developers.

• It’s cheaper than hiring developers.

• Somebody else can develop the program faster, cheaper,


and with higher quality than you can.

• You lack the necessary competence and resources.

• You want to use your staff for other tasks.

The following are examples of drawbacks with outsourcing:

• You lose some control over the software development


process, which implies that the risk is higher; possibly
you risk putting the fate of your competitiveness in the
hands of the contractor.

• You don’t build up your own software development


competence.

• You don’t get any indirect labor hours since you don’t
have your own development staff.

Haimes, et al. (1997) states that there are three groups of partici-
pants in a software acquisition project, each with separate goals:

• The end user wishes to have a product that function well


in operation. Another possible goal for the end user is to
get the product as quickly as possible.

• The customer, that is the participant that finances the pro-


ject, can have several goals: Similarly to the user, the cus-
tomer wishes to receive a software product that function
well in operation. The customer also wants to minimize
cost and deviation from time schedule. Furthermore, the
customer wants to maximize return on investment and to
have a well-functioning relationship with the contractor.

10 Software Acquisition Management Guidelines


2 Software Acquisition Environment

• The contractor on the other hand often wants to maximize


profits and to maximize the potential for future earnings,
such as additional contracts.

This gives us the optimization problem that a software acquisi-


tion project consists of, with several of the goals overlapping
and conflicting. For instance, the customer wants to maximize
the technical performance of the product and on the same time
minimize cost and deviation from time schedule. Furthermore,
the cost minimization goal of the customer conflicts with the
profits maximization goal of the contractor. Obviously trade-
offs are necessary to have an operational contract.

Participant Objective

End user Maximize technical performance


Minimize development time

Customer Maximize technical performance


Maximize customer-contractor relation
Maximize return on investment
Minimize cost
Minimize deviation from time schedule

Contractor Maximize profit


Maximize future earnings

Table 2. Participants in a software acquisition project and their goals.


Adapted from Haimes, et al. (1997).

The most common scenario when you think of an outsourcing


project is that you have two parties involved – a single cus-
tomer and a single contractor. Gallivan (1999) shows that this
simple dyadic relationship is not always the case, as more com-
plex outsourcing constellations are becoming more common-
place. Gallivan (1999) gives a taxonomy of outsourcing
constellations:

• The simple dyadic constellation consists of a single cus-


tomer outsourcing to a single vendor.

© Daniel Svennberg 2001 11


2 Software Acquisition Environment

• The multi-vendor constellation consists of a group of ven-


dors collaborating in a project outsourced by a single cus-
tomer.

• The co-sourcing constellation appears when a group of


customers that have similar needs cooperate and jointly
outsources a project to a single vendor.

• The complex constellation is a combination of the multi-


vendor and co-sourcing constellations, where you have a
group of customers jointly outsourcing to a group of col-
laborating vendors.

One Vendor Many Vendors

C V C V
One V
Client
Simple Dyadic Multi-Vendors
C C V

C V C V
Many
C C V
Clients
Co-Sourcing Complex

Figure 2. A taxonomy of outsourcing relationships. (Gallivan, 1999)

To make the outsourcing scenario even more complex you can


have one or more subcontractors involved and several support-
ing contractors and suppliers.

Compared to the dyadic constellation, the multi-vendor rela-


tionship allows vendor specialization and reduces the risk of a
vendor behaving opportunistically since the other vendors are
ready and willing to substitute a non-performing vendor.

12 Software Acquisition Management Guidelines


2 Software Acquisition Environment

Drawbacks are increased coordination costs and more complex


contracts, etc. (Gallivan, 1999)

Compared to the dyadic constellation, the co-sourcing relation-


ship allows risk sharing and reduction, and cost and time sav-
ings due to buyer economies of scale. Drawbacks are increased
coordination costs and that it is more difficult to control
information and keep secrets because of the interdependent
nature of the relationship between the customers, etc. (Gallivan,
1999)

2.5 Common problems


Marciniak and Reifer (1990) states that the main problems in
software acquisitions are:

• Development costs exceeds budget.


• Time schedule is overrun.
• Outcome does not meet expectations.

The Standish Group (1995) “Chaos” report investigated how


common these problems are. 8 380 software development pro-
jects were surveyed with respondents representing small, me-
dium, and large companies across major industry segments
such as banking, health care, and federal organizations, etc. The
report does not indicate the level of outsourcing in these soft-
ware development projects, but the statistics will still give an
idea of the extent to which software development projects suf-
fer from these problems.

The findings were:

• 16% of the projects were completed on time and on


budget, with all features and functions as initially speci-
fied. These projects were labeled as successful.

• 53% of the projects were completed and operational but


over budget, over the time estimate, and offered fewer

© Daniel Svennberg 2001 13


2 Software Acquisition Environment

features and functions than originally specified. These


projects were labeled as challenged.

• 31% of the projects were canceled at some point during


the development cycle. These projects were labeled as
canceled.

Successful
Canceled 16%
31%

Challenged
53%

Figure 3. Findings from the Standish Group (1995).

In addition, the average project exceeded planned budget with


90% and schedule with 120%.

The root cause to these problems is primarily ineffective man-


agement as stated in Brown, et al. (1998):

In 1987, the Defense Science Board concluded that these soft-


ware acquisition problems came not from technical difficulties,
but from poor management. Since 1987, the situation has gotten
worse, not better. Software has gotten bigger, more complex,
and more expensive, and ineffective management is still the
root cause of much that afflicts software acquisition.

Below is a list of some common problems in software acquisi-


tion projects. It is assembled and adapted from The Standish
Group (1995), Ganssle (1998), Marciniak and Reifer (1990),
Reifer (1997), and interviews with Urmi (2000), Oskarsson
(2000), and Ran (2000):

• Laissez-faire – The customer does not take an active part


in the project once the contract is signed.

14 Software Acquisition Management Guidelines


2 Software Acquisition Environment

• Administrative overload – Too much administratrivia to


monitor contract compliance instead of focusing on tech-
nical matters.

• Creeping scope – The customer keeps adding and chang-


ing scope and functionality to a job in progress on a tight
schedule with limited resources.

• Fragmentation – Members of the team, both customer


and contractor, are pulled off to work on other projects at
random times.

• Goldplating – Having overkill requirements or making


sophisticated and new rather than simple and proven
technical solutions.

• “I’m paid to engineer!” – The customer tells the contrac-


tor how to do the job, not what the job is.

• Missing or bad indicators – Measurement of progress


and overall performance is qualitative instead of quantita-
tive. The performance levels of the indicators are poorly
set.

• Who’s in charge? – Too many bosses, decisions aren’t


made in a timely manner.

• No end user involvement – Not getting and incorporat-


ing the end user’s point of view on the functionality and
usability of the product.

• Poorly defined requirements – The customer gives an in-


complete and unvalidated set of requirements to the con-
tractor.

• Acquisition incompetence – Failure to understand the


particular needs of software acquisitions.

• Overpromising – The marketing people of the contractor


makes promises that the technical staff can’t keep.
© Daniel Svennberg 2001 15
2 Software Acquisition Environment

• Lack of discipline – Reverting to an ad hoc development


process when deadlines are getting close. “Gotta crank
out that code!”

• Unrealistic expectations – Having an impossible sched-


ule, and/or being unaware of the limitations of the tech-
nology being used.

• Inadequate resources – Not having appropriate financial


funding or lacking appropriate staff, tools, and equip-
ment.

• No executive support – No support or interest in the pro-


ject from top management.

• Lack of clear objectives – No clear statement of goal or


vision causing project members to pull in different direc-
tions.

• Ineffective communications – Lack of effective commu-


nication channels, information not reaching the right peo-
ple in a timely manner.

• Lack of competence – Not having the appropriate techni-


cal and leadership skills.

• Friction – Cooperation is hindered by some of the in-


volved parties not getting along for some reason.

All of these problems can probably be avoided by better soft-


ware acquisition management practices. Also see the risk tax-
onomy given on page 80.

2.6 Frameworks
The following is a list of frameworks that in some way gives
guidance on software acquisition management. Apart from
standards, books and collections of guidelines are mentioned.

16 Software Acquisition Management Guidelines


2 Software Acquisition Environment

There are of course more frameworks that cover the subject


than those listed1.

SW-CMM – The Capability Maturity Model for Software is one of


the most well known frameworks for assessing the capabilities
of software contractors. The purpose of the model is to help or-
ganizations determine the maturity of their current software
development capabilities. The model characterizes the level of
an organization’s maturity based on the extent to which meas-
urable and repeatable software engineering and management
practices are institutionalized. The software acquisition activi-
ties examined for this thesis are in the Software subcontract man-
agement key process area of the model. See Paulk, et al. (1993).

SA-CMM – The Software Acquisition Capability Maturity Model is


a framework developed to help organizations assess and im-
prove their acquisition capabilities for software-intensive sys-
tems. The structure of the model is consistent with SW-CMM in
that it is a staged model with key process areas grouped in five
maturity levels: initial, repeatable, defined, managed, and op-
timizing. See Cooper, et al. (1999).

FAA-iCMM – The Federal Aviation Administration Integrated


Capability Maturity Model is an effort to combine the features of
SW-CMM, SA-CMM, and SE-CMM2 into one consistent model.
It is intended to be used as a tool for organizations to evaluate
their acquisition, management, and engineering practices and
to devise improvements on them. See FAA-iCMM (1997).

CMMI – The Capability Maturity Model Integration framework is


an effort to integrate existing maturity models. CMMI is under
development and will eventually consist of a series of disciplines
ranging from software engineering and systems engineering to

1www.software.org/quagmire is a good starting point to find out more


about available standards.

2 Systems Engineering Capability Maturity Model.

© Daniel Svennberg 2001 17


2 Software Acquisition Environment

software acquisition, etc. A requirement for the CMMI frame-


work is consistency with ISO 15504. This framework had not
been extended to contain the software acquisition discipline
when this thesis was written. See CMMI (2000).

IEEE 1062 – The IEEE Recommended Practice for Software Acquisi-


tion is a standard that takes a comprehensive approach to giv-
ing recommended practice for managing software acquisition
projects regarding COTS, MOTS, and fully developed software.
It covers the whole acquisition process from planning organiza-
tional strategy to using the software and is compliant with ISO
12207. See IEEE 1062 (1998).

ISO 9000 is a series of standards for quality management and


quality assurance. It is mainly intended to provide guidance in
a situation where a contract between two parties require the
demonstration of a contractor’s capability to develop, supply
and maintain products. In this series, the standards that are im-
portant for software development projects are ISO 9001 and ISO
9000-3. ISO 9001 – Quality systems – Model for quality assurance in
design / development, production, installation and servicing deals
with generic two-party contractual situations and ISO 9000-3
Guidelines for the application of ISO 9001 to the development, supply
and maintenance of software gives guidelines for the application
of ISO 9001 to software projects. ISO 9001 is roughly compara-
ble to SW-CMM level 2. See SS-ISO 9000-3 (1992).

ISO 12207 – The Standard for information technology – Software life


cycle processes takes a holistic and comprehensive approach to
the whole life cycle of a software product. It includes a well-
defined terminology for software life cycles that is intended to
standardize the terminology used by the industry. The standard
can be used to establish software acquisition, supply, develop-
ment, operation, maintenance, and supporting processes. It is
also intended to foster an improved understanding between
customers and vendors and among the parties involved in the
life cycle of a software product. Worth mentioning is the goal of
facilitating world trade in software. It replaces older U.S. mili-
tary standards such as DOD-STD-2167A and DOD-STD-2168.
See IEEE/EIA 12207.0 (1996).

18 Software Acquisition Management Guidelines


2 Software Acquisition Environment

ISO 15504 – The Information Technology – Software Process As-


sessment standard is a framework for assessing software proc-
esses regarding acquisition, development, support, etc. It is in-
fluenced by and similar to SW-CMM in such that it has a
capability scale. It is different from the SW-CMM in that it has
not a discrete pass/fail mechanism and that it is aimed at
assessing individual processes and not organizations. It is
strongly aligned with ISO 12207. See ISO/IEC 15504 (1998).

Euromethod is a framework with the intent to assist an acquisi-


tion at the contractual level. It aims to help the customer and
contractor understand each other, improve risk management,
and harmonize the terminology used in an acquisition effort.
According to Breu (1995), Euromethod is particularly aimed at
guiding public procurement processes in the context of the
open European market, but can be used for all types of software
acquisition projects. See Euromethod (1996).

Other frameworks that provide important and useful advice for


software acquisition management are the following: The Road to
Successful ITS Software Acquisition (Salwin, 1998), Software Acqui-
sition Management (Marciniak and Reifer, 1990), The Program
Manager’s Guide to Software Acquisition Best Practices (Brown, et
al., 1998), Guidelines for Successful Acquisition and Management of
Software-Intensive Systems (Mosemann, 1996), A Guide to the Pro-
ject Management Body of Knowledge (Duncan, 1996), and Defense
Acquisition Deskbook (DAD, 2000).

© Daniel Svennberg 2001 19


3 Customization

“Depending on the circumstances, you should be as hard as


diamond, flexible as willow, smooth-flowing like water or as
empty as space.”
– Morihei Ueshiba

This chapter investigates project categorization factors that in-


dicate what level of formality is necessary for the actual soft-
ware acquisition project.

3.1 Customization levels


Software acquisition projects have different characteristics.
Some projects are small, short-term, and easy to grasp, while
others are the other way around. Cockburn (n.d.) states that

… each organization must develop its own way of working,


and each project must get its own, tailored methodology, deal-
ing with culture, location, criticality, and technology.

To have effective acquisition processes that give good results, it


is beneficial to employ an adequate level of formality for those
processes according to the characteristics of the actual project.
For the purpose of this thesis I give suggestions on three levels
of formality for the acquisition management processes. These
levels are:

20
3 Customization

Minimal processes – The project is easily managed and the


purpose of the acquisition processes is achieved with a low
level of rigor. The processes contain just the activities without
which the project would really be in trouble. This level is suit-
able for low-risk projects.

Controlled processes – The project has a tangible complexity


and a reasonable risk of going out of hands. To contain the
complexity and risks, a higher level of formality is appropriate
for the acquisition processes. The processes on this level contain
the activities of the minimal processes and add some activities
that are not absolutely necessary but make things more con-
trolled and reliable. This level is suitable for medium-risk pro-
jects.

Robust processes – The project is highly complex and has a


strong tendency to get chaotic unless it is managed appropri-
ately and performed with discipline. The processes contain ac-
tivities that increase the formality and administrative effort but
in return make things more robust and reliable. This level is
suitable for high-risk projects.

An acquisition process level below minimal would imply ad


hoc, laissez-faire processes, which is not recommended.

3.2 Customization factors


To determine what level of formality is appropriate for your
project you can look at the customization factors in the follow-
ing sections. The factors have been inspired by D.I.R. (2000),
ISO 12207 (1995), and Euromethod (1996) and should be viewed
as guidelines and not exact values on what level to choose. It is
important to look at several factors and determine which are
most important for your project before you decide upon a for-
mality level.

© Daniel Svennberg 2001 21


3 Customization

3.2.1 Product characteristics


The product characteristics factors obviously stem from the char-
acteristics of the software product to be acquired.

Requirements volatility
Since software development is a creative process, and it is diffi-
cult to anticipate all technical problems beforehand, change of
requirements occur more or less in all software development
projects. However, due to the nature of the problem the soft-
ware is to solve, the total set of requirements can be more or
less change-prone.

Minimal proc- Stable requirements. Highly unlikely that the


esses suffi- overwhelming majority of the requirements will
cient change.

Controlled A significant subset of the requirements is vola-


processes tile.
recommended

Robust proc- Volatile requirements. High degree of uncer-


esses rec- tainty on a large or important subset of the
ommended requirements.

Program size
The estimated program size indicates the effort that is needed
to build the software. The values given in the table below are to
be considered as suggestions since program size depends on
what language is used and other factors.

Minimal proc- Small. Less than 32 KLOC1.


esses suffi-
cient

1 1 KLOC = 1 thousand lines of source code.

22 Software Acquisition Management Guidelines


3 Customization

Controlled Medium. 32 – 128 KLOC.


processes re-
commended

Robust proc- Large. More than 128 KLOC.


esses rec-
ommended

Criticality
Criticality is the impact malfunctioning software will have on
people’s safety and the magnitude of financial loss that can re-
sult, according to Oskarsson and Glass (1996).

Minimal proc- People’s safety is not at risk, nor will any signifi-
esses suffi- cant amount of money be lost if the software mal-
cient functions.

Controlled People’s safety is not at risk, but there is a risk of


processes moderate financial losses due to software mal-
recommended functioning.

Robust proc- The safety of people is at stake or huge amounts


esses rec- of money. A disaster occurs if the software
ommended doesn’t function properly.

Innovation
Oskarsson and Glass (1996) define innovative projects as those
in which the problem is new and different and hasn’t been
solved with computer programs before. Less innovative pro-
jects use mature and proven technology.

Minimal proc- Mature technology. Many similar applications


esses suffi- exist.
cient

Controlled The technology has been proven to work and is


processes expanding but few similar applications exist.
recommended

© Daniel Svennberg 2001 23


3 Customization

Robust proc- Emerging unproven technology where the ques-


esses rec- tion is more on whether the problem can be
ommended solved. Unique application.

Transformational impact
Transformational impact is the depth of change that the deliv-
ered software product will have on the user’s behavior and the
user organization’s processes and products.

Minimal proc- Minimal change for the user.


esses suffi-
cient

Controlled Moderate modifications or amendments to the


processes user’s behavior or the internal work processes
recommended and products of the user organization.

Robust proc- Fundamentally changes the behavior of the user


esses rec- and the work processes and products of the user
ommended organization.

3.2.2 Effort
The effort customization factors delivery time, total cost, and de-
velopment team size are implied by the estimated effort to build
the software. The estimate is based on the product characteris-
tics. The larger the effort is the higher formality is required to
avoid unnecessary chaos.

Delivery time
Delivery time is the estimated calendar time for delivery of the
finished software product. With a longer project there is a
higher likelihood that new people will join the development
team and others leave. Change in the requirements is also more
likely. The values given in the table below are to be considered
as suggestions.

24 Software Acquisition Management Guidelines


3 Customization

Minimal proc- 18 months or less.


esses suffi-
cient

Controlled 12 – 36 months.
processes
recommended

Robust proc- 24 months or more.


esses rec-
ommended

Total cost
Total cost is the estimated total cost to acquire the software in
order of magnitude. The value ranges of low, medium, and
high in the table below are difficult to specify since these values
may change over time and are only given as suggestions.

Minimal proc- Low. Less than $50,000.


esses suffi-
cient

Controlled Medium. $50,000 – $1,000,000.


processes
recommended

Robust proc- High. More than $1,000,000.


esses rec-
ommended

Contractor development team size


The more people that are involved in the contractor’s develop-
ment team the more communication is needed to coordinate the
work, sort out misunderstandings, and resolve conflicts. The
values in the table are not to be considered as absolute.

Minimal proc- Less than 10 developers.


esses suffi-
cient

© Daniel Svennberg 2001 25


3 Customization

Controlled 10 – 30 developers.
processes
recommended

Robust proc- More than 30 developers.


esses rec-
ommended

3.2.3 Contracting environment


The contracting environment customization factors are related to
the contractor and the organizational environment that the ac-
quisition takes place in.

Organizations involved
The more organizations, such as stakeholders, supporting con-
tractors, consultants, independent testing organizations, etc. are
involved, the more communication is needed to coordinate the
work.

Minimal proc- Two parties involved in a simple dyadic relation-


esses suffi- ship; customer and contractor.
cient

Controlled More than two parties are involved; either sev-


processes eral customers co-sourcing, or several contractors
recommended collaborating, or subcontractors are used, and
possibly supporting contractors. The organiza-
tional structure is not (too) complex.

Robust proc- Several organizations are involved in a complex


esses rec- organizational structure composed of some or all
ommended of the following: multiple customers, multiple
contractors, multiple subcontractors, multiple
supporting contractors, etc.

26 Software Acquisition Management Guidelines


3 Customization

Contractor development team characteristics


The contractor’s development team characteristics are the ex-
perience, constitution, and performance of the contractor’s de-
velopment team.

Minimal proc- Low staff turnover. Mostly experienced and well-


esses suffi- trained personnel. Well functioning teamwork.
cient Efficient and appropriate working methods.
High performance.

Controlled Moderate staff turnover. Some inexperienced and


processes untrained personnel. Teamwork could be func-
recommended tioning better. Working methods need some im-
provement. Medium performance.

Robust proc- High staff turnover. Lots of inexperienced and


esses rec- untrained personnel. Teamwork not functioning
ommended optimally. Inefficient and inappropriate working
methods. Low performance.

Experience with contractor


The type and level of experience with the contractor(s).

Minimal proc- The customer and contractor have worked to-


esses suffi- gether on several previous occasions and have a
cient good relationship built on trust. The customer
knows how the contractor works and vice versa.

Controlled First time working with a contractor with good


processes results working for other customers, or mixed
recommended experiences with a previously used contractor.

Robust proc- First time working with a contractor that is not


esses rec- well known or has had bad results working for
ommended other customers, or bad experiences with a previ-
ously used contractor.

© Daniel Svennberg 2001 27


3 Customization

Geographic dispersion
Management becomes more difficult with customer and con-
tractor on different geographical sites.

Minimal proc- The customer, contractor, and other involved


esses suffi- parties work at the same site or can visit and
cient work on each other’s site on a daily basis.

Controlled The customer, contractor, and other involved


processes parties work at different sites, with their respec-
recommended tive teams located at the same site, and cannot
meet on a daily basis.

Robust proc- The customer, contractor, and other involved


esses rec- parties have several teams involved in the project
ommended dispersed on several different sites and they can-
not meet on a daily basis.

3.2.4 Other factors


The factors stated above are only suggestions and you will
probably have to look at more factors to be able to customize
the project appropriately. Other factors include: standards re-
quired, laws and regulations, other projects, cultural differ-
ences, application domain, system complexity, etc.

28 Software Acquisition Management Guidelines


4 Roles

“No matter how much you want it to be a technical


problem, it’s a people problem.”
– Unknown

Having good people and good teamwork is key to the success


of the project. This chapter presents the different types of roles
that may be necessary for a successful software acquisition pro-
ject. One person can be assigned several roles. The roles de-
scriptions are assembled from and based upon the different
frameworks stated in appendix A, Marciniak and Reifer (1990),
Salwin (1998), and the interviews with Urmi (2000), Oskarsson
(2000), and Ran (2000).

4.1 Sponsor roles


The sponsor roles are played by persons representing the or-
ganizations that have the powers to initiate and cancel the ac-
quisition project.

4.1.1 Sponsor
The sponsor initiates and supports the software acquisition pro-
ject enthusiastically but also has the power to cancel it if it will
cost more to finish than will be gained from finishing the pro-
ject. The responsibilities of the project sponsor are:

29
4 Roles

• To define and communicate a goal and vision for the pro-


ject.

• To provide a stable and adequate funding and set the fi-


nancial limits and other bounds for the project.

• To appoint an acquisition manager that has the ultimate


responsibility for the success of the project. The sponsor
should be careful not to interfere too much with the way
the acquisition manager manages the project.

4.1.2 Collaborating customer


Should the acquisition project be a co-sourcing project with
several customer organizations involved, the collaborating cus-
tomer is another project sponsor whose collaboration needs to
be coordinated with the other sponsors.

4.1.3 Product manager


In some cases the software is part of a larger systems acquisi-
tion with the product manager for that system playing the spon-
sor role.

4.2 Managing roles


The managing roles are played by the people on the customer
side that are put in charge of managing, monitoring, and ad-
ministering the proceedings of the acquisition project.

4.2.1 Acquisition manager


The acquisition manager is appointed by the sponsor and is ulti-
mately responsible for the success of the project. Therefore, the
acquisition manager has the final word on how the project
should be performed. The tasks and responsibilities of the ac-
quisition manager involves the following:

30 Software Acquisition Management Guidelines


4 Roles

• To adapt and customize the acquisition strategy accord-


ing to the project characteristics.

• To plan the project and execute it accordingly, re-plan as


the project progresses, manage risks, and solve problems.

• To recruit and organize people for the acquisition team,


perform teambuilding and training activities, praise and
encourage people, smooth the path for everyone, and be
supportive.

• To select contractor and supporting contractors.

• To negotiate and sign agreements with the contractor and


supporting contractors.

• To manage the relationship with the contractor and other


involved organizations built on moral, trust, communica-
tion, and collaboration.

• To tell the contractor what the scope of the software is so


that you and the contractor knows when you are done.

• To manage the budget for the project within the limits set
by the sponsor.

• To coordinate the work for monitoring project progress


and report to the project sponsor.

• To perform reality checks periodically. Take necessary ac-


tions if the answers to the following questions aren’t satis-
factory:

o Are we acquiring the right software?


o Are we acquiring the software right?
o Is the contractor building the right software?
o Is the contractor building the software right?

© Daniel Svennberg 2001 31


4 Roles

• To make necessary trade-offs between delivery time, total


cost, product scope, and product quality if deviation from
the previous estimates and specifications of them should
occur.

• To coordinate work for evaluating and accepting the


product.

• To prepare for post-contract support and maintenance of


the product.

4.2.2 Technical manager


For large projects that are technically complex or with very high
quality demands, such as life-critical software, the acquisition
manager can delegate responsibilities for requirements and
quality issues to a technical manager.

4.2.3 Contract manager


For projects with many involved organizations or projects
where the administrative burden of the contract is high, such as
cost-reimbursement contracts, the acquisition manager can
delegate responsibilities for contract management issues to a
contract manager.

4.2.4 Administrator
The administrative burden depends on the size and type of pro-
ject. One should be careful not to over-administrate. Other
members of the acquisition team perform the tasks of the ad-
ministrator role, but for some projects a specific person can be
assigned this role. The responsibilities of the administrator role
include:

• To maintain configuration and change management on


project documents such as contract, requirements specifi-
cation, project plan, risk list, etc.

32 Software Acquisition Management Guidelines


4 Roles

• To document and file correspondence, meeting minutes,


and decisions.

• To administer payments to the contractor and supporting


contractors.

• To document and file measurements on progress, quality,


requirements changes, etc.

4.3 Assisting roles


The assisting roles are played by different experts and support
contractors that can assist and give advise to the managing,
steering, and executing roles.

4.3.1 Acquisition expert


An acquisition expert can be used for giving advice on how to
plan and perform the acquisition project, what contracting ve-
hicle to select, and to train the acquisition staff in software ac-
quisition management issues.

4.3.2 Technical expert


A technical expert can be used to evaluate the requirements and
to give independent cost and size estimates. Furthermore, the
technical expert can be used to review technical documents and
to audit the technical knowledge and capabilities of the contrac-
tor. The contractor can use technical experts for areas that they
lack competence in.

4.3.3 Domain expert


A domain expert is not necessarily a technical expert but knows
the field in which the software is intended to be used very well.
The domain expert can therefore be used to develop and vali-
date the requirements for the software. Another task for the
domain expert can be to develop user training for the software.

© Daniel Svennberg 2001 33


4 Roles

4.3.4 Legal expert


A legal expert is essential for giving advise on how to write the
contract and to inform on the actual laws that regulates the con-
tract and other matters concerning the project, such as intellec-
tual property rights, patents, licensing, warranties, copyright,
etc. The legal expert can also help during disputes and schisms
with the contractor. The use of a legal expert with software-
specific knowledge is money well spent, especially in interna-
tional projects, according to Salwin (1998).

4.3.5 Translator
A translator is, according to Salwin (1998), a role that can be
played by a person that is good at translating layman’s terms
into technical terms and vice versa. This can be useful in speci-
fying the requirements and transferring the requirements to the
technical staff of the contractor. This person should be skilled at
translating the needs of the customer into something that the
developers can use to build the system.

4.3.6 End user


The end user is the raison d’être of the software, unless of course
the software doesn’t interact directly with a human user. There-
fore, it is imperative to involve the end user on the elicitation of
the requirements for the software, on evaluation of the graphi-
cal user interface, and on evaluation of the functionality of the
software product.

4.3.7 Maintenance and support staff


The maintenance and support staff has the primary tasks of keep-
ing the software running, making upgrades, fixing bugs, and
adding new features as well as giving technical support to the
end users. Therefore, it is important to get suggestions from the
maintenance and support staff on what to require from the
software to make it easier to maintain and troubleshoot. They
should also be involved in evaluating the maintenance and
troubleshooting capabilities of the software and to review the

34 Software Acquisition Management Guidelines


4 Roles

documentation to see if it conveys the necessary information


required to do their jobs.

The maintenance and support staff should be involved early in


the project in order to be able to discuss how the maintenance
and support organization should be organized and to facilitate
a smooth transition of the software once it is delivered to the
maintenance and support organization with maintained con-
figuration management. This role can either belong to the cus-
tomer organization or be outsourced.

4.3.8 Independent verification and validation


An independent verification and validation contractor, IV&V, can
be hired by the customer to make an in-depth technical assess-
ment of the delivered software. This is mainly useful when the
software affects peoples’ health or lives or when large amounts
of money are at stake should the software malfunction. IV&V
naturally increases the costs for the project significantly.

4.3.9 System engineering and technical assistance


When the buyer or seller lacks adequate resources or special-
ized expertise, the use of a supplemental support or services
contractor such as a system engineering and technical assistance
contractor, SETA, can be necessary. The services provided by a
SETA contractor can range from planning the project to testing,
measuring, quality assurance, etc.

The use of SETA contractors should be considered for the fol-


lowing areas, according to Marciniak and Reifer (1990):

• Technical risk.
• Independent testing.
• Management support.
• Integration.

© Daniel Svennberg 2001 35


4 Roles

4.4 Executing roles


The executing roles are played by representatives from the con-
tractors side, i.e. those developing the actual software product.

4.4.1 Contractor
The contractor can be a sole contractor or a prime contractor in
case several collaborating contractors are used. Choose contrac-
tor carefully – the contractor with the lowest bid or most opti-
mistic schedule and budget estimates isn’t always the best
choice. No management practices can make up for having a bad
contractor. Once selected, the contractor should be seen as a
team member and not an adversary. The responsibilities of the
contractor are:

• To develop and deliver the requested software. Should


the project be cancelled, what has been developed up to
that point should be delivered.

• To demonstrate that the sufficient technical and manage-


rial capabilities exists to be able to develop the software.
Should subcontractors or supporting contractors be used,
their competence and capabilities needs to be demon-
strated as well.

• To provide a feasible plan and work breakdown struc-


ture for the project as well as realistic estimates on total
cost and delivery time.

• To show the customer that progress is being made by


holding regular demonstrations of the so far developed
software.

• To make sure that the requirements are understood cor-


rectly.

• To foster an effective, functional, supporting, and col-


laborative culture in the relationship with the customer
built on moral and trust. According to Ran (2000), both

36 Software Acquisition Management Guidelines


4 Roles

parties must realize that they have a mutual responsibil-


ity for the success of the project and should refrain from
taking advantage of each other. The involved parties
should work as a team, jointly solve problems, and re-
frain from blaming each other.

• To promptly and openly inform the customer of unan-


ticipated problems and schedule changes.

• To openly and jointly discuss, review, and mitigate risks.

4.4.2 Collaborating contractor


Should more than one contractor be used, the collaboration be-
tween the contractors needs to be coordinated. This could be
accomplished by assigning one of the contractors as a prime
contractor with the responsibility to coordinate the work of the
collaborating contractors.

4.4.3 Subcontractor
The contractor can use subcontractors to deliver parts of the
software. The competence and capabilities of the subcontractor
needs to be evaluated, and you as a customer should have the
right to decide which subcontractor should be used. If the con-
tractor is using subcontractors, make sure that they are in-
volved as soon as possible in the project.

4.4.4 Supplier
It might be necessary to deal with a supplier of COTS and/or
hardware during the course of the project.

© Daniel Svennberg 2001 37


5 Artifacts

“Everything has been said before, but since nobody listens


we have to keep going back and beginning all over again.”
– A. Gide

In the acquisition processes different documents, reports, deliv-


erables, and other artifacts are used. This chapter contains sug-
gestions for the contents of those artifacts. The artifacts have
been organized into the phase that they are most likely to ap-
pear for the first time of the following four generic software ac-
quisition project phases: inception, solicitation, monitoring, and
finalization. The contents of the artifacts have been assembled
from the different frameworks stated in appendix A unless oth-
erwise stated.

Many of the artifacts are documents, and there are many rea-
sons to document something, such as the following:

• To communicate vital information or to explain how you


think.

• To help remember something.

• To help you think – the process of writing something


down in a lucid and understandable way makes you do a
lot of thinking you wouldn’t have done otherwise, re-
gardless if the document is needed later or not.

38
5 Artifacts

• To legally prove that a transaction has occurred or that a


commitment exists.

There is always a risk of documenting too much, writing things


you won’t need later, or having too much bureaucracy. There-
fore it is necessary for you to evaluate the need of each artifact
and the applicability of each artifact checklist item in order to
customize the artifact for the actual project. Some of the artifacts
can be used as checklists during meetings while others require
some written documentation. It is also important to remember
that some of the artifacts, such as the acquisition plan, the re-
quirements specification, and the risk list are likely to change
during the course of the project. Therefore it is necessary to
have some form of configuration and change management of
these artifacts.

5.1 Inception
During the inception phase the needs of the software are identi-
fied, the requirements are elicited, risks are analyzed, and the
acquisition processes are planned.

5.1.1 Acquisition proposal


The purpose of the acquisition proposal is to present the needs for
and the feasibility of the acquisition, and also what benefits and
risks exists so that an informed decision can be made by the
sponsor on whether to initiate the acquisition project or not. In
addition to the frameworks stated in appendix A, this artifact is
based upon Pressman (1997).

Contents checklist
‰ Describe the organization’s needs for the software prod-
uct and how they are related to the organization’s objec-
tives. What is the background of the needs? What is the
current situation? What justifies the needs for the soft-
ware?

© Daniel Svennberg 2001 39


5 Artifacts

‰ Conceptualize the software product and give operational


scenarios. What will the software have done?
‰ What are the short-term and long-term consequences for
acquiring the software? What are the advantages and
drawbacks?
‰ Analyze the alternatives for acquiring the software:
COTS, MOTS, internal development, outsourced devel-
opment, joint venture development, enhancement of ex-
isting software, or combinations of these alternatives.
What are the risks, costs, and benefits for each alterna-
tive? Also do a market study of the alternatives.
‰ Describe whether the benefits, short-term or long-term,
outweigh the costs for acquiring the software?
‰ Analyze the technical feasibility of the software. Are there
any developmental risks? Can the necessary technical re-
sources such as skilled staff and equipment be made
available? Does the technology exist to develop the soft-
ware or is it better to wait?
‰ Determine any infringement, violation, or liability that
could result from development of the system. Examine
patents, copyrights, and other intellectual property rights.
‰ Describe any external constraints on the project, such as
standards, regulations, other projects, interfacing systems,
etc.
‰ Describe any public relations issues. Is there value in oth-
ers knowing about this? How do we do that?
‰ Describe how the project will be funded and give a time
frame for the project.
‰ State any priorities, assumptions, limitations, and trade-
offs considered.
‰ List potential contractors.
‰ Give a motivated acquisition decision recommendation.

5.1.2 Project specification


The purpose of the project specification is to document the deci-
sions made by the project sponsor regarding project goals,
40 Software Acquisition Management Guidelines
5 Artifacts

bounds, expected outcome, etc. which serves as a decision basis


for the acquisition manager.

Contents checklist
‰ Describe the organization’s needs.
‰ Conceptualize the software product and give operational
scenarios. What will the software have done?
‰ Define the purpose of the project. State the acquisition
project’s goal and vision in specific, measurable, accepted,
realistic, and time-based terms. Specify the expected out-
come of the project in terms of total cost, delivery time,
product scope, and product quality, etc.
‰ Specify the initially identified product requirements and
their relative priorities.
‰ Describe any identified risks.
‰ Specify funding and time bounds.
‰ Identify and give instructions regarding interfaces and
collaborations with other projects.
‰ Identify legal issues, such as intellectual property rights,
etc.
‰ Specify any external constraints, such as standards, regu-
lations, interfacing systems, etc.
‰ Give instructions on security, access to information, and
confidentiality issues.
‰ List potential contractors and/or capabilities that can be
used for identifying potential contractors, such as re-
quired competence, etc.
‰ Describe routines for reporting project status. See the
Project status report on page 56.

5.1.3 Acquisition plan


The purpose of the acquisition plan is to describe how the acqui-
sition project will be managed.

© Daniel Svennberg 2001 41


5 Artifacts

Contents checklist
‰ Describe the project’s background. Why is the project
done? What is its purpose? State the project’s objectives.
‰ State the project funding and schedule bounds.
‰ Define the terminology that will be used in the acquisition
project.
‰ Describe what tools, techniques, and methods will be
used.
‰ Describe any standards, laws, practices, and conventions
that are to be followed.
‰ Determine problem resolution procedures.
‰ Specify documentation requirements and procedures.
‰ Specify configuration and change management proce-
dures.
‰ Describe collaborations and interfaces with external or-
ganizations, support contractors, etc. and how the respon-
sibilities are divided between the involved organizations.
‰ Specify communication and reporting channels. List im-
portant addresses, etc.
‰ Determine acquisition project team inter-group coordina-
tion and division of responsibilities.
‰ Outline a master budget for the different processes.
‰ Outline a master schedule describing the chosen acquisi-
tion life cycle and appropriate milestones.
‰ Specify security, access to information and confidentiality
procedures.
‰ Describe how public relations issues will be handled.
‰ Describe routines for reporting project status
‰ Describe how measurements will be collected, analyzed
and used.

42 Software Acquisition Management Guidelines


5 Artifacts

‰ Plan each of the acquisition processes in chapter 6, Proc-


esses, on page 60:
o Identify the process group members, their respec-
tive responsibilities, and the organization of the
group.
o Describe any collaborations and interfaces with
other groups and organizations.
o Specify the scope and objectives of the process ac-
tivities.
o Determine the working methods of the process
group.
o Specify and schedule the process activities. Deter-
mine who is responsible for each activity.
o Specify the budget for the process activities.
o Describe other resources allocated for the process
activities in form of equipment, tools, materials,
documents, facilities, etc.
o Determine how the process activities will be coordi-
nated and how progress, problems, etc. will be re-
ported and reviewed.

5.1.4 Risk list


The purpose of the risk list is to describe the identified risks,
prioritize the risks and provide contingency and mitigation ap-
proaches for each risk. The risk list should be maintained cur-
rent and used as a tool for communicating risk issues between
the involved parties. Each item on the risk list can contain sev-
eral of the following:

Contents checklist
‰ Risk identification.
‰ Risk classification.
‰ Date of last update.
‰ Risk description, i.e. condition and consequences.
‰ Probability of risk occurring.
© Daniel Svennberg 2001 43
5 Artifacts

‰ Impact of risk occurrence.


‰ Risk exposure, i.e. probability × impact.
‰ Indicators of risk turning into a problem.
‰ Approaches to control, transfer, avoid, minimize, and
mitigate risk.
‰ Name of person responsible for mitigation activities and
date for last implementation of those activities.
‰ Status of indicators and mitigation activities.
‰ Contingency plan, what to do if the risk occurs.
‰ Contingency triggers, what conditions are necessary for
the contingency plan to be implemented.

5.1.5 Requirements specification


The purpose of the requirements specification is to capture and
describe the requirements of the software product and other de-
liverables. The requirements can be described several different
ways such as shall-statements, problem statements, use cases, or
user stories1. The contents checklist is partially based upon Kar
and Bailey (1996).

Contents checklist
‰ State the project goals. Which goal is the most important
one in case a trade-off decision needs to be made?
‰ Describe the software product in general terms; what is
the background for acquiring the software, what is its us-
age domain, what type of product is it, etc.

1 More information about requirements can be found in numerous web


sites and books. See for instance “Managing Software Requirements: A
Unified Approach” by Dean Leffingwell, Don Widrig, and Edward Your-
don (ISBN 0201615932). Information on user stories can be found on:
http://www.extremeprogramming.org/rules/userstories.html.

44 Software Acquisition Management Guidelines


5 Artifacts

‰ Describe the different kinds of users the software will


have. How do the different types of users use the soft-
ware? What will the software have done for the user?
‰ Are there any safety or security restrictions?
‰ Describe the target hardware for the software.
‰ Describe the external interfaces, to humans, hardware and
other software.
‰ Define the terminology used in the specification.
‰ State any assumptions that have been made.
‰ State the functional requirements. How shall the software
handle the different types of input, and what output
should it produce? How should errors be handled?
‰ State the non-functional requirements, in terms of per-
formance, portability, efficiency, reliability, usability,
availability, and maintainability.
‰ Verify and validate each requirement:
o Has the requirement been designated an identifica-
tion for traceability purposes?
o What is the justification and motivation for the re-
quirement?
o How will the fulfillment of the requirement be veri-
fied?
o What other requirements is the requirement related
to?
o Is the requirement necessary? Will a deficiency exist
if the requirement is deleted?
o Is the requirement minimal or can it be divided into
separate requirements?
o Does the requirement state what is required, and not
how it should be met?
o Is the requirement attainable?
o Is the requirement complete or does it need further
amplification?
o Is the requirement consistent with the other re-
quirements in such that it doesn’t contradict any
© Daniel Svennberg 2001 45
5 Artifacts

other requirement, isn’t a duplicate of any other re-


quirement and uses the same terminology as the
other requirements?
o Is the requirement unambiguous?
o Is the requirement written without the use of vague
words, such as “flexible” and “user friendly”?
o Is the requirement verifiable by inspection, analysis,
demonstration, or test and is a verification criterion
stated for each requirement?
‰ Prioritize the requirements.
‰ Describe what risks are connected with each requirement.
‰ Give implementation time estimates for each requirement.
‰ Verify and validate the total set of requirements:
o Is the total set of requirements complete, and
doesn’t need any further amplification?
o Is the total set of requirements consistent, in such
that no duplicate or contradictory requirements ex-
ist, and the same terminology is used for all re-
quirements?
‰ State any installation requirements.
‰ State any requirements regarding manuals and documen-
tation.

5.1.6 Maintenance plan


The purpose of the maintenance plan is to describe how the sup-
port and maintenance organization will be working.

Contents checklist
‰ Identify the members of the support and maintenance or-
ganization, their respective responsibilities, and how they
are organized.
‰ Describe any collaborations and interfaces with other
groups and organizations.

46 Software Acquisition Management Guidelines


5 Artifacts

‰ Describe the resources allocated for maintenance activities


in form of equipment, tools, material, documents, facili-
ties, etc.
‰ Specify allocated funding and budget for maintenance ac-
tivities.
‰ Determine maintenance scope and how the maintenance
activities will be coordinated.
‰ Describe procedures for problem and maintenance report-
ing.
‰ Describe how problem and modification analysis will be
performed.
‰ Describe procedures for modification implementations.
‰ Describe how maintenance activities will be reviewed and
accepted.
‰ State migration and software retirement procedures.
‰ State configuration management activities and proce-
dures.
‰ State the continuation of requirements management ac-
tivities and procedures.
‰ Specify what standards, methods, practices and conven-
tions will be followed.

5.2 Solicitation
During the solicitation phase requests for proposals are sent out
to potential contractors, proposals are received and evaluated,
and a contractor is selected with which a contract is negotiated
and signed.

5.2.1 Request for proposal


The purpose of the request for proposal, RFP, is to communicate
the scope of the work and the contractual conditions to poten-
tial bidders.

© Daniel Svennberg 2001 47


5 Artifacts

Contents checklist
‰ Describe the customer’s company.
‰ State the acquisition project’s goal and vision.
‰ Describe the target domain of the software product.
‰ List the deliverables of the project.
‰ Include the requirements specification (see page 44).
‰ State any issues regarding security, nondisclosure, and
confidentiality.
‰ Specify any support, training, installation, and mainte-
nance requirements.
‰ Give instructions for bidders regarding the proposal such
as: date for reply, number of copies, contents and struc-
ture of proposal, etc. See the Proposal on page 50.
‰ State any requirements regarding development practices,
quality assurance procedures, standards compliance, con-
figuration management, communication and reporting
progress and problems, etc.
‰ Specify the contract type (see page 88).
‰ Cost and schedule estimates.
‰ Describe any investment issues, such as facilities and
equipment.
‰ Specify the customer’s rights to inspect the bidder’s facili-
ties to investigate and evaluate financial position, techni-
cal capability, experience, quality practices, etc.
‰ State any legal issues regarding warranties, licenses, own-
ership rights, copyright, patent, usage rights, and other
intellectual property issues.
‰ Determine use and control of subcontractors and suppli-
ers.
‰ Describe identified risks and issues that need to be dealt
with.
‰ State any other significant contractual terms and condi-
tions, such as constraints. See the Contract on page 51.

48 Software Acquisition Management Guidelines


5 Artifacts

5.2.2 Contractor evaluation criteria


The purpose of the contractor evaluation criteria is to give a set of
criteria against which the bidding contractors’ proposals will be
compared in order to be able to select the most appropriate con-
tractor. Be explicit about the minimum standard for each crite-
rion. The set of criteria can cover the following issues:

Contents checklist
‰ Technical approach, suggested solution, and understand-
ing of the problem.
‰ Proposed mitigation and management of certain risks.
‰ Cost and schedule estimates.
‰ Company and owner history. Any bankruptcies or litiga-
tions? How long has the company been in business? What
previous customers has the company had?
‰ Contractor stability. Is the organization undergoing any
major change, such as changing ownership or moving of-
fice, etc?
‰ Outcome of contractor’s previous projects regarding cost,
schedule, performance, and quality as well as staffing
levels, error rates etc. See Project status metrics on page
100. Comments from customers to previous projects.
‰ Contractor’s financial status. Eventual independent finan-
cial rating.
‰ The staff’s experience, competence, constitution, and per-
formance.
‰ Technical capabilities.
‰ Management qualifications.
‰ Resources: equipment, tools, facilities, etc.
‰ Location.
‰ Adequacy of software development processes and quality
practices.
‰ Previous experience with contractor.

© Daniel Svennberg 2001 49


5 Artifacts

‰ Legal issues regarding warranties, licenses, intellectual


property rights, etc.
‰ Adherence to eventual standards.

5.2.3 Proposal
The purpose of the proposal given by the bidding contractor(s) is
to present their company, capabilities, suggested solution, esti-
mates, advantages, etc.

Contents checklist
‰ Company description. Its background and history.
‰ Previous and current customers. Outcome and data on
previous projects.
‰ Financial position.
‰ Understanding of problem, technical approach, solution
suggestion, description on how the different requirements
will be met.
‰ Technical and managerial capabilities.
‰ Working methods and development processes.
‰ Quality practices.
‰ Resources: equipment, tools, facilities, etc.
‰ Compliance with standards (such as ISO 9000).
‰ Technical and domain experience.
‰ Cost and schedule estimates.
‰ Mitigation and management of risks.
‰ Organization and staffing. Key personnel. The staff’s ex-
perience and constitution.
‰ Contact persons and addresses.
‰ Charges, prices, fees and payment issues.
‰ Use of subcontractors and suppliers.
‰ Needed investments in facilities and tools, etc.
‰ Legal issues regarding warranties, licenses, intellectual
property rights, etc.

50 Software Acquisition Management Guidelines


5 Artifacts

5.2.4 Contract
The purpose of the contract is to define the relationship obliga-
tions and promises that the customer and contractor make to
each other. The contents checklist is based upon Oskarsson (n.d.
a, b), Marciniak and Reifer (1990), and the frameworks in appen-
dix A.

Contents checklist
‰ State the purpose of the contract.
‰ Define the terminology used.
‰ State the scope of work. Refer to the requirements specifica-
tion (see page 44), which should be attached to the con-
tract. Address software and documentation deliverables,
support, installation, and training.
‰ Specify the acceptance procedures and criteria such as the
following: acceptance test completed with a satisfactory
result, installation completed, user training completed, all
deliverables received, correction for errors found a de-
termined period after delivery are corrected. Specify the
quality measures by which the contractor’s work will be
evaluated. Describe what constitutes a satisfactory per-
formance of the contractor.
‰ Specify procedures and conditions for making modifica-
tions or amendments to the scope of work.
‰ Specify who is authorized to make changes in the contract
and answer questions from the contractor. Also specify
contract change control procedures.
‰ Describe payment conditions, considerations, and
amounts. Determine allowable costs, expenses, charges
and their variation. Determine when payments are to be
made linked to deliverables and milestones. Specify con-
cerns regarding taxes, late payments, and interests.
‰ Specify use of incentives for earlier delivery, reduced
costs, significant results and achievements, or for use of
certain “best practices,” etc. See page 88 for more on in-
centives.

© Daniel Svennberg 2001 51


5 Artifacts

‰ Specify any penalties or payment reductions for delays or


for not meeting certain requirements, etc. Ran (2000) says
that you shouldn’t have penalties unless the risk is
shared, i.e. both sides should lose something. Marciniak
and Reifer (1990) state that penalties should only be used
to correct a situation in the project or to prevent a similar
situation from occurring in the future.
‰ Determine allowable royalty and license fees.
‰ State the timeframe of the contract and issues regarding a
final delivery date if that is applicable.
‰ State conditions, provisions and obligations concerning
contract termination. The contractor should deliver all
material and deliverables that are associated with the con-
tract if the contract is terminated and the customer should
pay for the work performed so far.
‰ Describe responsibilities for maintenance undertakings.
‰ State ownership rights, copyright, patent, trade secrets,
usage rights, licenses, and other intellectual property is-
sues.
‰ Describe legal concerns on warranties, representation, in-
fringement, limitation of liability, indemnification, non-
disclosures, etc.
‰ Describe security policies, nondisclosure, access to infor-
mation, and confidentiality issues.
‰ Specify how public relations issues will be handled.
‰ Specify the contractor’s rights and obligations to assign its
duties, partition the work, and use subcontractors.
‰ Determine how and where disputes will be solved, and
whether an arbitrator should be used.
‰ Specify the relationship between the parties in the con-
tract such as joint venture, partnership, employer and
employee relationship, etc.
‰ Describe the commitments by the involved parties and
the division of responsibilities, obligations, and tasks.
‰ Determine the amount and conditions of customer in-
volvement.

52 Software Acquisition Management Guidelines


5 Artifacts

‰ Specify relationships to other parties involved in the pro-


ject, such as IV&V and SETA.
‰ Specify use and replacement of key personnel.
‰ Specify use of facilities and equipment. Also determine
any investment issues regarding facilities and equipment.
‰ Establish means for monitoring progress, through WBS,
short iterations, milestones, or other means.
‰ Specify the allowable requirements change rate (for fixed
price contracts).
‰ Specify procedures for reporting problems and progress.
Determine what status metrics should be reported by the
contractor and how often. See the Project status metrics on
page 100.
‰ Specify any requirements on the contractor’s software de-
velopment, management and quality assurance processes.
‰ Specify any requirements on a software development
plan for the project and a latest date to receive the plan
from the contractor and what it should contain.
‰ Specify any requirements on the use of COTS or hard-
ware or any restrictions on what suppliers to use.
‰ Determine the customer’s inspection rights and proce-
dures for audits.
‰ Signatures.

5.3 Monitoring
During the monitoring phase the progress and performance of
the contractor is monitored and the software product is evalu-
ated on a regular basis.

5.3.1 Artifacts from the contractor


During the monitoring phase some of the communication from
the contractor can consist of the following artifacts, depending
on the actual project:

© Daniel Svennberg 2001 53


5 Artifacts

• Project plans.
• Test reports.
• Problem reports.
• Payment requests.
• Progress and status reports.
• Change requests.
• Quality reports.

5.3.2 Product evaluation plan


The purpose of the product evaluation plan is to specify the objec-
tives, scope, and activities of the different product evaluations.
See Product evaluation approaches on page 109 for a list of differ-
ent kinds of evaluations that can be performed.

Contents checklist
‰ Specify who will participate in the evaluation and the re-
sponsibilities of each participant.
‰ Describe the objectives and scope of the evaluation.
‰ Give instructions for preparing for, carrying out, and
evaluating the results of the product evaluation.
‰ Give a schedule for the evaluation.
‰ Specify what resources are required regarding equipment,
tools, materials, documents, facilities, etc.
‰ Determine the test configuration, technical environment,
and other prerequisite conditions.
‰ Describe each test case in the evaluation. What require-
ments are addressed during each test case? Specify input
and expected results for each test case.
‰ Describe the coverage of the test cases in the evaluation.
For instance, for an acceptance test it should be shown
that all implemented requirements are covered by the test
cases in the evaluation.

54 Software Acquisition Management Guidelines


5 Artifacts

‰ State criteria for evaluating and reporting evaluation re-


sults. See the Product evaluation report on page 55.

5.3.3 Product evaluation report


The purpose of the product evaluation report is to document the
outcomes of a performed product evaluation. See Product
evaluation approaches on page 109 for a list of different kinds of
evaluations that can be performed.

Contents checklist
‰ Describe the objectives and scope of the evaluation.
‰ State the time for the evaluation, the participants and
their responsibilities.
‰ Identify the test configuration, technical environment,
and other prerequisite conditions.
‰ Describe any special considerations, simplifications, and
assumptions.
‰ List the outcomes of each test case.
‰ Determine whether the coverage of the test cases in the
evaluation was sufficient or not.
‰ Summarize all errors and problems encountered.
‰ Analyze the most common errors and identify trends.
‰ State how the errors and problems will be followed-up.
‰ Give recommendations for decisions, such as changes to
requirements, acceptance, etc.

5.3.4 Change request


The purpose of the change request is to provide motivation and
relevant information to enable an informed decision on whether
to approve a change to requirements, contract or plans.

Contents checklist
‰ State what should be changed.
‰ State who originated the request.
© Daniel Svennberg 2001 55
5 Artifacts

‰ Give a justification for the proposed change.


‰ State the priority of the change request.
‰ Describe any assumptions and constraints that affect the
change request.
‰ State the impact on cost, time, quality, scope, and risks.

5.3.5 Project status report


The purpose of the project status report is to summarize the
status of the project so that it can be used as a basis for a deci-
sion on whether to continue the project or cancel it.

Contents checklist
‰ List the status of the risks in the risk list. Any changes?
Any problems that need to be dealt with?
‰ Report results from product evaluations and the contrac-
tor’s testing efforts.
‰ Analyze the technical feasibility of the contractor’s ap-
proach. State any technical problems or changes in the
technical approach.
‰ State any changes to the requirements.
‰ Describe the available resources regarding equipment,
tools, materials, documents, facilities, etc. Are the origi-
nally planned resources available? Are the resources ap-
propriate?
‰ List measurements regarding schedule and budget, etc.
See section 6.9.1, Project status metrics, on page 100.
‰ Identify and analyze trends regarding schedule and
budget, etc. Is the project on schedule? Forecast delivery
date. How much of the funding have been expended? Is
the current funding appropriate? Will the project reach its
budget goals?
‰ List the results achieved since the last report. What mile-
stones have been accomplished?
‰ Describe ongoing activities.

56 Software Acquisition Management Guidelines


5 Artifacts

‰ What actions need to be taken within the project? What


actions need to be taken by the sponsor or steering group?
‰ Does the benefits, short-term or long-term, outweigh the
predicted remaining costs for acquiring the software?
‰ Give conclusions and recommendations and relate them
to project objectives. Should the project continue or be
canceled? Any changes to the project direction?

5.4 Finalization
During the finalization phase the product and other deliverables
are delivered to the support and maintenance organization and
a post partum review is held.

5.4.1 Deliverables
The deliverables received from the contractor could consist of
some of the following artifacts, depending on the actual project:

• Software executables.
• Source code.
• Technical documentation.
• User training material.
• Rest list – lists requirements not implemented and known
errors.
• User manual.
• Support publications.

5.4.2 Post partum plan


The purpose of the post partum plan is to describe how the post
partum will be conducted. The plan is based upon Kerth (n.d.).

© Daniel Svennberg 2001 57


5 Artifacts

Contents checklist
‰ Determine who should attend the post partum. Preferably
representatives from all involved organizations and roles
should participate.
‰ Determine when to hold the post partum. Preferably
within 7-14 days after project completion.
‰ Select an appropriate location for the post partum. Pref-
erably away from the office.
‰ Determine the length of the post partum. 2-4 hours may
be sufficient for minimal projects while 3 days could be
needed for robust projects.
‰ Determine the rules for the post partum in order to create
a non-threatening environment.

5.4.3 Post partum


The purpose of the post partum is to document, analyze, and
package experiences and wisdoms for reference in future acqui-
sition projects.

Contents checklist
‰ Give the background for the project and the time of
events in the project.
‰ Document the customization factors values (see the
Customization factors on page 21) for the project.
‰ Compare project goals to actual outcome regarding the
deliverables, time schedule, costs, etc. Indicate deviations
and the reasons for them.
‰ Document what processes and methods were used and
how the project was organized. Document how the pro-
ject was customized.
‰ Document special difficulties and measures taken and the
effect of these measures.
‰ Document the performance of the contractor and other
involved organizations – their strengths and weaknesses.
‰ If earned value was used – document SPI, CPI, etc.

58 Software Acquisition Management Guidelines


5 Artifacts

‰ Document what development methodology and working


methods were used by the contractor.
‰ Document what risks occurred and what measures were
taken regarding risks.
‰ Document experiences and observations.
‰ Document what activities/techniques were considered
key to the success of the project.
‰ Propose improvements on the acquisition processes and
other things.

© Daniel Svennberg 2001 59


6 Processes

“I have it anecdotally that the most commonly used


methodology in the world is chaos.”
– Ron Jeffries

“A good process will tell you to do what a good manager


and staff would do anyway.”
– Tom DeMarco

This chapter describes the different acquisition processes. The


contents of the processes have been assembled and adapted
from the different frameworks stated in appendix A unless oth-
erwise stated.

6.1 Overview
The different aspects of managing a software acquisition project
have been divided into the following eleven processes:

• Project steering
• Project management
• Planning and organizing
• Acquisition training
• Requirements management
• Risk management

60
6 Processes

• Contractor selection
• Contract management
• Product evaluation
• Transition to support
• Post partum

Each process contain suggestions on what activities to perform


for the different project customization levels suggested in chap-
ter 3, Customization, on page 20. The process template in the ta-
ble below explains how to interpret the process descriptions.
Keep in mind that the activities given in the different customi-
zation levels are only suggestions. When planning for an acqui-
sition project you can use the suggestions as a basis but you
have to remain flexible and adapt to the specific circumstances
of the actual project. For instance, if you have a project where
all of the factors described in chapter 3, Customization, on page
20, are suggesting minimal processes except for requirements
volatility, which suggest robust processes, then you have to
consider having most processes minimal, but a higher customi-
zation level for requirements management, and as a conse-
quence a higher customization level on product evaluation, or
other adaptations that fit your needs. And remember: following
a process requires discipline.

Process name

Possible start condition for the Possible end condition for the
process. process.

Possible inputs to the process. Possible outputs from the


process.

Process group

Roles that could be considered as participants in the process


group. The roles that are in bold face are roles that should par-
ticipate as a recommended minimum.

© Daniel Svennberg 2001 61


6 Processes

Minimal process activities

‰ Activities suitable to perform in a project with minimal pro-


ject characteristics.

Additional controlled process activities

‰ Additional activities suitable to perform in a project with


controlled project characteristics.

Additional robust process activities

‰ Additional activities suitable to perform in a project with ro-


bust project characteristics.

Table 3. Process template explaining the elements of the process de-


scriptions.

As an illustration, Figure 4 below depicts at what point in time


of the acquisition lifecycle the processes occur.

Project steering
Project management
Planning and organizing
Acquisition training
Requirements management
Risk management
Contractor selection
Contract management
Product evaluation
Transition to support
Post partum
Inception Solicitation Monitoring Finalization

Figure 4. Timeline of the acquisition processes.

Figure 5 below shows how the processes are related to the dif-
ferent roles and major artifacts. The numbers in the figure refer
to the following example of a sequence of major activities from
all process customization levels:

62 Software Acquisition Management Guidelines


6 Processes

Acquisition
proposal
Sponsor Domain expert Technical
8 7 expert
1
Project Risk Requirements
steering management management 15
2 18 End user

Project Project status Risk list Requirements


specification report specification Change
request
IV&V
Project 3
management 12 Product
Acquisition evaluation
manager Translator

Reports and Evaluation Evaluation


measurements report plan
Administrator 16
14
13 Deliverables
Contract Reports,
management requests, etc.
Contractor
17
19
11

Contract Proposal RFP Transition to


9 support

10
Contractor Maintenance
Legal expert evaluation plan
Contractor selection
criteria
5

Acquisition Planning and Post partum


6 training organizing
Maintenance &
support staff
Acquisition Post partum Post
4 plan plan partum

Acquisition 20
expert
Figure 5. Overview of the software acquisition processes, major roles
and artifacts.

© Daniel Svennberg 2001 63


6 Processes

1. The sponsor initiates the project steering process. The needs


of the organization, and other project concerns are elicited,
analyzed, and documented in the acquisition proposal,
which serves as a decision basis.

2. If the sponsor decides to initiate the project, the project goal,


funding and time bounds, etc. are specified in the project
specification.

3. An acquisition manager is appointed.

4. The acquisition manager customizes, plans, and organizes


the acquisition project based upon the project specification,
possibly with the assistance of an acquisition expert and
others. The plan is documented in the acquisition plan.

5. The maintenance and support organization is identified and


the maintenance plan is drafted.

6. The acquisition team performs the necessary acquisition


training activities.

7. The requirements management process is initiated. The


product requirements are elicited with the help from end
users, maintenance and support staff, translator, domain
experts, and technical experts, etc. The requirements are
documented in the requirements specification.

8. The risk management process is initiated. Identified risks


are documented in the risk list by the acquisition manager
and others.

9. The contractor selection process is initiated with the contrac-


tor evaluation criteria being written followed by writing and
sending out request for proposals to potential contractors.
The acquisition manager is responsible for this with assis-
tance from acquisition expert, legal expert, etc.

10. The proposals from the bidding contractors along with data
gathered from audits and background checks are compared

64 Software Acquisition Management Guidelines


6 Processes

with the contractor evaluation criteria and the most appro-


priate contractor is selected.

11. The contract is negotiated, reviewed and signed between the


acquisition manager and the manager on the contractor’s
side, with the legal expert advising the proceedings.

12. A joint review and discussion of the requirements specifica-


tion and each of the parties risk lists is performed prior to
commencing development.

13. During the development of the product the contractor deliv-


ers project plans, test reports, progress reports, payment re-
quests etc. and the software acquisition team gives feedback
and takes necessary action.

14. The contractor submits deliverables for evaluation by the


software acquisition team on a regular basis. Feedback is
given to the contractor through an evaluation report. Possi-
bly the software is evaluated by and independent verifica-
tion and validation organization.

15. Changes to requirements are handled and possibly the con-


tract is renegotiated.

16. The acquisition team and the contractor collaborate to solve


problems, and jointly reviews project status, plans, risk lists,
etc. on a regular basis.

17. For lengthier projects it might be necessary to audit the con-


tractor with the assistance of a technical expert, etc.

18. The acquisition manager regularly briefs the project spon-


sors on the status of the project through a project status re-
port. The sponsors use this as a basis for a decision to con-
tinue or cancel the project.

19. After the final evaluation and acceptance of the software


product it is transitioned to the maintenance and support

© Daniel Svennberg 2001 65


6 Processes

organization. The contract is settled, open items are resolved


and final payment is made.

20. A post partum is planned and performed. Celebrate a pro-


ject well done and document experiences and lessons
learned. Remember not to make it a blame session.

A more detailed description of the different activities that can


be performed in a software acquisition project are given in the
process descriptions that follow.

6.2 Project steering


The purpose of the project steering process is to give the project
sponsors control over the project. It is a two-phase process,
where the first phase consists of eliciting and analyzing the
needs of the organization, set a goal for the project and take a
decision on whether to initiate the project or not. The second
phase consists of evaluating the status of the project on a regu-
lar basis, at least for lengthier projects, in order to determine
whether to continue the project or not.

Project steering process

Starts when the idea of the Ends when the project is ter-
project is established. minated.

Input: Organizational needs Output: Project specification


and objectives, Project status
report

Project steering group

The participants to consider for this process should be those


who are authorized to make decisions about the project’s exis-
tence and those who can give advice on different project issues
such as the following roles: sponsor, collaborating customer,
product manager, administrator, acquisition expert, technical
expert, domain expert, legal expert, translator, end user, main-

66 Software Acquisition Management Guidelines


6 Processes

tenance and support staff, and system engineering and techni-


cal assistance. Make sure to involve the necessary organizations
as early as possible in the project.

Minimal process activities

‰ Elicit the organizational needs.


Review the organization’s objectives. Elicit, analyze, and
conceptualize the needs and expectations of the organiza-
tion. Determine the scope and qualities the software product
must possess to achieve the organization’s objectives.

‰ Analyze and select acquisition type.


Analyze the advantages, drawbacks, risks, costs, and bene-
fits for the following alternatives: COTS, MOTS, internal de-
velopment, outsourced development, joint venture devel-
opment, or combinations thereof, and enhancement of exist-
ing software. Select the most suitable alternative. COTS
should be the primary candidate, then MOTS, and only as a
final solution fully developed software should be considered
for most acquisition projects.

‰ Investigate intellectual property rights.


Determine if there are any conflicts regarding patents, copy-
rights, trademarks, and registered names, etc.

‰ Decide to initiate project or not.


Is this the right time to acquire the product? Is the product
really needed or is it just cementing the old way of working?
Are there any foreseeable risks that are very likely to termi-
nate the project? What are the risks for not acquiring the
software? Does the benefits, short-term or long-term, out-
weigh the costs for acquiring the software?

‰ Determine project parameters.


Determine the scope of work of the project. Who will pro-
vide software support? Identify potential contractors or de-
velop a list of capabilities that can be used for identifying po-
tential contractors. Identify the extent of the responsibilities
for the acquiring organization and contracting organization

© Daniel Svennberg 2001 67


6 Processes

respectively. Identify interfaces and collaborations with


other projects and organizations. Define the expected out-
come; roughly estimate cost limits, time schedule, etc. See
the Project specification on page 40.

‰ Establish and communicate a project goal.


Define the purpose of the project and determine what is con-
sidered a successful outcome of the project. Inform all in-
volved parties.

‰ Appoint an acquisition manager.


Establish project responsibility, authority, and accountabil-
ity.

‰ Decide to cancel or continue the project based on the status


of the project or external factors.
For minimal, and thus short projects, this decision should be
taken in the beginning of the development (about 15-20%
into the project). For controlled and robust project this deci-
sion needs to be taken on regular intervals. Remember that
termination can be success. Document and communicate the
decision. The decision should be well motivated and given
in a timely manner.

Additional controlled process activities

‰ Investigate constraints and consequences.


What short-term and long-term consequences (advantages
and drawbacks) of the acquisition are there for different in-
dividuals and organizations? Identify external constraints on
the project such as standards, regulations, interfacing sys-
tems, technical constraints, etc.

‰ Prepare an acquisition proposal.


See the Acquisition proposal on page 39. Use it as a basis for
the decision on whether to start the project or not.

‰ Document the project parameters.


Write a Project specification (see page 40).

68 Software Acquisition Management Guidelines


6 Processes

‰ Evaluate the status of the project on a regular basis.


Review the Project status report (see page 56). Evaluate the
feasibility to achieve project goals with available resources
and constraints. Estimate the tasks and resources necessary
to complete the project. Has the business objectives changed?
Has the market changed? Has other external constraints
changed? Is the project progressing satisfactory? Are there
any delays or budget overruns? Are appropriate resources
allocated? Use the evaluation as a basis for decisions on
whether to continue the project or not.

Additional robust process activities

‰ Analyze the technical feasibility of the software.


Identify technical and developmental risks. Investigate
whether the necessary technical resources can be made
available, etc.

6.3 Project management


The purpose of the project management process is to perform the
project according to plan, replan if necessary, and handle issues
not planned for.

Project management process

Starts when the acquisition Ends when the acquisition


manager has been appointed. project is complete.

Input: Acquisition plan, Risk Output: Project status report


list, Measurements, Product
evaluation report

Project management group

The participants to consider for this process should be those


with the responsibility and authority to manage and administer
© Daniel Svennberg 2001 69
6 Processes

the project such as the following roles: acquisition manager,


technical manager, contract manager, and administrator.

Minimal process activities

‰ Customize, plan, and organize the acquisition project.


Develop an acquisition plan. See the Planning and organizing
process on page 72.

‰ Train the acquisition staff.


Make sure that the people involved in the project are trained
in how to perform the different process activities. See the
Acquisition training process on page 74.

‰ Review project feasibility.


Ensure the feasibility of the project regarding adequate and
timely resources and that the goal is achievable with the
given scope, funding, and schedule. Remember that you can
have fixed scope, product quality, delivery date, or total cost
but not all four. You may therefore need to make trade-offs
to achieve the most important goal of the project. Discuss
any feasibility issues with the project sponsors.

‰ Regularly review the work of the acquisition process


groups.
Regularly review what has been done, and decide on next
actions for the different process groups. Follow up on previ-
ous decisions.

‰ Regularly review the work of the contractor(s).


Determine how measurements are to be assembled, reported
and analyzed. Gather measurements on progress, resource
usage, cost, quality, and schedule, etc. and compare to plans
and specifications. Identify trends. Evaluate the status of the
project and take necessary action. See the Contract manage-
ment process on page 96.

‰ Report project status.


The acquisition manager should at agreed upon points in
time assemble and report the status of the project to the

70 Software Acquisition Management Guidelines


6 Processes

steering group. See the Project status report on page 56.

‰ Perform team-building and social activities.


Perform suitable project kick-off and other social or team-
building activities. Recognize the efforts of the staff mem-
bers. Make the project fun and exciting. Don’t over-use over-
time.

Additional controlled process activities

‰ Identify and coordinate activities with other projects and


sign contracts with supporting organizations.

‰ Establish problem resolution procedures.


Any problems discovered during the project should be duly
reported, investigated, and solved. The impact of any
changes due to problems should be determined, controlled
and monitored. Establish means for resolving the issues that
will arise during the project, such as consensus building, ne-
gotiating, voting, brainstorming, etc. Inform the involved
parties on the problems status. Document and track status of
problems until resolved. Resolve both problem and cause.
Evaluate the problem resolutions: Were they correctly im-
plemented? Were new problems introduced?

‰ Establish quality assurance and configuration management


procedures.
Establish a configuration management methodology. Iden-
tify the items that should be placed under configuration
management such as plans, agreements and specifications.
Control and track changes to those items. Identify and stan-
dardize documents. Control the release and delivery of
documents, correspondence, and other items in the project.
Verify that the project documents are adequate, complete,
and consistent by peer review or other means. Maintain
plans and other documents current throughout the project as
re-planning occurs, issues are resolved, requirements are
changed, and risks are mitigated or discovered.

© Daniel Svennberg 2001 71


6 Processes

Additional robust process activities

‰ Perform in-depth problem analysis.


Analyze, categorize, and prioritize problems and identify
root causes. Detect trends. Develop a system for identifying,
recording, and tracking problems that occur during the pro-
ject. Update the risk list.

‰ Document lessons learned.


For long-term projects it might be of value to prepare inter-
mediate reports that documents observations, experiences
and lessons learned and gives suggestions on improvements
for the rest of the project and future projects.

6.4 Planning and organizing


The purpose of the planning and organizing process is to custom-
ize the acquisition processes for the project and make plans for
each process accordingly.

Planning and organizing process

Starts when the acquisition Ends when the planning ac-


manager has been appointed. tivities are finished.

Input: Project specification Output: Acquisition plan

Planning and organizing group

The participants to consider for this process should be those re-


sponsible for performing the project and those who can give in-
put to and assistance on planning and organizing issues such as
the following roles: sponsor, collaborating customer, product
manager, acquisition manager, technical manager, contract
manager, administrator, acquisition expert, system engineering
technical assistance, contractor, collaborating contractor, and
subcontractor.

72 Software Acquisition Management Guidelines


6 Processes

Minimal process activities

‰ Customize the acquisition processes.


Review the Project specification (See page 40). Get input from
involved organizations. Analyze the characteristics of the
project and select an appropriate customization level, see
Customization on page 20. Review stored information on how
previous acquisition projects were managed. Select suitable
working methods and make adaptations to suit the actual
project.

‰ Form groups for all acquisition processes.


Determine what roles are necessary for the project. Deter-
mine the necessary skills for those roles. Employ the best and
most competent staff as possible. Form groups for the differ-
ent acquisition processes. Assign responsibility, authority,
and accountability for the different project activities.

‰ Develop an acquisition plan.


Develop an appropriate Acquisition plan (see page 41). As a
minimum the acquisition plan should cover the following:

o Define the acquisition life cycle, activities, and mile-


stones to be used in the project. Address all acquisition
processes as appropriate: project steering, project man-
agement, planning and organizing, acquisition training,
requirements management, risk management, contractor
selection, contract management, product evaluation,
transition to support, and post partum.

o Schedule and budget the project activities.

o Document how the project team is organized.

o Determine resource allocation. Resources to consider are:


tools, equipment, facilities, materials, documents, etc.

‰ Review the acquisition plan.


All affected parties should review the plan.

© Daniel Svennberg 2001 73


6 Processes

Additional controlled process activities

‰ Determine what type of help and services need to be ob-


tained from external organizations besides the contractor.

‰ Organize and establish commitments for all involved ex-


ternal supporting organizations and individuals.
Assign responsibility, authority, and accountability for the
delegated project activities.

‰ Develop an acquisition plan.


A more elaborate and detailed plan should be made for con-
trolled and robust processes, i.e. most of the items on the
Acquisition plan checklist on page 41 should be covered.

Additional robust process activities

‰ None.

6.5 Acquisition training


The purpose of the acquisition training process is to determine
and develop the skills needed and the skills that the team
members have in order to be able to perform their roles appro-
priately.

Acquisition training process

Starts when the acquisition Ends when the training needs


team has been assembled. have been met.

Input: Output:

Acquisition training group

The participants to consider for this process should be those


who can determine training needs of the staff and provide or

74 Software Acquisition Management Guidelines


6 Processes

acquire the required training and those in need of training.


Roles to consider for the acquisition training group are: acquisi-
tion manager, technical manager, contract manager, administra-
tor, acquisition expert, and all other roles in need of training.

Minimal process activities

‰ Determine skills and knowledge needs.


Determine the skills and knowledge that is required to per-
form each role in the acquisition team.

‰ Determine training needs.


Determine the current skills and knowledge of the persons
that will be designated to the different roles.

‰ Determine and prioritize what training activities are essen-


tial for the project.
Develop individual training plans.

‰ Determine who will provide the training and how the


training will be performed.

‰ Develop or acquire training material.


Develop or acquire the necessary training material from
within the organization or from external sources. Review the
quality of the training courses and material.

‰ Perform the training activities.

‰ Determine if the performed training was sufficient or if


more training is required.

Additional controlled process activities

‰ Maintain training records.


Maintain training records such as who has completed which
training.

Additional robust process activities

© Daniel Svennberg 2001 75


6 Processes

‰ Spread knowledge.
For long-term projects it might be of value to arrange semi-
nars where different team members share their knowledge to
the other team members.

6.6 Requirements management


The purpose of the requirements management process is to elicit,
review, and manage change of the product requirements. Apart
from the frameworks in appendix A, this process has partially
been based upon Christel and Kang (1992).

Requirements management process

Starts when the needs for the Ends when the software has
software have been identified. been transitioned to the sup-
port and maintenance organi-
zation.

Input: Project specification, Output: Requirements specifi-


Change requests cation

Requirements management group

The participants to consider for this process should be those


whose needs are to be fulfilled by the software and those who
will maintain and support the software. Other participants
should be roles that can give advise and valuable input on how
to formulate the requirements. Thus the following roles should
be considered: sponsor, collaborating customer, product man-
ager, acquisition manager, technical manager, contract man-
ager, administrator, acquisition expert, technical expert, domain
expert, legal expert, translator, end user, maintenance and
support staff, independent verification and validation, system
engineering and technical assistance, contractor, collaborating
contractor, and subcontractor roles.

76 Software Acquisition Management Guidelines


6 Processes

Minimal process activities

‰ Elicit the requirements.


Analyze the needs, conceptualizations, and requirements
given in the project specification. Determine if the require-
ments should be elicited mainly before or after contract
award. Define and analyze the operational and problem con-
text. Analyze similar systems. Define operational modes,
goals, and scenarios. Get requirements wish lists from suit-
able organizations, groups and individuals, especially end
users and maintainers. Categorize and integrate the re-
quirements. Consider possible future expansions of the sys-
tem when eliciting the requirements.

‰ Document the requirements.


Document the requirements in a suitable form: use cases,
user stories, or requirements statements. See the
Requirements specification on page 44.

‰ Prioritize the requirements.


Identify the most critical requirements that have a strong in-
fluence on usability, functionality, efficiency, reliability,
portability, maintainability, other quality factors, perform-
ance, cost, schedule, or risk and prioritize accordingly.

‰ Establish evaluation criteria for each requirement.

‰ Jointly review the requirements and evaluation criteria


with the contractor prior to development start.

‰ Review and approve change requests.


Review and appraise all requirements change requests on
what impact they will have on schedule, scope, cost, quality,
and risks. Avoid creeping scope, goldplating, and other
common problems. Document and inform all parties about
all approved requirements changes.

Additional controlled process activities

‰ Formally validate and verify the requirements prior to the


© Daniel Svennberg 2001 77
6 Processes

joint review with the contractor.


Analyze the feasibility of the requirements and validate
them against the needs of the organization. Verify that the
requirements reflect end user needs, are verifiable, neces-
sary, concise, implementation free, minimal, attainable,
complete, consistent, and unambiguous. Verify especially
that the requirements related to safety, security, and privacy
protection are correct as shown by suitably rigorous meth-
ods.

‰ Measure the requirements change rate.


Baseline the requirements before development start and
measure the rate of changes to the requirements. The target
value for total requirements growth in function points or
equivalent is ≤1% per month according to Brown (1998).

Additional robust process activities

‰ Get external review on critical requirements.


Let external experts review the requirements concerning
safety, security, and privacy protection.

6.7 Risk management


The purpose of the risk management process is to identify, man-
age, mitigate, monitor, and handle the risks of the acquisition
project.

Risk management process

Starts as soon as the need for Ends when the acquisition


acquiring the software is iden- project is complete.
tified.

Input: Project specification Output: Risk list

78 Software Acquisition Management Guidelines


6 Processes

Risk management group

The participants to consider for this process should be those


who are in charge of managing the project and those who can
give input on possible risks such as the following roles: spon-
sor, collaborating customer, product manager, acquisition
manager, technical manager, contract manager, administrator,
acquisition expert, technical expert, domain expert, legal expert,
end user, maintenance and support staff, independent verifica-
tion and validation, system engineering and technical assis-
tance, contractor, collaborating contractor, and subcontractor.

Minimal process activities

‰ Encourage risk identification and management.


Encourage and reward project-wide participation in the
identification and mitigation of risks and foster a non-
threatening environment in which the involved parties can
discuss risks.

‰ Identify risks and develop a project risk list.


Identify the project risks, see section 6.7.1, Risk identification,
on page 80. Develop a risk list that contains all identified
risks, their estimated probability and impact. Identify
“show-stoppers.” See the Risk list on page 43.

‰ Analyze the developed risk list and decide which risks to


take action on.

‰ Conduct joint risk reviews.


Review risks jointly with all affected parties. Establish a
common view of risk scenarios and a mutual understanding
and expectation of what can go wrong in the project.

Additional controlled process activities

‰ Develop a more elaborate risk list.


Add mitigation and contingency plans and other informa-
tion as deemed necessary for all risks in the risk list. Most
items in the Risk list check list on page 43 should be covered
© Daniel Svennberg 2001 79
6 Processes

for controlled and robust processes.

‰ Continuously track risk status.


Track risk status and update and maintain the risk list con-
tinuously throughout the project. Track mitigation activities
and reassess risks on the risk list on a regular basis. Analyze,
track, and control risks until mitigated.

‰ Establish means for reporting and communicating risk in-


formation throughout the project team.

Additional robust process activities

‰ None.

6.7.1 Risk identification


A risk can be defined as a potential problem. The risks for a
project can be identified via brainstorming, reviewing previous
projects, reviewing checklists and taxonomies, etc. The factors
to look for are those that are likely to impact the project objec-
tives of scope, quality, time, and cost according to Wideman
(1992). There are several taxonomies and checklists to choose
from. The table below has been assembled and adapted from
Carr, et al. (1993), D.I.R. (2000), and Wideman (1992). It can be
used as a basis for risk identification and classification. You can
also have a look at the problems listed in section 2.5, Common
problems, on page 13. It is recommended to build a company
risk database.

A. External risks
Risk category Problem examples Possible impact
A.1 Natural hazards Storm, flood, earth- Destruction of equip-
and accidents quake, fire, etc. ment and stored data.
A.2 Crime Vandalism, sabotage, Destruction and theft
theft, hacking, crack- of property and data.
ing, etc.
A.3 Political unrest Revolution, war, etc. Cancellation of project.

80 Software Acquisition Management Guidelines


6 Processes

Risk category Problem examples Possible impact


A.4 Regulatory Unanticipated gov- Increased total cost.
ernment intervention Cancellation of project.
in product require-
ments, pricing, export
regulations, etc.
A.5 Economical factors Currency changes, in- Increased total cost.
flation, taxation
changes, etc.
A.6 Indirect effects Unwanted social or Lawsuits, loss of com-
environmental indirect pany credibility, etc.
effects from the use of
the software.
B. Project and contract management risks
Risk category Problem examples Possible impact
B.1 Project manage- Insufficient support Increased total cost.
ment from corporate man- Increased delivery
agement, poor com- time.
munication, vague Staff morale is low-
goals and direction. ered.
Cancellation of project.
B.2 Acquisition staff Unavailability of suffi- Increased total cost.
ciently competent staff. Increased delivery
Fragmentation. time.
Cancellation of project.
B.3 Organizational The product concept Product gives no value
needs and needs of the or- to the end user. Benefit
ganization has not to develop product is
been sufficiently less than costs.
investigated.
B.4 Estimation Substantial Increased total cost.
underestimation of Increased delivery
required time and time.
budget. Cancellation of project.
B.5 Contractor selec- Selection of a contrac- Increased total cost.
tion tor that is inexperi- Increased delivery
enced, low- time.
performing, non- Poor quality.
cooperative, or in risk Cancellation of project.
of bankruptcy.
B.6 Contract Inappropriate contract Friction between par-
– not addressing neces- ties.
sary issues, or with too Increased total cost.
restricting or impeding Increased delivery
clauses. time.

© Daniel Svennberg 2001 81


6 Processes

Risk category Problem examples Possible impact


B.7 Contract manage- Over-administration. Increased total cost.
ment Increased delivery
time.
Staff morale is low-
ered.
B.8 Relationship be- Friction, the parties do Increased total cost.
tween parties not communicate or Increased delivery
work well together. time.
Poor quality.
Cancellation of project.
B.9 Coordination be- Poor coordination be- Rework.
tween parties tween parties regard- Increased total cost.
ing configuration Increased delivery
changes, division of time.
work, etc. Staff morale is low-
ered.
B.10 Funding Unstable or insufficient Cancellation of project.
funding.
B.11 Schedule Time pressure – project Poor quality.
is rushed. Product doesn’t meet
end user’s needs.
B.12 Project scale and Large-scale develop- Increased total cost.
complexity ment of a complex sys- Increased delivery
tem with many in- time.
volved organizations.
B.13 Internal politics Conflicting political Poor quality.
agendas between in- Increased total cost.
ternal groups and/or Increased delivery
contracted groups. time.
Cancellation of project.
B.14 Suppliers of Suppliers not deliver- Increased delivery
hardware and software ing as specified or not time.
supporting critical sys-
tem components in a
timely manner.
B.15 Legal risks Lawsuits. Force Ma- Increased total cost.
jeure. Increased delivery
time.
Cancellation of project.
C. Technical and product development risks
Risk category Problem examples Possible impact
C.1 Requirements The requirements are Rework
changing frequently Product doesn’t meet
due to incompleteness, end user’s needs.

82 Software Acquisition Management Guidelines


6 Processes

Risk category Problem examples Possible impact


unclarity, or invalidity. Increased total cost.
Scope creep. Goldplat- Increased delivery
ing. time.
Underestimation of to-
tal effort required.
C.2 Technical feasibil- A requirement is im- Product doesn’t meet
ity possible to implement end user’s needs.
or a set of require- Increased total cost.
ments are conflicting Increased delivery
and cannot be imple- time.
mented simultane- Cancellation of project.
ously. The design is
more difficult to im-
plement than antici-
pated.
C.3 Product quality The delivered product Damage to persons or
is buggy and crashes expensive equipment.
frequently. Lawsuits.
C.4 Project status indi- Lack of indicators. In- Not knowing where
cators adequate indicators. project is heading. Di-
Reporting erroneous minished ability to cor-
data. Reports are not rect the situation early
given on a regular ba- on.
sis.
C.5 Development Inexperienced manag- Friction between par-
management ers. Inefficient organi- ties.
zation. Ineffective Product doesn’t meet
communications. No end user’s needs.
control over schedule Poor quality.
and budget. Poor co- Increased total cost.
operation between de- Increased delivery
velopment teams and time.
with external organiza-
tions such as other con-
tractors.
C.6 Development staff Inexperienced staff. Product doesn’t meet
Low moral, enthusi- end user’s needs.
asm and productivity. Poor quality.
No team spirit. High Increased total cost.
turnover. Difficult to Increased delivery
recruit staff members. time.
Bad attitude towards
doing quality work
and following stan-
dards. Fragmentation.

© Daniel Svennberg 2001 83


6 Processes

Risk category Problem examples Possible impact


Loss of key personnel.
C.7 Development sys- Inadequate develop- Product doesn’t meet
tem ment system. New sys- end user’s needs.
tem – training re- Poor quality.
quired. Development Increased total cost.
system doesn’t match Increased delivery
the user’s or the main- time.
tainer’s system.
C.8 Development Inefficient or unsuit- Rework.
process able process. New and Poor quality.
unfamiliar process. Increased total cost.
Too complicated or Increased delivery
administrative process. time.
Process is not fol-
lowed. Lack of control
and coordination of
process. Poor quality
assurance and lack of
configuration man-
agement.
C.9 Design and coding Large parts of the Product doesn’t meet
product are unspeci- end user’s needs.
fied – developers have Rework.
to guess. Harder to Poor quality.
implement design than Increased total cost.
anticipated. Goldplat- Increased delivery
ing. time.
C.10 Integration Integration of the soft- Rework.
ware with COTS and Increased total cost.
hardware is harder Increased delivery
than anticipated. time.
C.11 Testing and Not sufficiently tested Rework.
evaluation by developers. Not Product doesn’t meet
tested in a realistic en- end user’s needs.
vironment prior to de- Poor quality.
ployment. Not ade- Increased total cost.
quately tested. Not Increased delivery
evaluated by end user. time.
C.12 Maintenance Maintenance of prod- Expensive to maintain.
uct impaired by poorly Increased total cost of
designed product, lack ownership.
of documentation, etc.
Maintainer might go
out of business.

84 Software Acquisition Management Guidelines


6 Processes

6.8 Contractor selection


The purpose of the contractor selection process is to evaluate and
select a suitable contractor for the acquisition project. For a
more in-depth study of contractor selection, see Assmann
(2000).

Contractor selection process

Starts when the initial set of Ends when the contract has
requirements has been elicited. been awarded.

Input: Project specification, Output: Contract


Requirements specification,
Risk list, Proposals

Contractor selection group

The participants to consider for this process should be those


who have the authority to sign a contract and those who can
give advice on the selection of a suitable contractor and contrac-
tual issues such as the following roles: sponsor, collaborating
customer, product manager, acquisition manager, technical
manager, contract manager, administrator, acquisition expert,
technical expert, domain expert, legal expert, and translator.

Minimal process activities

‰ Certify that the solicitation activities are conducted in ac-


cordance with relevant laws, policies, and guidance.

‰ Identify potential contractors.


List contractors to send requests for proposals to. Well-known
contractors should be primary candidates to minimize risk of
getting unserious or over-optimistic bids according to Os-
karsson (2000).

‰ Make cost and time estimates.


Use estimates by analogy or parametric models for minimal
processes as they are the least time-consuming to do accord-

© Daniel Svennberg 2001 85


6 Processes

ing to Marciniak and Reifer (1990). Have the estimates inde-


pendently reviewed.

‰ Develop and send out a request for proposal to the poten-


tial contractors.
See the Request for proposal on page 47.

‰ Select contractor.
Select the winning contractor based upon all available data
on the contractors such as the proposals (see page 50), even-
tual audits, previous customer comments, previous data on
contractor performance, etc. Remember that the lowest bid
doesn’t necessarily mean the lowest total cost.

‰ Develop and negotiate a contract.


Select contract type (see page 88) and develop a suitable con-
tract. Get legal assistance for this. Standard software con-
tracts can be achieved from legal firms. Have legal counsel
review all contracts. Determine how payment is to be made.
Establish acquirer and contractor obligations. Incorporate
the requirements and evaluation criteria in the contract. Es-
tablish acceptance test procedures. See the Contract on page
51.

‰ Jointly review the requirements.


Review the requirements with the contractor prior to signing
the contract to ensure a mutual understanding of the re-
quirements.

‰ Jointly review the contract.


Review the contract prior to signing it to ensure that both
parties have a common understanding of it. Especially ac-
ceptance criteria and procedures, handling of changes to re-
quirements, handling of problems after delivery, the obliga-
tions of the acquirer, and legal issues regarding intellectual
property, warranty, confidentiality, etc. should be reviewed.

‰ Sign the contract.

86 Software Acquisition Management Guidelines


6 Processes

Additional controlled process activities

‰ Develop contractor evaluation criteria and evaluate the


proposals against them.
Develop Contractor evaluation criteria (see page 49) and
evaluate the proposals against them and not against each
other, according to Salwin (1998).

‰ Audit the candidate contractors.


Conduct audits at the candidate contractors’ sites; Evaluate
their financial position, technical capability, experience, and
quality practices. Review the experience of the contractor’s
staff. Consider interviewing the staff members. Assess the
stability of the contractor, is the organization undergoing
change, such as changing ownership or moving office? In-
terview previous customers. Check the background of the
contractor and their staff. Evaluate the adequacy of the con-
tractor’s development process. Evaluate the quality of previ-
ous work; see code samples for instance. Evaluate the con-
tractor’s overall capabilities and identify any limitations and
liabilities in meeting the organization’s objectives.

‰ Make cost and time estimates.


Make bottom-up estimates based upon the work breakdown
structure (WBS). Use the Delphi method, i.e. let various ex-
perts reach a consensus on the most plausible estimates.
Have the estimates independently reviewed.

Additional robust process activities

‰ Distribute proposal guidelines for review to potential bid-


ders prior to issuing the formal request for proposal.

‰ Consider using two-phase acquisitions.


Consider diving the project into two separate phases with
separate contracts. The initial phase is used to define the re-
quirements, develop alternate design approaches, and proto-
type implementations. The second phase consists of the full-
scale development of the product. (Reifer, 1997)

© Daniel Svennberg 2001 87


6 Processes

‰ Make cost and time estimates.


Use several different estimation techniques, such as the Del-
phi method, Monte Carlo-simulations, parametric estimates,
etc. Have the estimates independently reviewed.

6.8.1 Contract types


Selecting contract type is a fundamental decision in the acquisi-
tion process. There are two main types of contracts, fixed-price
and cost-reimbursable, and a number of variants of them. In
fixed-price contracts, the contractor assumes the main part of
the cost risk. Because of this, the scope of the work needs to be
well defined and attainable, and the work and any changes to
the scope require strict management. In contrast, the customer
assumes the main part of the cost risk for cost-reimbursable
contracts. This requires an administrative overhead of keeping
detailed and accurate cost and progress records. A benefit of
cost-reimbursable contracts is that you can have a more flexible
management of the work and changes to the scope, suitable for
innovative projects or projects where the scope of the work is
volatile. A drawback is that there is a lack of incentives to con-
trol the cost and schedule, according to Urmi (2000).

According to Marciniak and Reifer (1990), there are two types


of incentives: cost incentives and award incentives. Cost incen-
tives can be used to motivate the contractor to shorten schedule
or reduce cost. The fee is based upon how well the contractor
meets cost or schedule objectives. However, a caveat should be
given that if the target cost of the incentive fee structure is se-
verely off target to the disadvantage of the contractor, the con-
tractor might consider continuing to accrue labor hours in order
to increase revenue and make up for the loss of fee. This possi-
bility could be negated by having negative fees or penalties.
The customer should also consider stopping the work of the
contractor to protect the budget.

Award incentives can be used to motivate the contractor to


meet certain success criteria regarding end user satisfaction and
88 Software Acquisition Management Guidelines
6 Processes

software quality, or for using certain software development


“best practices” such as peer reviews, configuration manage-
ment, earned value, etc. Marciniak and Reifer (1990) argue that
award fees need to be connected with the completion of a well-
defined event. Brown, et al. (1998) presents some questions that
should be of concern when using award incentives: How can
the incentive targets be objectively evaluated? How will cases
where factors beyond the control of the contractor impact the
contractor’s achievement of the incentives criteria be dealt
with? Are the additional administrative costs that will incur in
managing the contract outweighed by the benefits achieved?

In the tables below the different variations on these two con-


tract types and the time and materials contract type are pre-
sented. Each table contains a brief description on how the con-
tract works, when it is applicable, who takes the main cost risk
– contractor or customer, and also rates its administration level
on a subjective scale from very low to very high. The contract
types presented are the most common, although there are other
contracting arrangements, such as basic ordering agreements,
etc. When choosing contract type you could consider dividing
the project into phases, such as a requirements elicitation phase
followed by a software development phase, and have different
contracting vehicles for each phase.

The contents of the contract types presented below are based


upon Brown et al. (1998), Marciniak and Reifer (1990), Reifer
(1997), and Wideman (1992).

FFP – Firm fixed price

Description The customer pays a fixed price for the con-


tractors work.

Applicability FFP contracts apply for projects where the


cost estimates are fairly reliable, the scope of
the work is well defined and non-volatile,
and the attainability of the project is very
likely.

© Daniel Svennberg 2001 89


6 Processes

Cost risk Customer Contractor

Administration Very low – if a lump sum is paid at final de-


livery.
Low – if partial payments are used (e.g.
monthly).
High – if partial progress payments are used
(e.g. milestone driven).

FPEPA – Fixed price with economic price adjustment

Description A fixed price contract with price adjustments


for certain costs for material, labor, travel ex-
penses, etc.

Applicability Similarly to FFP contracts, FPEPA contracts


apply for projects where the scope of the
work is well defined and non-volatile and the
attainability of the project is very likely. The
FPEPA contract is specifically applicable for
lengthier contracts where both parties need to
be protected from certain cost fluctuations,
such as airfares, etc. This contract can be
combined with FPR.

Cost risk Customer Contractor

Administration Moderate – with lump sum payment.


High – with partial payments.

FPR – Fixed price redetermination

Description An initial fixed price is negotiated. Certain


factors are agreed upon à priori for adjusting
the price at specific times such as require-

90 Software Acquisition Management Guidelines


6 Processes

ments growth, changes in function points, etc.

Applicability The FPR contract applies to situations where


the scope of the work is partially well defined
and non-volatile and partially volatile.

Cost risk Customer Contractor

Administration Moderate

FPIF – Fixed price plus incentive fee

Description In addition to paying a fixed price, a variable


fee is determined by negotiating the follow-
ing parameters:

• Target and ceiling cost


• Minimum, maximum, and target fee
• Adjustment formula (or share ratio)

The adjustment formula specifies how target


cost underruns and overruns – without going
over the ceiling cost – will be shared between
the customer and contractor.
Applicability Similarly to FFP contracts, FPIF contracts ap-
ply for projects where the cost estimates are
fairly reliable, the scope of the work is well
defined and non-volatile, and the attainability
of the project is very likely. FPIF contracts can
specifically be used as an instrument to moti-
vate the contractor to increase efficiency,
shorten schedule, and reduce cost. This con-
tract can be combined with FPR.

Cost risk Customer Contractor

© Daniel Svennberg 2001 91


6 Processes

Administration Moderate – with lump sum payment.


High – with partial payments.

FPAF – Fixed price plus award fee

Description The contractor is paid a fixed price and a base


fee. In addition, the contractor receives award
fees when certain criteria are met. The base
fee should be set low, around 3% of the target
cost according to Marciniak and Reifer (1990),
and the maximum fee could go up to 20%.

Applicability Similarly to FFP contracts, FPAF contracts


apply for projects where the cost estimates
are fairly reliable, the scope of the work is
well defined and non-volatile, and the attain-
ability of the project is very likely. FPAF con-
tracts can specifically be used as an instru-
ment to stimulate the contractor’s perform-
ance in certain areas that are difficult to
measure objectively such as end user satisfac-
tion, software quality, use of “best practices,”
etc.

Cost risk Customer Contractor

Administration High

CS – Cost and cost share

Description In a cost share contract the customer and con-


tractor agree on the ratio by which they will
share costs. In a cost contract the customer
pays for all allowable costs. The contractor re-
ceives no fee in either contract type.

92 Software Acquisition Management Guidelines


6 Processes

Applicability Cost share contracts apply for joint develop-


ment efforts where both parties share risk in
order to share the future rewards from the
product. Cost contracts apply for projects
with nonprofit organizations.

Cost risk 100% by the customer for cost contracts.


Shared, according to the agreed ratio, for cost
share contracts.

Administration High

CPFF – Cost plus fixed fee

Description The contractor's allowable costs are reim-


bursed and a fixed fee is paid upon delivery.

Applicability CPFF contracts apply for projects where the


costs cannot be reasonably estimated such as
innovative projects or projects where the
scope of the work is volatile or difficult to de-
fine.

Cost risk Customer Contractor

Administration High

CPPF – Cost plus percentage fee

Description The contractor’s allowable costs are reim-


bursed and a percentage of the costs is re-
ceived as a fee.

Applicability CPPF contracts apply for projects where the


costs cannot be reasonably estimated such as

© Daniel Svennberg 2001 93


6 Processes

innovative projects or projects where the


scope of the work is difficult to define.

Cost risk Customer Contractor

Administration High

CPIF – Cost plus incentive fee

Description In addition to reimbursing the contractor’s al-


lowable costs, a variable fee is determined by
negotiating the following parameters:

• Target and ceiling cost


• Minimum, maximum, and target fee
• Adjustment formula (or share ratio)

The adjustment formula specifies how target


cost underruns and overruns – without going
over the ceiling cost – will be shared between
the customer and contractor.

Applicability Similarly to CPFF contracts, CPIF contracts


apply for projects where the costs cannot be
reasonably estimated such as innovative pro-
jects or projects where the scope of the work
is difficult to define. CPIF contracts can spe-
cifically be used as an instrument to motivate
the contractor to increase efficiency, shorten
schedule, and reduce cost.

Cost risk Customer Contractor

Administration Very high

94 Software Acquisition Management Guidelines


6 Processes

CPAF – Cost plus award fee

Description The contractor is reimbursed for allowable


costs and receives a base fee. In addition, the
contractor receives award fees when certain
criteria are met. The base fee should be set
low, around 3% of the target cost according to
Marciniak and Reifer (1990), and the maxi-
mum fee could go up to 20%.

Applicability Similarly to CPFF contracts, CPAF contracts


apply for projects where the costs cannot be
reasonably estimated such as innovative pro-
jects or projects where the scope of the work
is difficult to define. CPAF contracts can spe-
cifically be used as an instrument to stimulate
the contractor’s performance in certain areas
that are difficult to measure objectively such
as end user satisfaction, software quality, use
of “best practices,” etc.

Cost risk Customer Contractor

Administration Very high

TM/L – Time and materials / Labor hours

Description The contractor is paid an hourly rate for the


time to perform the agreed upon work and is
reimbursed for any necessary purchases.

Applicability TM/L contracts apply for projects where the


extent and duration of the work cannot be
reasonably estimated. The contract type suits
projects where you want to be able to quickly
change the direction of the work and can
monitor the work closely. A ceiling price

© Daniel Svennberg 2001 95


6 Processes

should be determined.

Cost risk Customer Contractor

Administration Low

The table below gives suggestions for what contract types to


consider for the different customization levels:

Minimal FFP, FPR, FPIF, TM/L

Controlled FPEPA, FPR, FPIF, CS, CPFF, CPPF, CPIF

Robust FPEPA, FPR, FPIF, FPAF, CS, CPFF, CPPF,


CPIF, CPAF

6.9 Contract management


The purpose of the contract management process is to monitor
the work of the contractor, gather data on project status, resolve
problems, administer payments, and other contractual issues.

Contract management process

Starts when the contract has Ends at the conclusion of the


been signed. contract’s period of perform-
ance.

Input: Contract Output: Measurements

Contract management group

The participants to consider for this process should be those


with the ability and authority to administer the contract, resolve
contractual problems and issues, and those who can give assis-

96 Software Acquisition Management Guidelines


6 Processes

tance in monitoring and assessing the progress of the contractor


such as the following roles: acquisition manager, contract man-
ager, administrator, acquisition expert, technical expert, domain
expert, translator, and legal expert.

Minimal process activities

‰ Perform the acquirer’s contractual obligations.


Perform the acquirer’s obligations as stated in the contract
such as delivering software, hardware, data, or other deliv-
erables essential for the development of the software product
in a timely manner and with the agreed specifications.

‰ Gather project status data.


Determine what to monitor, for instance measurements on
budget, schedule, progress, risk, scope, quality, etc. Establish
routines for gathering the data. See Project status metrics on
page 100. Require frequent status reports etc. from the con-
tractor (see Artifacts from the contractor on page 53).

‰ Review budget and schedule status.


Track budget and schedule and identify trends. Compare to
plan and what was accomplished. See Earned value on page
102. Watch out for delays, small delays accumulate. Force
replanning if schedule slippage is greater than 10% accord-
ing to Brown (1998). (Oskarsson, 2000)

‰ Evaluate the software product regularly.


See the Product evaluation process on page 106.

‰ Report measurements and problems.


Report measurements on the process and product, etc. on a
regular basis to the acquisition manager for inclusion in the
next Project status report (see page 56) to the steering group.
Also give feedback to the contractor. Eventual problems
should be reported immediately.

‰ Communicate regularly and promote a supporting rela-


tionship with the contractor.
Create an effective, functional, supporting and collaborative

© Daniel Svennberg 2001 97


6 Processes

culture in the relationship with the contractor built on trust.


Both parties should realize that they have a mutual respon-
sibility for the success of the project. The involved parties
should work as a team, jointly solve problems, and refrain
from blaming each other. Conduct regular project meetings
with all involved parties to give feedback and discuss pro-
gress, requirements, budget, schedule, risks, problems, etc.

‰ Manage contractual change.


Coordinate all contractual changes with all affected groups
and individuals. Ensure that all proposed changes to the
contract are analyzed, documented, agreed upon or denied
and communicated to all affected parties.

‰ Administer partial payments, incentives, etc.

‰ Settle the contract.


On final delivery and acceptance of the product, the contract
should be completed and settled, any open issues should be
resolved. Final payment should be withheld until all is de-
livered. Check that deliveries have been completed success-
fully, ownership has been transferred, etc. See also the
Transition to support process on page 110.

Additional controlled process activities

‰ Review the contractor’s plans.


Review the contractor’s plans regarding project manage-
ment, risk management, quality control, configuration man-
agement, integration, and testing and use them as a basis for
overseeing the contractor’s development effort. The contrac-
tor’s use of subcontractors, software/hardware suppliers,
and tools should be reviewed and approved.

‰ Reestimate total cost and delivery time when appropriate.


Base the estimates on WBS and extrapolate from actuals.

‰ Audit the contractor on a regular basis.


Hold audits to determine project and product status. Audits
can be performed both on predetermined occasions and on

98 Software Acquisition Management Guidelines


6 Processes

an ad hoc basis. Audit procedures should be agreed upon by


both parties. Analyze the outcome of the audit, identify
trends, and adjust the acquisition approach to mitigate risks
if necessary. The outcome regarding risks, problems and
eventual change of project direction should be discussed
with the contractor. The outcome of the audit and responsi-
bilities for resolving found problems should be agreed upon
by both parties and documented. Technical experts or ac-
quisition experts can be of assistance during audits.

Examples on what to review:

o That the working methods of the contractor regarding


methodologies, processes, procedures, practices, qual-
ity assurance, etc. are adhered to and conducted in an
effective fashion.

o That the contractor has planned the project adequately


and timely.

o That the project is staffed appropriately, that the staff


is working on the actual project, and that the personnel
is trained as required by the contract.

o That the allocated resources are appropriate.

o That documents and deliverables meet specifications


and follows coding and documentation standards, etc.

o That the technical approach and working methods are


appropriate.

o The configuration management practices.

o Risk mitigation activities.

o Interview managers and technicians on technical and


managerial difficulties.

© Daniel Svennberg 2001 99


6 Processes

Additional robust process activities

‰ Use external auditors.


For longer projects you could have an external auditor visit
the contractor and review the status of the project, interview
the managers and especially technicians about the project
and eventual problems. The auditor can also look at random
error reports and track the error as a test of the contractor’s
configuration management routines, etc. (Oskarsson, 2000)

6.9.1 Project status metrics


It is important to choose metrics that motivates the right behav-
ior, or in the words of Jim Coplien:

What gets measured gets done.

The following are examples of metrics that can be used to


measure the status of the project:

• Requirements change rate. The target value for total re-


quirements growth is ≤ 1% per month. (Brown, 1998)
• Contractor staff turnover rate. The target value for volun-
tary staff turnover per year is 1-3%according to Brown
(1998). High staff turnover indicates low staff moral.
• Contractor staff overtime rate. (Brown, 1998)
• Contractor staffing rate. Actual staffing compared to
needs. (Marciniak and Reifer, 1990)
• Test progress. Executed test cases compared to planned,
successful tests, tests with errors, etc. (Oskarsson, 2000)
• Software maturity index, SMI. Indicates the stability of
the product. The product is stabilizing when SMI ap-
proaches 1.0. (IEEE 982.1-1988)

MT = number of modules in current version.


Fc = number of changed modules in current version.

100 Software Acquisition Management Guidelines


6 Processes

Fa = number of added modules in current version.


Fd = number of deleted modules in current version.

MT – (Fa + Fc + Fd)
SMI=
MT
• Developmental progress. Function points completed, re-
quirements completed, incremental releases, modules
completed, etc. (Marciniak and Reifer, 1990)
• Milestones completed. Actual vs. planned, see the figure
below. (Oskarsson, 2000)

Time

Replan 2

Replan 1

1 2 3 4 Time

Milestones
Figure 6. Milestone tracking and trend diagram, Oskarsson (2000).

• Problem reports. Open vs. closed. Marciniak and Reifer


(1990) define problems as anomalies discovered in re-
quirements, design, or software.
• Size growth. How the size of each software item grows as
a function of time. Size is directly related to cost accord-
ing to Marciniak and Reifer (1990).
• Error rate. The amount off discrepancies found during
testing and evaluation. High error rates indicate product
immaturity and low quality according to Marciniak and
Reifer (1990).
• Earned value metrics. See Earned value, on page 102.

© Daniel Svennberg 2001 101


6 Processes

6.9.2 Earned value


According to Fleming and Koppelman (1996) the concept of
earned value has been around for at least a hundred years, first
used on the factory floors in the late 1800s. The usage of earned
value enables the prediction of final costs and final schedule re-
sults, within a certain range, for the actual project. Since 1967,
every new major systems project implemented by the United
States Department of Defense (DoD) has been required to use
the Cost/Schedule Control Systems Criteria, or C/SCSC for
short, which is a set of 35 requirements embedding the earned
value concept. Over 700 major DoD contracts since 1977, where
C/SCSC (and thus earned value) has been used has shown that
you can, as early as 15-20% into the project, predict the final
costs and time requirements within a predictable range of val-
ues.

Fleming and Koppelman (1996) suggests a simplified version of


earned value to be used for virtually any project who relates
planned costs to actual expenses and don’t have a clue about
what they got for what they spent. I agree with them and sug-
gest that you can use the concept for any size of project and
with any type of development model, even iterative.

Earned value project requirements


The earned value concept requires the following:

1. An appropriate defined scope of the project (whole pro-


ject or iteration), or work to be done, which enables you to
tell what’s in the scope and what’s not. This is preferably
made with a work breakdown structure, WBS, down to
the necessary level of granularity. For cost-reimbursable
contracts, the buyer defines the top two or three levels of
the WBS. For fixed-price contracts the contractor defines
all of the WBS. The work should be defined down to dis-
crete work tasks that can be managed and easily deter-
mined complete not complete.

102 Software Acquisition Management Guidelines


6 Processes

2. The work segments defined in the previous step must be


planned and scheduled. Typically, CPM1 is appropriate to
accomplish this. If several contractors and subcontractors
are used, their schedules must of course be in accordance
with a master schedule.

3. Responsibility for each work segment should be assigned


and resources will need to be estimated and budgeted for
all work segments within the defined project scope. When
estimating, refrain from adding reserve time. Do this later
when all of the work segments have been estimated.
Thus, the so called Cost Account Plan should contain the
following:

a. Statement of work (scope).

b. Schedule (start/stop dates for each task).

c. Budget (in dollars or hours or units).

d. Responsible person.

e. Responsible department.

f. Type of effort (recurring or non-recurring).

g. Division into discrete work packages.

h. Method used to measure earned value performance


(milestone, formula, percent complete, standards,
apportioned).

4. Establish a baseline of the estimated, scheduled and


budgeted work segments in the previous step, not includ-
ing any reserves, from which project or iteration perform-
ance can be measured. The baseline should be under con-

1 Critical Path Method

© Daniel Svennberg 2001 103


6 Processes

figuration management to enable traceability back to the


original baseline. All changes to the baseline have to be
reviewed, controlled, and approved or rejected. The base-
line is needed to be able to tell how much of the project is
finished, to be able to compare completed work against
actual work and actual costs.

5. Monitor iteration/project performance against the base-


line and forecast final results. Also manage baseline
changes to maintain the baseline current.

An example
Assume you have a one-year $100,000 project scheduled with
an expenditure rate of $25,000 per quarter. At the end of the
first quarter the actual expenditures are $22,000 and thus $3,000
under the planned effort. From these values you can’t really tell
whether the project is behind schedule or underrunning costs.

By adding the earned value dimension, which measures the


value of the actual work accomplished, we can tell much more
about the actual status of the project. Let’s say that the work
segments completed at the end of the first quarter had an
earned value of $20,000. Then we can immediately tell that we
are $5,000 behind the planned
$ 25 000 Planned work of $25,000. Since the
actual expenditures to accom-
plish the work are $22,000 we
22 000 Spent can also tell that we have a cost
20 000 EV overrun of 10%. So, with
earned value we are able to tell
that the actual project is
running late and
1Q overspending. See the figure to
the left.
Figure 7. Earned value example.

104 Software Acquisition Management Guidelines


6 Processes

Testable requirements
Peter B. Wilson (2000) suggests the use of testable requirements
with earned value. A testable requirement is defined as a re-
quirement that someone is able to write one or more test cases
for that would validate whether the requirement has or has not
been implemented correctly.

As an example you plan to implement 80 testable requirements


in 500 labor hours during the first month of the project. When
the month has passed 475 labor hours has been expended and
72 of the requirements have been implemented correctly. This
gives you an earned value of (72/80) * 500 = 450 labor hours. So
in this case you overspent 25 labor hours.

Schedule and cost performance indices


Schedule performance index, or SPI, is determined by the follow-
ing formula:

Earned Value
SPI =
Planned Value

This value tells you how much of the work planned to be ac-
complished actually was accomplished. A SPI < 1.0 is a warning
signal. The SPI can be a valuable tool for use in conjunction
with critical path analysis to forecast the expected completion
date for the project.

Cost performance index, or CPI, is determined by the following


formula:

Earned Value
CPI =
Actual Costs

This value tells you how much work was accomplished for
every project dollar spent. A CPI < 1.0 is a warning signal. The
CPI can be used to forecast a statistical range of estimated final
costs to complete the project.
© Daniel Svennberg 2001 105
6 Processes

Forecasting final cost and schedule


A realistic bottoms-up estimate of the remaining tasks is the
most desirable forecasting method, but CPI and SPI can be used
to produce statistical forecasts of final required funds to com-
plete the project.

Statistical range of cost-to-complete forecasts:

Total budgeted funds


Low end forecast =
(Best case)
CPI

Total budgeted funds


High end forecast =
(Most likely)
CPI * SPI

The To-Complete Performance Index, or TCPI, can be used to de-


termine what performance level it will take to accomplish all
remaining effort to achieve some specified management objec-
tive.

Earned Value of work remaining


TCPI =
Funds remaining

6.10 Product evaluation


The purpose of the product evaluation process is to evaluate the
deliverables from the customer to see if they meet the specified
requirements and quality objectives. This includes evaluation of
partial or incremental deliveries and the final evaluation, i.e. ac-
ceptance test.

106 Software Acquisition Management Guidelines


6 Processes

Product evaluation process

Starts when the requirements Ends when the product and


are being developed. other deliverables have been
finally accepted and delivered
as agreed.

Input: Requirements specifica- Output: Product evaluation


tion reports, Change requests

Product evaluation group

The participants to consider for this process should be those


whose needs are to be fulfilled by the product, those who can
give assistance and advice on the evaluation of the deliverables,
and those who have the authority to accept the deliverables
such as the following roles: sponsor, collaborating customer,
product manager, acquisition manager, technical manager,
administrator, technical expert, domain expert, legal expert,
translator, end user, maintenance and support staff, independ-
ent verification and validation, and system engineering and
technical assistance.

Minimal process activities

‰ Determine what testing approaches to use.


Determine the appropriate testing approaches and the test-
ing effort required. See Product evaluation approaches on page
109.

‰ Identify the artifacts to be evaluated.

‰ Write test cases up-front.


Make sure that evaluation criteria have been established for
all the requirements with instructions on how to perform the
evaluation for each requirement. Establish evaluation proce-
dures and criteria for all chosen testing approaches.

‰ Conduct periodical evaluations.


Conduct periodical evaluations throughout the acquisition
© Daniel Svennberg 2001 107
6 Processes

project to minimize the risk of getting a product that doesn’t


live up to the expectations of the end user. Write plans for
the different evaluations if that is deemed necessary (see the
Product evaluation plan on page 54).

‰ Analyze and take action on evaluation results.


Certify that the coverage of the tests is sufficient. Analyze
the test results regarding requirements and quality fulfill-
ment and determine what action needs to be taken for the
found deficiencies. Determine if any changes need to be
made to the requirements. Make appropriate documentation
of the test results (see the Product evaluation report on page
55).

‰ Review the evaluation results with the contractor.


Discuss the found deficiencies, non-conformances, remedies,
and eventual requirements changes with the contractor.
Don’t forget to give some positive feedback too to maintain
good relations with the contractor.

‰ Conduct acceptance testing.


Conduct the procedures for final evaluation and acceptance
of the product as agreed upon in the contract. Use all evalua-
tion results as a basis for accepting the product.

Additional controlled process activities

‰ Document requirements change requests.


See the Change request on page 55.

‰ Track found deficiencies.


Document and follow-up on the deficiencies that were found
during the evaluations.

‰ Overview the contractor’s testing efforts.


Review the results of the contractor’s testing activities.

Additional robust process activities

‰ Identify common errors.

108 Software Acquisition Management Guidelines


6 Processes

Collect statistics on common errors. Suggest improvements


and fault preventions.

‰ Consider using IV&V for products that can affect people’s


health or huge amounts of money should they malfunc-
tion.

6.10.1 Product evaluation approaches


The following is an overview of some different approaches for
evaluating a software product. Several of these tests should be
conducted throughout the project and during the final accep-
tance testing. It is important to remember that tests cannot
show that a product is 100% error-free. I’ve not included tests
that are primarily performed by the contractor such as unit and
integration tests, but limited the list to tests in which the cus-
tomer takes part:

Alpha tests are conducted on pre-release versions of the soft-


ware product by the customer in a controlled environment at
the contractor’s site with developers present. (Pressman, 1997)

Beta tests are conducted on pre-release versions of the software


product by end users at one or more customer sites. The con-
tractor is normally not present during beta tests. (Ibid.)

Functional tests demonstrate that the functional requirements


have been met. These tests can be conducted by working
through use cases or performing several tests defined for each
requirement or other means. Not only correct but also errone-
ous inputs should be given to the system.

Installation tests should be conducted to ensure that the prod-


uct can be installed correctly.

Support tests should be conducted to ensure that the needs of


the support and maintenance organization are met.

© Daniel Svennberg 2001 109


6 Processes

Recovery tests forces the software to fail and verifies that the
system recovers properly. (Ibid.)

Security tests should be performed to verify that the protection


mechanisms built into a system will in fact protect it from im-
proper penetration. (Ibid.)

Stress tests executes a system in a manner that demands re-


sources in an abnormal quantity, frequency, or volume in an at-
tempt to break the program. (Ibid.)

Performance tests are important for real-time and embedded


systems to see if they fulfill the performance requirements.
(Ibid.)

User interface tests are important tests performed by the end


users to evaluate the user interface’s appropriateness.

6.11 Transition to support


The purpose of the Transition to support process is to prepare for
a smooth transition of the deliverables to the support and main-
tenance organization and to verify that all deliverables and
other services, such as user training and installation, has been
received as agreed upon.

Transition to support process

Starts when the software re- Ends when the software and
quirements are being devel- other deliverables have been
oped. turned over to the support or-
ganization.

Input: Deliverables Output: Maintenance plan

Transition to support group

The participants to consider for this process should be those in


charge of approving the delivery of the product and those who

110 Software Acquisition Management Guidelines


6 Processes

are to install, maintain and support the product such as the fol-
lowing roles: acquisition manager, technical manager, contract
manager, administrator, technical expert, domain expert, legal
expert, maintenance and support staff, and contractor, collabo-
rating contractor, and subcontractor.

Minimal process activities

‰ Identify the support and maintenance organization.


Identify and provide a budget for the support and mainte-
nance organization early in the project, before the request for
proposal is issued. Will maintenance be performed by the ac-
quirer organization or will it be outsourced?

‰ Identify the deliverables to be transferred.


Make a list of the deliverables that are to be transferred to
the support organization, such as maintenance and support
documentation, software, training materials, configuration
management systems, etc.

‰ Plan the installation of the software product.


Establish installation procedures regarding schedule, access
to facilities, personnel, availability and access to systems and
equipment, approval of installation, etc.

‰ Review the results of the user training activities.


Make sure that the agreed upon user training has been per-
formed.

‰ Verify that all deliverables has been received as specified.

Additional controlled process activities

‰ Prepare the support and maintenance organization.


Transfer and customize necessary processes from the devel-
opment organization to the support organization. A mainte-
nance plan should be developed. See the Maintenance plan on
page 46.

‰ Review the deliverables and preparations prior to transfer.

© Daniel Svennberg 2001 111


6 Processes

Review the usability of the software, its readiness for instal-


lation, status of user training, user manuals, status of instal-
lation preparations and activities. Review the readiness of
the software for transition, the maintenance plans and
manuals, the status of transition preparation activities. Re-
view copyright and licensing concerns addressed and agreed
to. Review concerns regarding back-up copies of the soft-
ware.

Additional robust process activities

‰ Test the support capabilities of the support and mainte-


nance organization.
Before transferring responsibility for the software, the sup-
port organization needs to demonstrate its capability to sup-
port and maintain the software:

o Does it have the appropriate hardware, software,


physical, fiscal, and personnel resources?

o Does it have a maintenance plan and other documenta-


tion, processes and procedures?

o Does it have a configuration management system ca-


pable of supporting the software?

o Is the personnel trained?

‰ Maintain configuration and change management through-


out the transition.

112 Software Acquisition Management Guidelines


6 Processes

6.12 Post partum


The purpose of the post partum1 process is not about passing
judgment, but to learn from experience. What can be done bet-
ter on future acquisition projects? It also includes documenting
data on the project effort and on the contractor’s performance
for future reference. Apart from the frameworks mentioned in
appendix A, this process is based upon Kerth (n.d.).

Post partum process

Starts when the project has Ends when the project has
been finalized. been sufficiently reviewed and
documented for future refer-
ence.

Input: Project data Output: Post partum

Post partum group

Preferably representatives from all involved organizations and


roles should participate in the post partum.

Minimal process activities

‰ Form a post partum group.


Also assign responsibility, authority, and accountability for
the post partum activities. Make sure that the people in-
volved in the process are trained in how to perform the
process activities.

‰ Allocate adequate resources for post partum activities.


Resources to consider are: funding, people, tools, equipment,
facilities, etc.

1Post partum means “after birth” in Latin. I find this to be a more appro-
priate name for the process instead of the more common term post mortem,
which means “after death.”

© Daniel Svennberg 2001 113


6 Processes

‰ Set a goal for the post partum.


There can be many different purposes for holding a post par-
tum:

o To capture effort data in order to understand the exist-


ing process, or to improve the existing process, or to
provide data for future acquisitions, or to identify
training needs, etc.

o To document what happened and why. To capture the


collective wisdom.

o To repair damage to the team.

o To enjoy the accomplishment.

‰ Plan the post partum.


See the Post partum plan on page 57.

‰ Prepare for the post partum.


Inform the participants that the post partum is a learning
experience and not a blame session. Establish rules that cre-
ate a non-threatening atmosphere for the post partum.
Gather the data that will be analyzed during the post partum
such as data on efforts, resources, product quality, etc. Each
participant should refresh their memory and individually
review what happened.

‰ Conduct the post partum review.


Review what happened during the acquisition project and
identify good and bad acquisition practices. What went
wrong and what went right? Review the efforts expended
regarding budget, staff, schedule, etc. Identify deviations
and reasons to them. What was accomplished? What was the
quality of the delivered product? Review any special diffi-
culties and how they were handled. Review the contractor’s
performance and compliance with requirements. If the con-
tractor doesn’t participate in the post partum review, a post
partum from the contractor should be reviewed. Give im-
provement suggestions. What should be done different the
114 Software Acquisition Management Guidelines
6 Processes

next time?

‰ Document data and lessons learned.


Identify and document the software acquisition lessons
learned, project data, customization data, and data on the
contractor’s performance. See the Post partum on page 58.

Additional controlled process activities

‰ None.

Additional robust process activities

‰ Get a skilled facilitator.


A suitable facilitator is preferably an outsider to the organi-
zations involved in the project, though knowledgeable about
software projects and with good people interaction skills.

© Daniel Svennberg 2001 115


7 Results and summary

“Caution: Cape does not enable user to fly.”


– Batman costume warning label

This chapter summarizes the guidelines and gives an outlook


on possible future research on software acquisition manage-
ment.

7.1 Guidelines summary


The objective of this master’s thesis is to provide guidelines
from the viewpoint of the customer for managing an acquisition
project concerning an outsourced development of a software
product customized for the specific needs of the customer. The
objective is further broken down into seeking answers to the
following problems:

• What problems are common?

• What activities should be performed?

• What roles are conceivable?

• What artifacts are conceivable?

• How can the activities, roles, and artifacts be customized


for different kinds of projects?

116
7 Results and summary

The guidelines presented in this thesis can be used as a basis for


planning an acquisition project or be read as an overview over
the many issues that concern software acquisitions.

The following summarizes the answers to each of the above


stated questions briefly:

What problems are common?


The most common problems with software acquisition projects
are the following:

• Development costs exceeds budget.


• Time schedule is overrun.
• Outcome does not meet expectations.

According to the Standish Group (1995) Chaos report, which


surveyed over 8 000 software development projects, only 16%
of the examined projects were completed on time, within
budget, and with all features and functions as initially speci-
fied.

What activities should be performed?


There are many activities that can and should be performed for
software acquisition projects. In this thesis they have been col-
lected into the following eleven processes:

• Project steering
• Project management
• Planning and organizing
• Acquisition training
• Requirements management
• Risk management
• Contractor selection
• Contract management
• Product evaluation
© Daniel Svennberg 2001 117
7 Results and summary

• Transition to support
• Post partum

The activities and guidelines presented in the processes can be


summarized into the following high-level activities and guide-
lines:

1. Define and communicate a goal and vision for the pro-


ject.
2. Appoint an acquisition manager that has the ultimate re-
sponsibility for the success of the project. The role of the
project sponsor is to set the outer limits for the project,
provide stable and adequate funding and to support the
project enthusiastically or cancel it.
3. Adapt and customize the acquisition approach and strat-
egy according to the characteristics of the project.
4. Avoid having too much administration, but as a mini-
mum: have written requirements and agreements, docu-
ment important decisions and why they were taken, also
measure progress, software quality, and requirements
changes quantitatively.
5. Know what you want. Have as stable, complete, feasible,
and validated requirements as possible. It is important to
have a defined scope so that you and the contractor
know when you are done and when the contract needs to
be changed.
6. Define how the fulfillment of each requirement will be
evaluated.
7. Ensure end user involvement in the definition of the
product requirements and in the evaluation of the prod-
uct.
8. Tell the contractor what you want and not how to do it.
Jointly review the requirements before the development
starts to sort out misunderstandings and ambiguities.
9. Choose contractor carefully. The contractor with the low-
est bid or most optimistic schedule and budget estimates

118 Software Acquisition Management Guidelines


7 Results and summary

isn’t always the best choice. No management practices


can make up for having a bad contractor.
10. Create an effective, functional, supporting, and collabora-
tive culture in the relationship with the contractor based
on trust and mutual benefit. Both parties should realize
that they have a mutual responsibility for the success of
the project. The involved parties should work as a team,
jointly solve problems, and refrain from blaming each
other.
11. Establish effective communication channels and break
down barriers between people and departments in the
organizations involved in the project. Ensure that you
understand each other.
12. Don’t take advantage of the contractor. Create a win-win
situation. Have a mutually beneficial contract that you
would be comfortable signing as either party.
13. Expect the best, but be proactive and prepare for all even-
tualities. Jointly discuss possible risks and how they
should be mitigated with the contractor.
14. Employ the best and most competent people that is pos-
sible, train them, plan and organize their work. Provide
necessary assistance and resources.
15. Plan for maintenance and identify the maintenance and
support organization before issuing request for propos-
als.
16. Evaluate the evolving software product early and often
prior to final acceptance evaluation to reduce risk of get-
ting a software product that won’t meet the needs of the
end user.
17. Perform reality checks periodically. Take necessary ac-
tions if the answers to the following questions aren’t sat-
isfactory:
o Are we acquiring the right software?
o Are we acquiring the software right?
o Is the contractor building the right software?
o Is the contractor building the software right?

© Daniel Svennberg 2001 119


7 Results and summary

18. Be realistic and reassess your expectations as you learn


more about the problem domain and the technical feasi-
bility of the program as the project progresses. The initial
estimates and specifications of the following four pa-
rameters are likely to change to some degree during the
course of the project:
o Delivery time
o Development costs
o Product scope
o Product quality
You may therefore need to make trade-offs to achieve the
most important goal of the project.

What roles are conceivable?


The roles that are conceivable for a software acquisition project
can be divided into four categories:

The sponsor roles provide the funding for the project and have
the power to cancel the project. The sponsor roles presented in
this thesis are the following: sponsor, collaborating customer,
and product manager.

The managing roles plans, manages, monitors, and administers


the project. The managing roles presented in this thesis are the
following: acquisition manager, technical manager, contract
manager, and administrator.

The assisting roles are played by different experts and support


contractors that can assist and give advise to the managing,
steering and executing roles. The assisting roles presented in
this thesis are the following: acquisition expert, technical expert,
domain expert, legal expert, translator, end user, maintenance
and support staff, independent verification and validation, and
system engineering and technical assistance.

The executing roles are played by representatives from the con-


tractors side, i.e. those developing the actual software product.

120 Software Acquisition Management Guidelines


7 Results and summary

The executing roles presented in this thesis are the following:


contractor, collaborating contractor, and subcontractor.

What artifacts are conceivable?


There are many artifacts that are conceivable for a software ac-
quisition project. The artifacts in this thesis are presented ac-
cording to which phase of the project they are most likely to
appear for the first time:

During the inception phase the needs of the software are iden-
tified, the requirements are elicited, risks are analyzed, and the
acquisition processes are planned. The artifacts presented in
this thesis for the inception phase are the following: acquisition
proposal, project specification, acquisition plan, risk list, re-
quirements specification, and maintenance plan.

During the solicitation phase requests for proposals are sent


out to potential contractors, proposals are received and evalu-
ated, and a contractor is selected with which a contract is nego-
tiated and signed. The artifacts presented in this thesis for the
solicitation phase are the following: request for proposal, con-
tractor evaluation criteria, proposal, and contract.

During the monitoring phase the progress and performance of


the contractor is monitored and the software product is evalu-
ated on a regular basis. The artifacts presented in this thesis for
the monitoring phase are the following: project management
plans, test reports, problem reports, payment requests, progress
reports, change requests, quality reports, software demonstra-
tions and incremental releases, product evaluation plan, prod-
uct evaluation report, change request, and project status report.

During the finalization phase the product and other deliver-


ables are delivered to the support and maintenance organiza-
tion and a post partum review is held. The artifacts presented in
this thesis for the finalization phase are the following: software
executables, source code, technical documentation, user train-
ing material, rest list, user manual, support publications, post
partum plan, and post partum.

© Daniel Svennberg 2001 121


7 Results and summary

How can the activities, roles, and artifacts be


customized for different kinds of projects?
To answer that question we must first find a way to differenti-
ate projects. This thesis examines a number of customization fac-
tors that are used to suggest an appropriate formality level of
the acquisition processes based upon the project’s complexity
and risk.

By examining these factors you can determine which customiza-


tion level is appropriate for your project. Three different cus-
tomization levels are given in the thesis:

Minimal processes: suitable for small projects with stable, well-


defined and attainable requirements, developed by a small, ef-
ficient team from a well-known contractor, etc.

Controlled processes: suitable for somewhat lengthier me-


dium-sized projects with somewhat volatile requirements, de-
veloped by an average contractor, etc.

Robust processes: suitable for large, long-term projects, with


innovative, critical, and volatile requirements, and with a com-
plex set of contractors and subcontractors involved in the de-
velopment effort, etc.

The thesis suggests what activities are appropriate for the dif-
ferent customization levels, and also gives some suggestions for
customization of roles and artifacts.

7.2 Outlook
Software acquisition management involves a wide variety of
topics ranging from general management theory to special is-
sues regarding software testing, contract types, risk manage-
ment, etc. It isn’t possible for a single document to cover any of
these topics completely. Therefore, the reader probably needs to
study texts that cover these topics in depth to gain a better un-
derstanding on how to carry out a successful software acquisi-
tion project.
122 Software Acquisition Management Guidelines
7 Results and summary

However, the following are some suggestions for future re-


search on software acquisition management that could be based
upon this thesis:

• It could be further investigated how to customize and


adapt the different processes and artifacts. Perhaps two
customization levels are sufficient?

• Are there more customization factors? Which factors are


important for which process?

• Similar guidelines for the acquisition of COTS software


could be made.

• Can an expert system or other tools be made for assisting


software acquisition management?

• How can the guidelines be adapted to match common


software development methodologies?

© Daniel Svennberg 2001 123


8 Bibliography

Assmann, Danilo (2000), MASS – a Method for Assessing Soft-


ware Subcontractors, Kaiserslautern, Fraunhofer IESE

Beck, Kent (1999), Embracing Change with eXtreme Programming,


Computer, Vol. 32, No. 10, Oct 1999, pp.70-77,
IEEE 0018-9126/99

Breu, Michael (1995), A Concise Guide to Euromethod, European


Software Institute, ESI-1995-CG004.01

Brooks, Frederick P., Jr (1995), The Mythical Man-Month: essays


on software engineering, Anniversary ed., Reading, Addi-
son-Wesley, ISBN 0-201-83595-9

Brown, Norm et al. (1998), The Program Manager’s Guide to Soft-


ware Acquisition Best Practices version 2.2, Arlington, Soft-
ware Program Managers Network

Brown, Norm et al. (1999), The Guidebook of Software Acquisition


Questions version 1.0, Arlington, Software Program Manag-
ers Network

Carr, Marvin J. et al. (1993), Taxonomy-Based Risk Identification,


Pittsburgh, Software Engineering Institute, CMU/SEI-93-
TR-6

Christel, Michael G. and Kang, Kyo C. (1992), Issues in Re-


quirements Elicitation, Pittsburgh, Software Engineering In-
stitute, CMU/SEI-92-TR-12
124
8 Bibliography

CMMI (2000), Capability Maturity Model Integration,


http://www.sei.cmu.edu/cmmi/ (15 September 2000)

Cockburn, Alistair (n.d.), Balancing Lightness with Sufficiency,


http://members.aol.com/acockburn/papers/barelysuffici
ent.htm (2 August 2000)

DAD (2000), Defense Acquisition Deskbook,


http://web2.deskbook.osd.mil/default.asp (1 September,
2000)

D.I.R. (2000), Tailoring the Guidelines,


http://www.dir.texas.gov/eod/qa/tailor.htm (1 August
2000)

Duncan, William R. (1996), A Guide to the Project Management


Body of Knowledge, Newtown Square, Project Management
Institute, ISBN 1-880410-12-5

Euromethod (1996), Euromethod version 1, Euromethod Project

FAA-iCMM (1997), The Federal Aviation Administration Inte-


grated Capability Maturity Model (FAA-iCMM) Version 1.0,
U.S. Federal Aviation Administration

Fleming, Quentin W. and Koppelman, Joel M. (1996), Earned


value project management, Newtown Square, Project Man-
agement Institute, ISBN 1-880410-38-9

Gallivan, Michael J. (1999), Analyzing IT Outsourcing Relation-


ships as Alliances among Multiple Clients and Vendors, Pro-
ceedings of the 32nd Hawaii International Conference on
System Sciences

Ganssle, Jack G. (1998), The Seven Habits of Highly Defective Pro-


grammers, Embedded Systems Programming – July 1998

© Daniel Svennberg 2001 125


8 Bibliography

Haimes, Yacov Y. et al. (1997), A Holistic Management Framework


for Software Acquisition, Acquisition Review Quarterly –
Winter 1997, Defense Systems Management College, pp.
55-86

IEEE 982.1 (1988), IEEE Standard Dictionary of Measures to Pro-


duce Reliable Software, New York, The Institute of Electrical
and Electronics Engineers, Inc.

IEEE 1062 (1998), IEEE Recommended Practice for Software Acqui-


sition, New York, The Institute of Electrical and Electronics
Engineers, Inc., ISBN 0-7381-1514-2

IEEE/EIA 12207.0 (1996), Standard for information technology –


Software life cycle processes, New York, The Institute of Elec-
trical and Electronics Engineers, Inc., ISBN 1-55037-077-4

ISO/IEC 9126 (1991), Information technology – Software product


evaluation – Quality characteristics and guidelines for their use,
ISO/IEC

ISO/IEC 15504 (1998), Information technology – Software process


assessment, ISO/IEC

Kar, Pradip and Bailey, Michelle (1996), Characteristics of Good


Requirements, 1996 INCOSE Symposium

Kerth, Norman L. (n.d.), An Approach to Post Morta, Post Parta,


and Post Project Reviews, Beaverton, Elite Systems

Marciniak, John J. and Reifer, Donald J. (1990), Software Acqui-


sition Management, New York, John Wiley & Sons, Inc.,
ISBN 0-471-50643-5

Mosemann, II, Lloyd K. (1996), Guidelines for Successful Acquisi-


tion and Management of Software-Intensive Systems, U.S. De-
partment of the Air Force, Software Technology Support
Center

126 Software Acquisition Management Guidelines


8 Bibliography

Oskarsson, Östen (2000), interviewed by author at Östen


Oskarsson’s home, Linköping, 21 April 2000

Oskarsson, Östen (n.d. a), Tutorial – Acquisition and Supply of


Software Development,
http://www.oskarsson.se/useful_info/acqtutorial.html
(4 April 2000)

Oskarsson, Östen (n.d. b), Example of Software Contract Require-


ments,
http://www.oskarsson.se/useful_info/Contract.html (3
May 2000)

Oskarsson, Östen and Glass, Robert L. (1996), An ISO 9000


Approach to Building Quality Software, Upper Saddle River,
Prentice-Hall, Inc., ISBN 0-13-228925-3

Paulk, Mark C. et al. (1993), Capability Maturity Model for Soft-


ware, Version 1.1, Pittsburgh, Software Engineering Insti-
tute, CMU/SEI-93-TR-24

Ran, Mats (2000), interviewed by author at Q-Labs AB,


Linköping, 3 May 2000

Reeves, Jack W. (1992), What is Software Design?, C++ Journal


Vol. 2, Nr. 2 1992

Reifer, Donald J. (1997), Software management, New York, The


Institute of Electrical and Electronics Engineers, Inc.,
ISBN 0-8186-8001-6

Salwin, Arthur E. (1998), The Road to Successful ITS Software Ac-


quisition, Washington, U.S. Department of Transportation,
Report No. FHWA-JPO-98-035

SS-ISO 9000-3 (1992), Kvalitetssystemstandarder – Del 3: Riktlinjer


för tillämpning av SS-9001 vid utveckling, leverans och under-
håll av programvara, SIS – Standardiseringskommissionen i
Sverige

© Daniel Svennberg 2001 127


8 Bibliography

The Standish Group (1995), The Standish Group Report Chaos,


http://www.scs.carleton.ca/~beau/PM/Standish-
Report.html (28 August 2000)

Urmi, Jaak (2000), interviewed by author at Ericsson AB, Kista,


14 April 2000

Wideman, R. Max (1992), Project and program risk management: a


guide to managing project risks and opportunities, Project
Management Institute, ISBN 1-880410-06-0

Wilson, Peter B. (2000), Sizing Software with Testable Require-


ments, Systems Development Management, Auerbach Pub-
lications

128 Software Acquisition Management Guidelines


A Frameworks coverage

This chapter shows that the processes in the guidelines covers


most issues in the main frameworks that have comparable
processes or sections.

A.1 SA-CMM
Guidelines SA-CMM

Project steering Implicitly covered in all key


process areas by the “verifying
implementation” section.

Requirements management Requirements development


and management

Risk management Acquisition risk management

Project management Project management, Project


performance management

Planning and organizing Software acquisition planning,


Project performance manage-
ment

Acquisition training Training program

Contractor selection Solicitation

129
A Frameworks coverage

Contract management Contract tracking and over-


sight, Project performance
management, Contract per-
formance management

Product evaluation Evaluation, Contract perform-


ance management

Transition to support Transition to support

Post partum Project performance manage-


ment

Not covered Process definition and mainte-


nance, Quantitative process
management, Quantitative ac-
quisition management, Con-
tinuous process improvement,
Acquisition innovation man-
agement

A.2 FAA-iCMM
Guidelines FAA-iCMM

Project steering Needs, Coordination

Requirements management Needs, Requirements

Risk management Risk management

Project management Project management, Configu-


ration management, Coordina-
tion

Planning and organizing Project management

Acquisition training Training

130 Software Acquisition Management Guidelines


A Frameworks coverage

Contractor selection Outsourcing

Contract management Contract management, Project


management, Measurement

Product evaluation System test and evaluation

Transition to support Transition

Post partum Not covered

Not covered Integration, Software devel-


opment and maintenance, Ar-
chitecture, Alternatives, Qual-
ity assurance and manage-
ment, Peer review, Prevention,
Organization process defini-
tion, Organization process im-
provement, Innovation, Prod-
uct evolution

A.3 IEEE 1062


Guidelines IEEE 1062

Project steering Planning organizational strat-


egy

Requirements management Defining the software re-


quirements

Risk management Defining the software re-


quirements

Project management Implementing organization’s


process

Planning and organizing Planning organizational strat-


egy, Implementing organiza-
© Daniel Svennberg 2001 131
A Frameworks coverage

tion’s process

Acquisition training Implementing organization’s


process

Contractor selection Implementing organization’s


process, Identifying potential
suppliers, Preparing contract
requirements, Evaluating pro-
posals and selecting supplier

Contract management Managing for supplier per-


formance

Product evaluation Accepting the software

Transition to support Not covered

Post partum Using the software

A.4 ISO 9000-3


Guidelines ISO 9000-3

Project steering Management responsibility

Requirements management Purchaser’s requirements


specification

Risk management Not covered

Project management Not covered

Planning and organizing Not covered

Acquisition training Not covered

Contractor selection Contract review

132 Software Acquisition Management Guidelines


A Frameworks coverage

Contract management Measurement, Purchasing, In-


cluded software product,
Management responsibility

Product evaluation Testing and validation

Transition to support Replication, delivery and in-


stallation, Maintenance

Post partum Not covered

Not covered Quality System, Internal qual-


ity system audits, Corrective
action, Development planning,
Quality planning, Design and
implementation, Configura-
tion management, Document
control, Quality records, Rules,
practices and conventions,
Tools and techniques, Training

A.5 ISO 12207


Guidelines ISO 12207

Project steering Acquisition, Joint review,


Management

Requirements management Acquisition, Development,


Verification

Risk management Management

Project management Management, Configuration


management

Planning and organizing Management, Tailoring, Veri-


fication

© Daniel Svennberg 2001 133


A Frameworks coverage

Acquisition training Training

Contractor selection Acquisition, Verification

Contract management Acquisition, Quality assur-


ance, Joint review, Audit,
Problem resolution, Verifica-
tion, Validation

Product evaluation Acquisition, Verification, Vali-


dation

Transition to support Not covered

Post partum Not covered

Not covered Supply, Operation, Mainte-


nance, Documentation, Infra-
structure, Improvement

A.6 ISO 15504


Guidelines ISO 15504

Project steering Acquisition preparation, Man-


agement, Project management

Requirements management Acquisition preparation, Re-


quirements elicitation

Risk management Risk management

Project management Configuration management,


Documentation, Problem reso-
lution, Management, Project
management, Measurement

Planning and organizing Management

134 Software Acquisition Management Guidelines


A Frameworks coverage

Acquisition training Human resource management

Contractor selection Supplier selection

Contract management Supplier monitoring, Verifica-


tion, Validation, Joint review,
Audit, Measurement

Product evaluation Customer acceptance, Verifica-


tion, Validation

Transition to support Not covered

Post partum Not covered

Not covered Operation, Customer support,


Development, System and
software maintenance, Quality
assurance, Quality manage-
ment, Organizational align-
ment, Improvement, Infra-
structure, Reuse

A.7 Euromethod
Guidelines Euromethod

Project steering Acquisition goal definition,


Acquisition planning

Requirements management Acquisition goal definition,


Tendering

Risk management Acquisition planning, Tender-


ing,

Project management Acquisition planning, Tender-


ing, Contract monitoring, Con-

© Daniel Svennberg 2001 135


A Frameworks coverage

tract completion

Planning and organizing Acquisition planning

Acquisition training Not covered

Contractor selection Tendering

Contract management Contract monitoring, Contract


completion

Product evaluation Contract monitoring

Transition to support Contract completion

Post partum Contract completion

136 Software Acquisition Management Guidelines


Index

CMMI ...........................................17
A code and fix .....................................8
Acquisition expert......................... 33 Collaborating contractor................37
Acquisition manager..................... 30 Collaborating customer .................30
Acquisition plan............................ 41 Commercial-off-the-shelf..............7
Acquisition proposal..................... 39 Common problems ........................13
Acquisition training ...................... 74 communications ...........................16
Administrative overload............. 15 configuration management.........71
Administrator................................ 32 Contract .........................................51
Alpha tests ................................. 109 Contract management....................96
Approach......................................... 3 Contract manager ..........................32
Artifacts ........................................ 38 Contract types................................88
Assisting roles............................... 33 Contracting environment...............26
Assmann ........................... 2, 3, 4, 85 contractor ... i, 2, 3, 9, 10, 11, 18, 33,
Audit....................................... 87, 98 34, 35, 36, 47, 50, 56, 57, 58, 59,
auditor ......................................... 100 64, 65, 73, 74, 81, 94, 97, 109
award incentives ........................... 88 Contractor development team
characteristics ............................27
B Contractor development team size 25
baseline ....................................... 103 Contractor evaluation criteria........49
best practices................................. 89 Contractor selection.......................85
Beta tests .................................... 109 Controlled processes ...................21
co-sourcing....................................12
C Cost and cost share.......................92
C/SCSC....................................... 102 cost incentives ...............................88
Capability Maturity Model ........... 17 Cost performance index ..............105
Capability Maturity Model Cost plus award fee.......................95
Integration ................................ 17 Cost plus fixed fee.........................93
Change request ............................. 55 Cost plus incentive fee..................94
Chaos” report ................................ 13 Cost plus percentage fee...............93
characteristics of software .............. 6 cost-reimbursable ..........................88

137
Index

COTS ................ 7, 18, 37, 40, 53, 84 FFP................................................89


CPAF............................................ 95 Finalization....................................57
CPFF ............................................ 93 Firm fixed price ............................89
CPI .............................................. 105 Fixed price plus award fee ...........92
CPIF ............................................. 94 Fixed price plus incentive fee ......91
CPPF ............................................ 93 Fixed price redetermination.........90
Creeping scope ............................ 15 Fixed price with economic price
Criticality ...................................... 23 adjustment.................................90
CS ................................................. 92 fixed-price .....................................88
customer i, 2, 7, 8, 10, 11, 12, 34, 35, forecast ........................................106
109 FPAF.............................................92
Customization factors ................... 21 FPEPA ..........................................90
Customization levels..................... 20 FPIF ..............................................91
Customize .............................. 70, 73 FPR ...............................................90
Fragmentation .............................15
D Frameworks ...................................16
D.I.R. ............................................ 21 Fraunhofer Institute ........................ ii
Deliverables .................................. 57 Frederick P. Brooks.........................1
Delivery time ................................ 24 Friction .........................................16
discipline ...................................... 16 Fully developed software ..............8
DOD-STD-2167A ........................ 18 Functional tests..........................109
DOD-STD-2168 ........................... 18
Domain expert .............................. 33 G
dyadic............................................ 11 Geographic dispersion...................28
Goldplating...................................15
E groups ...........................................73
Earned value ............................... 102
Effort............................................. 24 I
end user........... 10, 15, 81, 82, 83, 84 IEEE 1062 ......................... i, 4, 7, 18
End user ........................................ 34 IEEE/EIA 12207............................18
Error rate .................................. 101 incentives.......................................88
estimates................................. 85, 87 Inception........................................39
Euromethod................................. 19 Independent verification and
executive support ........................ 16 validation ...................................35
Experience with contractor........... 27 Innovation......................................23
experts........................................... 33 Installation tests.........................109
External risks .............................. 80 intellectual property rights.........67
eXtreme Programming ................... 9 ISO 12207 .....................................18
ISO 15504 .....................................19
F ISO 9000 .......................................18
FAA-iCMM ................................. 17 ISO 9000-3 ....................................18
feasibility...................................... 69 IV&V.....................................35, 108
Federal Aviation Administration.. 17
138 Software Acquisition Management Guidelines
Index

L Product evaluation plan.................54


Labor hours.................................. 95 Product evaluation report ..............55
Laissez-faire................................. 14 Program size ..................................22
Legal expert .................................. 34 Project and contract management
Linköping University.................. 1, ii risks...........................................81
Project management ......................69
M project parameters ......................67
Project specification ......................40
Maintenance and support staff...... 34
Project status metrics...................100
Maintenance plan.......................... 46
Project status report.......................56
Milestones .................................. 101
Project steering ..............................66
Minimal processes....................... 21
Proposal .........................................50
Modified-off-the-shelf................... 7
Monitoring .................................... 53 Q
MOTS ............................. 7, 8, 18, 40
multi-vendor.................................. 12 Q-Labs ........................................ 1, ii
quality assurance .........................71
N
R
nature of software ........................... 6
reality checks.................................31
O Recovery tests ............................110
Request for proposal......................47
Objective......................................... 2
requirements change rate ...........77
objectives ..................................... 16
Requirements change rate ........100
Organizations involved................. 26
Requirements management ...........76
Outline ............................................ 4
Requirements specification ...........44
Outsourcing..................................... 9
Requirements volatility .................22
Overpromising ............................ 15
resources.......................................16
overtime ..................................... 100
RFP ................................................47
P Risk identification .........................80
Risk list..........................................43
participants.................................... 10 Risk management ..........................78
Performance tests ..................... 110 Robust processes..........................21
phase ............................................. 38 Roles..............................................29
Planning and organizing ............... 72 root cause.......................................14
Post partum ........................... 58, 113
Post partum plan ........................... 57 S
problem analysis ......................... 72
SA-CMM ......................................17
Problem reports ........................ 101
Schedule performance index .......105
problem resolution...................... 71
Scope ...............................................3
Process group .............................. 61
SE-CMM .......................................17
Processes....................................... 60
Security tests ..............................110
Product evaluation ...................... 106
SETA .............................................35
Product evaluation approaches ... 109
sites ................................................28
© Daniel Svennberg 2001 139
Index

SMI............................................. 100 Technical expert ............................33


Software development .................... 8 Technical manager ........................32
Software maturity index .......... 100 test cases .....................................107
Solicitation.................................... 47 Testable requirements .................105
SPI............................................... 105 The Standish Group.......................13
Sponsor ......................................... 29 Time and materials.......................95
staffing ....................................... 100 Timeline.........................................62
Standard........................................ 18 TM/L.............................................95
Stress tests.................................. 110 To-Complete Performance Index 106
Subcontractor................................ 37 Total cost .......................................25
Supplier......................................... 37 Transformational impact ...............24
support and maintenance Transition to support ...................110
organization........................... 111 Translator.......................................34
support contractors ....................... 33 turnover .........................................27
Support tests.............................. 109 turnover rate ..............................100
SW-CMM .................................... 17
System engineering and technical U
assistance .................................. 35 use cases ........................................44
User interface tests ....................110
T user stories.....................................44
Target audience............................... 2
taxonomy .......................... 11, 12, 16 W
TCPI............................................ 106 waterfall...........................................9
team-building .............................. 70 WBS ............................................102
Technical and product work breakdown structure...........102
development risks ................... 82

140 Software Acquisition Management Guidelines

You might also like