Understanding Software Engineering Failure as Part of the SWEBOK

Ira Monarch
SofnYare Engineering Institute (SEI)
Carnegie Mellon University
Pittsburgh, PA 15213-3890
e-mail: iam @sei.cmu.edu

The Guide to the Software Engineering Body of Knowledge (GSWEBOK) is a good first step in
characterizing the contents of the software engineering discipline and in providing a topical
access to the Software Engineering Body of Knowledge (SWEBOK). However, the GSWEBOK
is seriously lacking as a guide to how the S W B O K can be used to explore system failures as a
way of enhancing practitioner’s preparations for the software engineering profession. This lack
is not limited to the Guide. This is why it is very important, in order that software system failure
may be understood from a software engineering point of view, that a new conception of what
the SWEBOK is, as well as new approaches to its access, need to be developed.

1. Including the literature of software and software engineering failure

The Guide’s conception of the SWEBOK as existing solely in traditionally published

literature needs to be extended significantly if the common factors that lead to failure of
installed systems and systems under development, are to be identified and better understood. It
will be important that the SWEBOK includes information about how software failure is
produced by, or at least associated with, software engineering failure. Because the Guide has a
limited conception of the SWEBOK, very little of the knowledge on software engineering
failure and its relation to software failure is mobilized.

2. Need for software engineering reflection on software faults and failures

The Guide makes a distinction between ‘fault’ meaning the cause of a software system
malfunctioning and ‘failure’ meaning the undesired effect observed in a software system
delivered service. The Guide’s introduction recognizes that software without the requisite
features and degree of quality is an indicator of failed (or at least flawed) software engineering,
but nowhere explores this further. The chapter on software quality covers software failure but
only in the sense of failure as the result of faults found in testing. Even though the chapter
distinguishes such failures from mistakes, defined as human actions that produce incorrect
results, it relegates mistakes in this sense to the knowledge area of software engineering process
and refers the reader to the chapter on that topic. However, there is nothing in the latter chapter
on mistakes or failed software engineering as such. What is described is software engineering
process know how or best practice. Nothing is said about how in actual practice software faults
leading to failures can be determined by software engineering failures or how individuals and
groups can learn to improve their software engineering practices by better understanding how
software engineering failure leads to software failure.

$10.000 2001 IEEE

0-7695-1059-0/01 191
192

3. Enhancing the information architecture of the SWEBOK

In addition to extending the conception of the SWEBOK to include new kinds of knowledge,
a different approach will also have to be taken to its organization and the way it is accessed.
Access to topics organized hierarchically will not be enough. Access must be more fine-grained.
To make knowledge on software engineering failure accessible and relevant in practice,
knowledge of failure will need to be analyzed into concepts more detailed than topics and cross-
referenced in myriad ways with concepts extracted from SWEBOK knowledge areas. Expertise
in using text mining tools has progressed to the extent that such conceptual analysis is possible.
A key will be to get enough information sources online. It will then be possible to identify and
link concepts involved in articulations of software engineering know how and best practice with
concepts involved in articulations of cases of software and especially software engineering
failure. Establishing such links will enable new and long-time practitioners to better
appreciate the limitations and modifications needed to put software engineering know how into
practice. Such limitations can be gleaned from, for example, the Ariane 5 software reuse failure,
failures to achieve various CMM levels and failures in technologically innovative contexts like
software development and acquisition at Internet speed.

4. The Software Engineering Institute’s work on software engineering failure

The Software Engineering Institute (SEI) is in mid phase of an Independent Research and
Development (IR&D) project to investigate the feasibility of reconceptualizing the software
engineering body of knowledge to include understanding of the relation between failed software
products and failed software engineering processes. The objective is to identify repeatable
factors constituting software failure pattems to help diagnose and recommend remedies for
projects and organizations who are failing or who have failed. Information sources being
considered come not only from the published literature but also public repositories of
information like the Risk Digest, Centre for Army Lessons Learned, Commercial Aircraft
Computer Incident Archive and more private repositories like the ones at SEI in process, risk
identification and intervention consulting work on troubled government projects. In addition to
mining software engineering ‘‘literature’’ in a much more encompassing sense, the IR&D project
is investigating how to use various forms of fieldwork on an ever-increasing number of sites to
extend and validate software failure pattems.