2,000 Ways to Shine

July 2000

The Software Engineering Institutes process improvement meeting is a

must-see.
by Victor Stachura
Are you interested in software process improvement? Have you been charged with leading
your organization to Capability Maturity Model level two, three, four or five? Then you
shouldnt miss the Software Engineering Institutes (SEI) conference on software process
improvement. This years focus at "SEPG 20002,000 Ways to Make Software Better" was on
the practical ways to improve software quality. Were 2,000 ways to do so actually presented?
Im not sure, but the four-day event is certainly the premier gathering for people who want to
increase the maturity of their development organizations.
SEPG 2000 was held in downtown Seattle from March 20 to 23, with a backdrop of Mt. Rainier,
the Cascades and the Olympic Mountains. Attendance was the best ever, with approximately
2,100 people filling the halls of the Washington State Convention and Trade Center. The
conference consisted of 97 different presentations and tutorials divided into three categories:
"Getting Started," "Progressing" and "Cutting-Edge." For the novice, the "getting started"
presentations were perfect. For those of us who have been doing software process
improvement for a while, the "progressing" sessions were extremely helpful, and the "cuttingedge" tracks were real eye-openers.
Two of the four days were reserved for 49 exhibitors. Vendors were classified either as
consultants (those willing to help you achieve higher levels of process maturity) or toolmakers (estimating, assessment tracking, requirements and workflow management). As I
strolled through the exhibitor hall, I decided to take an informal poll of those vendors showing
software products. I would walk into a booth and ask, "What CMM level has your organization
been assessed at?" If youre showing software at a conference where the CMM is the focus,
then you had better have some sort of process improvement program in place or, better yet,
have been assessed at some level of the CMM. To my dismay, I was met with surprise by most
of the vendors. The responses were generally "Gee, no one has ever asked me that before," or
"I dont know; Id better ask my colleagues." Of course, this was an unscientific survey, but
the results were interesting.
Three or four vendors were not rated or assessed at any level of the CMMor certified by any
model, for that matter. These people said things like, "We didnt want to take the time for full
certification" or "We have strict internal processes but have not implemented any activities of
the CMM." Lets see, your tool is supposed to help me with my process-improvement efforts,
but your organization, or at least the developers of your product, have not formally
implemented key process-improvement activities? One vendor was ISO-certified (cool), two
were CMM level three (excellent) and one was CMM level five. Level five? The nirvana of
process improvement! I had to be sure, so I asked, "Were the developers of the product youre
showing here part of the group assessed at CMM level five?" "Yes" was the answer. What
company was this? I-Flex Solutions Ltd., a vendor from India.
Tools on Display
The tool vendors were varied, with products covering configuration management, estimating,
source code analysis based on quality criteria, requirements management, process asset
repositories, software process management, defect tracking and metrics analysis. If you dont
consider yourself a proficient user of process tools, its time to reconsider your position. These
tools can improve your productivity and software quality. S. Ramadorai, chief executive officer

of TATA Consultancy Services (TCS), a firm with offices throughout India, mentioned in his
keynote speech that his organization was able to reduce the amount of project management
time to an average of five percent of the total project effort by using project management
tools. In his view, their project leaders were able to focus their efforts on managing the
project instead of spending excessive amounts of time on the "administrative" aspects.
The Keynotes
There were four interesting speeches. Scott Griffin, chief information officer for the Boeing
Company, described the process-improvement activities that his company began in 1991.
More than 10,000 people at Boeing have been trained in the Software CMM, over 200
assessments have been performed, and several Boeing organizations are at CMM levels three,
four and five. Boeings results are impressive. The accuracy of project estimates has improved
by 87 percent, software quality has improved by 130 percent and productivity has improved
by 62 percent.
Steve McConnell, president and chief software engineer for Construx Software and editor in
chief of IEEE Software magazine, made a compelling case for creating a true profession of
software engineering. Several high-profile studies have shown that the software industry has
some serious problems. Projects are canceled at an alarming rate, and software is routinely
late and over budget. The popular press chuckles about the "blue screen of death," but this is
really no joke. Interestingly enough, McConnell points out that the industries best practices
have been around for 15 years or more. A few examples include automated estimation tools,
introduced in 1975, JAD sessions from 1985, information hiding from 1972 and inspections
from 1976. So what is the problem? The varying rate of the diffusion of information. People
and organizations adopt technology and techniques at different rates, ranging from innovators
to laggards. Increasing the rate at which companies adopt industry best practices is one way
the overall quality of software could improve. McConnell believes that creating a true
profession of software engineering is the best way to advance the industry. Steps are under
way to make this a reality, with new undergraduate, accreditation and licensing programs
becoming available.
James R. Driscoll, vice president for EDSs Wireless Division, delivered a keynote address
entitled "Ensuring Clients Achieve Superior Value in the Digital Economy." Driscoll described
key components of the digital economy and stressed that companies must be fast, flexible and
innovative in order to survive. Competition is changing, with greater value being placed on
knowledge and innovation, and markets overall are becoming more efficient. To effectively
compete and meet the demands of the digital economy, EDS is applying software process
improvement on a global scale. All EDS organizations use a single consistent process set for
software development and the management of the development process. By implementing a
corporate process repository, regional estimating centers and a CMM lead assessor network,
EDS can maximize the return on investment for process improvement and effectively compete
globally.
Ramadorai presented a view of his companys drive toward CMM level five. TCS, Indias largest
IT consultancy, has seven centers at CMM level five and one at level four. By applying the CMM
and quality concepts, they have achieved an estimate accuracy of plus or minus five percent
and a schedule accuracy of plus or minus 0.09 percent.
High-Maturity Organizations
This year it was evident that there are more and more high-maturity organizations in the
industry. High maturity is defined as those organizations at CMM level four and level five. (See
the characteristics of these organizations in Figure 1.) The SEI performed a global survey in
1999 of such firms and found, as of November 1999, 40 level-four organizations and 21 levelfive organizations. Of these, 26 are offshore, with 24 in India and one each in Australia and
Israel. India alone has 14 level-four organizations and 10 at level five. In The Decline and Fall

of the American Programmer (Prentice Hall, 1992), Ed Yourdon makes the case for India
taking market share from American programmers, just as Japan did from U.S. auto makers in
the 1970s. It does seem that India is embracing the CMM as a competitive advantage over the
U.S. software industry.
Figure 1. Characteristics of High-Maturity Organizations
Project
Management

Many use Delphi methods for estimating and Earned Value for project
tracking.

Risk Management

Use a "top 10 risk" list.

Risk management is a high priority task.

Software Sizing

Typically use lines of code, though some use function points.

Quality

Process assurance is performed by a software quality assurance group.

Product assurance is performed by the development teams.

Processes

Most use process notations, and process models.

Most use the Web (intranet) to deploy their processes.
Most perform satisfaction surveys about their processes.

Training

Most have formal mentoring programs in place.

Source: "Understanding High Maturity Practices: A Software CMM Tutorial," Mark C. Paulk,
SEPG 2000
Watts Humphreys New Process
There were several presentations on the Personal Software Process (PSP) and the Team
Software Process (TSP), both developed by Watts Humphrey to improve software quality and
increase developer productivity. Basically, the PSP is about planning for quality and removing
defects early in the development cycle, and is designed to fit within your current development
methodology. Each defect is recorded, tracked and analyzed to determine how to improve the
development process.
Several people presented defect and productivity data before and after the PSP and TSP was
implemented. The results are impressive. Boeing, Hill Air Force Base, AIS and Teradyne have
all published data on their PSP and TSP pilots, as summarized in Figure 2.
Figure 2. Results of the Personal and Team Software Processes
Category

Without PSP/TSP

With PSP/TSP

Average schedule deviation

27% to 112%

-8% to 5%

Average effort deviation

17% to 85%

-8% to -4%

Defects per 1,000 LOC*

0.1 to 0.7

0.02 to 0.1

System test savings per 1,000 LOC

1 to 5 days

0.1 to 1 days

Number of post-release defects per 1,000 LOC

0.2 to 1+

0 to 0.1

Boeing, Hill Air Force Base, AIS and Teradyne have all published data on the
improvements seen from their Personal Software Process and Team Software
Process pilot programs.
* Lines of code
Chart adapted from the presentation "A Quantitative Approach to Software Quality
Management," SEPG 2000.
Where Were the "Dot-Coms"?
Looking at the roster of attendees, I noticed that a certain type of software company was
missing from the conference. There were only a handful of dot-com or Internet companies.
Such powerhouses as Yahoo, AOL, Sun Microsystems, Microsoft and Dell had no representation
whatsoever. Naturally, there were a large number of defense agencies, defense contractors,
aerospace companies and the larger commercial service companies, namely EDS, IBM, CSC,
Anderson Consulting and others. The documented charter of the SEI is to "advance the state
of the practice of software engineering" (The Capability Maturity ModelGuidelines for
Improving the Software Process, Carnegie Mellon University Software Engineering Institute,
Addison Wesley, 1994). I believe the SEI is fulfilling this goal, as evidenced by the popularity
of the CMM and the data presented at the conference. But there were many organizations that
should have attended the conference. Why do we accept the "blue screen of death" and
operating systems with bugs listed in the tens of thousands? I sometimes wonder when I hear
glowing government reports stating that productivity is increasing because of technology. If
so, why must I reboot my PC several times a week?
All in all, this years SEPG conference was worthwhile. Each presentation I attended was high
quality, both in terms of content and delivery. Most of the time I just couldnt take notes fast
enough; much of this years focus on the characteristics of higher maturity organizations
(those at level four and level five), along with the benefits of applying the PSP and TSP, was
especially relevant to my organizations needs.
I recommend the SEPG conference to anyone who is even remotely interested in software
process improvement. Better still, next years conference is being held in New Orleans. I can
hardly wait.