482612main 2010 Tuesday 4 Cmmi Kulpa Margaret Executive Overview of CMMI v1.4b

www.agiledigm.
com
Empowering People through Process
Executive Overview of
CMMI
18 Aug 2010
NASA IT Summit
Margaret Kulpa
Chief Operating Officer of AgileDigm, Inc.
CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.
AgileDigm, Inc.
Copyright Notices
The presentation material in this seminar is copyrighted by
AgileDigm, Incorporated, 2010. For further information, please
contact:
AgileDigm, Inc.
11 Twelve Oaks Trail
Ormond Beach, Florida 32174 U.S.A.
Phone: +1.386.673.1384
Email: info@agiledigm.com
Terms like these are often used in the following material:
CMMI
SCAMPIsm and SCAMPI Lead Appraisersm
SM SCAMPI and SCAMPI Lead Appraiser are service marks of Carnegie Mellon University.
2010
Executive Overview of CMMI
Who is AgileDigm?
AgileDigm, Incorporated is a US based
corporation working with organizations around
the world
providing consulting, training, and
appraisals for organizations that build
software-intensive systems
Software Engineering Institute (SEI)
Partner
Scrum Training Institute (STI) Partner
Authors of Interpreting the CMMI, best
selling book on using the CMMI.
Unique experience working with one of the
only CMMI Maturity Level 5 Agile companies.
SCAMPI and SCAMPI Lead Appraiser are service marks of Carnegie Mellon University.
SM
2010
What is the CMMI?

CMMI is an acronym for Capability Maturity Model
Integration.
Approach used by organizations to improve.
CMMI is a collection of best practices
Over 100 organizations formally assisted in its development,
including NASA
There is a wide range of freedom in how an organization

decides to embrace the CMMI. You can
Go slow or fast
Choose the areas you need to focus on
Choose the appraisal mode you are comfortable with (SCAMPI A,B,C, High Maturity)
2010
Three Constellations
CMMI for Development
Development and maintenance of products
Integrates software engineering, systems engineering,
collaborative teams, acquisition from the suppliers side
Not just for software development!
Can be used for developing complex systems (healthcare,
bridges, valves, cleanroom technology)
CMMI for Services

Organizations that provide services (from taxis to call centers)
CMMI for Acquisition

Acquiring products and services
Common Process Areas are embedded within all three

This presentation focuses on CMMI for Development.
2010
CMMI Concepts
What the CMMI is not:
CMMI is not a process, standard, directive,
regulation, or description.
Not a list of Shalls that might require big
changes in how you do your work.
You can make the CMMI in your organization

as intrusive and painful as you want --- OR
NOT! Its up to you.
2010
Does the CMMI tell me how to do stuff?

NO!
The CMMI documents best practices from many
organizations.
The CMMI describes WHAT to do, not HOW to
do it.
Organizations need to determine how to apply
the best practices of the CMMI to how the
organization works in order to improve how stuff
gets done.
2010
Why Should I Use the CMMI?

Structures the effort (Where do I start? How
do I know when I am done?)
Helps identify an organizations improvement
objectives and priorities
Provides benefit from a larger communitys
experience
Has an an appraisal methodology to diagnose
the state of improvement efforts.
2010
Why Should I Use the CMMI?

CMMI focuses on Process
A process is a set of practices performed
to achieve a given purpose (from CMMI
Glossary)
Four major elements of process are tools,
methods, materials, and people
Major determinants of product cost, schedule,
and quality are tools, methods, materials, and
people.
2010
CMMI Performance Results Summary

This is the summary of improvements from all sources collected
regardless of maturity levels.
Improvement
Category
Nbr. of Data
Points
Median
Lowest
Highest
Cost
34%
3%
87%
29
Schedule
50%
2%
95%
22
Productivity
61%
11%
329%
20
Quality
48%
2%
132%
34
Customer
Satisfaction
14%
-4%
55%
4.0 : 1
1.7 : 1
27.7 :1
22
Return on
Investment
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report CMU/SEI2006-TR-004
2010 10
Cost Benefits and Impact

Cost metrics (and reductions in cost) defined as:
Cost of delivery
Cost of quality/Cost of poor quality
Overhead rate
Costs of rework/Defect find & fix cost
Software unit costs
Number/Cost of process staff
Variation in cost performance index (CPI)
Organizations have reported improved

Budget estimation accuracy
Average in cost performance index (CPI)
Cost example on next slide cost per
function reduced by 52%
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006, Technical Report
CMU/SEI-2006-TR-004
2010
11
Cost Savings at DB Systems GmbH
CMU/SEI-2006-TR-004
2010
12
Schedule Benefits and Impact

Schedule metrics (and reductions) defined as:
Variation in schedule
Schedule performance index (SPI)
Number of days late
Days variance from plan
Slippage of project delivery
Organizations have reported improved or

increased
Cycle time
Average SPI
Proportion of milestones met

Estimation accuracy
Example from NCR Corporation (on next slide)

Average days schedule variance reduced from approximately
130 days to less than 20 days one year after reaching CMMI
maturity level 2
CMU/SEI-2006-TR-004
2010
13
Schedule Improvement at NCR
CMU/SEI-2006-TR-004
2010
14
Productivity Benefits and Impact

Productivity measures (and improvements)
defined as:
Time comparisons by build Source statements per month

Lines of code per labor hour Number of releases per year
Testing rates
Software production
Function points (FP) per full time equivalent (FTE) staff
Example from IBM Australia (on next slide)

Over 20 percent improvement in account productivity as the
organization moved from SW-CMM maturity level 3 toward
CMMI maturity level 5.
CMU/SEI-2006-TR-004
2010
15
Productivity Improvement at IBM Australia

Account Productivity (FP/FTE)
CMU/SEI-2006-TR-004
2010
16
Quality Benefits and Impact

Quality most frequently defined as reductions
in numbers of defects.
Measurements include:
Defect counts by phase of injection
Defect counts by phase of discovery
Total defect density
Example from IBM Australia (on next slide)

40 percent reduction in overall production and over 80 percent
reduction in Severity 1 problems, as the organization moved
from SW-CMM maturity level 3 toward CMMI maturity level 5
CMU/SEI-2006-TR-004
2010
17
Quality Improvement at IBM Australia
CMU/SEI-2006-TR-004
2010
18
Customer Satisfaction Benefits and Impact

Customer satisfaction most frequently defined by
the results of customer surveys.
Award fees are sometimes used as surrogate
measures.
An example from Lockheed Martin Management

and Data Systems
Increased award fees by 55 percent
CMU/SEI-2006-TR-004
2010
19
ROI Benefits and Impact

ROI defined by cost avoidance measures
and improvements including:
Rework avoided due to fewer defects
Improved productivity
Increased revenue due to shorter cycle times
Example from Raytheon Corporation,

anonymous site
6:1 ROI in a CMMI maturity level 3
organization
Source: Performance Results of CMMI-Based Process Improvement, Aug 2006,
Technical Report CMU/SEI-2006-TR-004
2010
20
From personal experience

Ensuring I am working on the latest (correct) version of
the code
Running tests that prove requirements are satisfied
(not that the system is just not blowing up)
Predicting the number of projects to start up and the
number of people I need ahead of time
Increasing communication throughout the organization
(and hearing things I sometimes did not want to hear)
Not making phone calls to my QA person while she
was in labor.
2010
21
CMMI Structure
CMMI is a collection of best practices
These best practices reside in 22 focus areas called Process
Areas.
Each process area has specifically stated goals that have to be met
in order to have maturity or capability in the process area.
Each process area has specific practices, subpractices, typical work
products (outputs), examples, and other informative material
Most of the goals are specific to the process area, but there are
some generic or global goals that are common to all of the
Process Areas like:
Provide Resources for this area
Train People for this area
Identify Stakeholders for this area
Lets look at the book
2010
22
Structure of the CMMI

Maturity Level or Capability Level
Specific and Generic Goals
Specific and Generic Practices
Subpractices, Typical Work Products, Examples
Two types of Goals and Practices
We will see these examples in the Example Process
Area
Specific Goals pertain to one Process Area (SG1 in Project
Planning is Establish Estimates)
Specific Goals have associated Specific Practices (For SG1
above, SP 1.4 is Determine Estimates of Effort and Cost
There are also Generic Goals and Generic Practices that pertain to
all of the Process areas throughout the model. They are Global.
2010
23
Example Process Area (Maturity Level 3)

Decision Analysis and Resolution
The purpose of Decision Analysis and Resolution (DAR)
is to analyze possible decisions using a formal
evaluation process that evaluates identified alternatives
against established criteria.
The CMMI then continues with half a page of Introductory Notes that
further describe what this Process Area is about.
Note: Underlines and emphasis added by the presenter here and in
the following slides. I have omitted some of the Informative Material.
Informative material is very useful in interpreting the intent of the
process area/practice/subpractice and should not be ignored.
2010
24

SG1: Evaluate Alternatives
SP1.1 Establish Guidelines for Decision Analysis

SP1.2 Establish Evaluation Criteria
SG Specific Goal
SP1.3 Identify Alternative Solutions
SP Specific Practice
SP1.4 Select Evaluation Methods ***
SP1.5 Evaluate Alternatives
SP1.6 Select Solutions
Note: Other Process Areas may have more Specific

Goals and more Specific Practices. There is more
informative material presented in the book.
***This is the practice used in the following example.
2010
25

SP 1.4 Select the Evaluation Methods
Typical Work Products:
1. Selected evaluation methods
Subpractices
1. Select the methods based on the purpose for analyzing a decision
and on the availability of the information used to support the
method.
Typical evaluation methods include the following:
Modeling and simulation
Engineering studies
Manufacturing studies
Cost studies
Business opportunity studies
2010
26

Surveys
Extrapolations based on field experience and prototypes
User review and comment
Testing
Judgment provided by an expert or group of experts
2. Select evaluation methods based on their ability to focus on the
issues at hand without being overly influenced by side issues.
3. Determine the measures needed to support the evaluation method.
Consider the impact on cost, schedule, performance, and risks.
2010
27
Generic Common Goals and Practices for All

Process Area
Generic Goal 2 Institutionalize a Managed Process
GP 2.1
Establish an Organizational Policy
GP 2.2
Plan the Process
GP 2.3
Provide Resources
GP 2.4
Assign Responsibility
GP 2.5
Train People
GP 2.6
Manage Configurations
GP 2.7
Identify and Involve Relevant Stakeholders
GP 2.8
Monitor and Control the Process
GP 2.9
Objectively Evaluate Adherence
GP 2.10
Review Status with Higher-Level Management
GG Generic Goal
GP -Generic Practice
Generic Goal 3 Institutionalize a Defined Process
GP 3.1
Establish a Defined Process
GP 3.2
Collect Improvement
Information
2010
28
Two representations of the model

Staged Representation (or View):
Most like other currently accepted models

Focuses on all process areas contained within a level
Used for Maturity Level ratings for contract awards
Start with any Process Area within a level.
Continuous Representation (or View):

Was designed to focus on specific, individual process areas
Process areas are arranged into functional categories
Start with any Process Area.
Staged is the most popular CMMI representation.

When your customers ask for a level or when an organization
says We are Level N, they are referring to a Maturity Level of the
staged representation.
2010
29
Not to Worry
The information contained is the same in
both!
The only difference is in your approach for
implementation and appraisal.
2010
30
Process Areas by Maturity Level (Staged)

Maturity Level
5 Optimizing
4 Quantitatively Managed
3 Defined
2 Managed
1 Initial
Process Areas
Organizational Innovation and Deployment
Causal Analysis and Resolution
Organizational Process Performance
Quantitative Project Management
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition +IPPD
Organizational Training
Integrated Project Management +IPPD
Risk Management
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process and Product Quality Assurance
Configuration Management
2010
31
Process Areas by Category (Continuous)

Process Management
Project Management
Organizational Process Focus

Organizational Process Definition (with IPPD)
Organizational Training
Organizational Process Performance
Organizational Innovation & Deployment
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Integrated Project Management (with IPPD)
Risk Management
Quantitative Project Management
Engineering
Support
Requirements Management
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Measurement & Analysis

Process and Product Quality Assurance
Configuration Management
Decision Analysis & Resolution
Causal Analysis and Resolution
2010
32
CMMI
Staged
Maturity Levels
Level 1 Initial
Level 2 Managed
Level 3 Defined
Level 4 Quantitatively Managed
Level 5 - Optimizing
Continuous
Capability Levels
Level 0 - Incomplete
Level 1 Performed
Level 2 Managed
Level 3 Defined
Level 4 Quantitatively Managed
Level 5 - Optimizing
Process Management
Project Management
Engineering
Support
Process Areas (22)

There are no
shalls.
Appraisals
yield
Ratings
Its a Model,
not a specification
Process Areas (22)
Active PA Subset
Specific
Goals
Realm
of
Appraisals
Specific
Practices
Generic
Practices
2010
Generic
Goals
33
It can be hierarchical.
Its highly detailed..
Its complex, but

nothing is complicated.
What is a SCAMPIsm Appraisal?

SCAMPIsm Standard CMMI Appraisal Method for
Process Improvement
Used for baselining current behaviors to begin
improvement activities or see into ongoing
improvement activities
To assign a formal rating for contract award or fees (if
requested by appraisal sponsor)
Several appraisal methods (class A, B, C, plus High
Maturity)
Uses the CMMI as reference model.
SCAMPI goal is to provide an accurate picture of your
organization relative to the reference model.
2010
34
Reported Appraisals
CMMI For Development SCAMPIsm Class A Appraisal Results 2009 End-Year Update
Red indicates countries with reported SCAMPI Class A Appraisals
2010
35
Maturity Profile End of Year 2009

The percentage of organizations at each maturity level
based on most recent appraisals of 2753 organizations.
Maturity Profile - CMMI/SCAMPI Results published
March 2010
70.0%
63.0%
60.0%
50.0%
40.0%
25.2%
30.0%
20.0%
10.0%
1.3%
5.2%
Level 4
Level 5
0.5%
0.0%
Level 1
Level 2
2010
Level 3
36
Appraisal Classes A, B, and C
2010
37
Appraisal Team (SCAMPI A)

Minimum team size is four - A Lead Appraiser
and three additional team members.
At least one team member must be from the
appraised organization.
All team members must be trained in the official
CMMI and SCAMPI courses
Conflicts of interest must be avoided
Exclude process authors and individuals directly
impacted by the outcome of the appraisal
Team members cannot be managers of any of

the selected projects or groups
Appraisal sponsors cannot be team members.
2010
38
What does the SCAMPI Team do?
Reviews an organizations documentation
Conducts interviews
To ensure the documentation is actually used
Maps results to the CMMI
Processes, procedures, policies, status reports,

meeting minutes
Practices, Goals, Process Areas
Assigns ratings (if requested and allowed by

the type of SCAMPI).
2010
39
What does the Team NOT do?

Does not go looking for the documentation
Easter egg hunts are not allowed!
Does not judge the goodness of the

process
Does not judge how well you are doing
your job.
2010
40
PPQA Example Evidence

ID
Goal/Practice
Potential Artifacts
SG1
Objectively Evaluate Processes and Work Products
SP1.1
Objectively Evaluate Processes QA audit report
SP1.2
Objectively Evaluate Work

Products and Services
SG2
Provide Objective Insight
SP2.1
Communicate and Ensure

Resolution of Noncompliance
Issues
QA audit report
SP2.2
Establish Records
QA Reports, Status of
Corrective Actions, Report
of Quality Trends
2010
41
QA audit report
Simplified Point of the SCAMPI:

Do you have a (documented) process?
Do you follow it?
How closely does it map to the CMMI?
2010
42
Conclusion (1of 2)
Organizations can and have benefited from using the
CMMI.
Implement the CMMI to support your organizations
business objectives, goals, and culture.
Engage your brain. Interpretation of the information
must be used.
Do not ignore the informative information and the
subpractices. They help to understand the intent of the
practice or Process Area.
Use a SCAMPI to guide your implementation, measure
your progress, and verify your achievements.
2010
43
Conclusion (2 of 2)
My favorite websites for more information:
www.sei.cmu.edu (main SEI website)
seir.sei.cmu.edu (contains helpful
presentations and articles from the
community)
2010
44
Questions?
Time for some questions?
2010
45
Contact Information
The presentation material in this seminar is copyrighted by
AgileDigm, Incorporated, 2010. For further information, please
contact:
Margaret Kulpa
Chief Operating Officer
AgileDigm, Inc.
11 Twelve Oaks Trail
Ormond Beach, FL 32174 USA
+1 386 673 1384
Margaret.kulpa@agiledigm.com
www.agiledigm.com
2010
46
Backup Slides
2010
47
Combined View of Staged and Continuous
2010
48
Maturity Levels (MLs)

A maturity level signifies the level of
performance that can be expected from an
organization. There are 5 maturity levels.
Following are the key features of each maturity
level (ML):
ML 1 (Initial) ad hoc processes

ML 2 (Managed) basic project management system
ML 3 (Defined) process standardization
ML 4 (Quantitatively Managed) quantitative
management by the numbers
ML 5 (Optimizing) continuous process improvement
ML 2 provides a foundation for ML 3, ML 3 for ML4, and ML 4 for ML 5.
2010
49
Appraisal Sponsor
SCAMPI appraisals require an appraisal sponsor.
Usually this is a senior manager.
The appraisal sponsor is responsible for
Verifying that the appraisal team leader has the
appropriate experience, knowledge, and skills to
lead the appraisal.
Ensuring that the appropriate organizational units
(projects and support) participate in the appraisal.
Supporting the required confidentiality and nonattribution rules.
Some appraisals have multiple sponsors.
2010
50
Typical Appraisal Roadmap

May 2010
Plan
Plan SCAMPI activities

Monitor and adjust plan
August 2010 (1 Week)

Train Team
Train Appraisal Team

Begin PIID creation
Finalize Plan
Organization Develops PIID documents
October 2010 (1 Week)

Reviewing PIIDs is a key focus of
the required readiness review
Readiness
Review
December 2010 (1 3 Weeks)
Conduct Appraisal
2010
51
Appraisal Onsite
Presenter Margaret Kulpa

Chief Operating Officer of AgileDigm,
Incorporated
She is the primary author of the book
Interpreting the CMMI: A Process Improvement
Approach, Second Edition, Auerbach (2008)
which details how to identify, define, and
improve business, systems and software
processes on both enterprise-wide and project
levels.
Ms. Kulpa has developed and taught courses in software quality
assurance, project management, peer reviews, requirements
development and management, process baselines and modeling,
safety and security, and other software engineering-related
subjects. She has developed courses in CMMI and has served as
an external member of numerous SCAMPIsm Class A Appraisals.
2010
52

482612main 2010 Tuesday 4 Cmmi Kulpa Margaret Executive Overview of CMMI v1.4b

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

482612main 2010 Tuesday 4 Cmmi Kulpa Margaret Executive Overview of CMMI v1.4b

Uploaded by

Copyright:

Available Formats

www.agiledigm.

What is the CMMI?

There is a wide range of freedom in how an organization

CMMI for Services

CMMI for Acquisition

Common Process Areas are embedded within all three

You can make the CMMI in your organization

Does the CMMI tell me how to do stuff?

Why Should I Use the CMMI?

Why Should I Use the CMMI?

CMMI Performance Results Summary

Cost Benefits and Impact

Organizations have reported improved

Cost Savings at DB Systems GmbH

Schedule Benefits and Impact

Organizations have reported improved or

Proportion of milestones met

Example from NCR Corporation (on next slide)

Schedule Improvement at NCR

Productivity Benefits and Impact

Time comparisons by build Source statements per month

Example from IBM Australia (on next slide)

Productivity Improvement at IBM Australia

Quality Benefits and Impact

Example from IBM Australia (on next slide)

Quality Improvement at IBM Australia

Customer Satisfaction Benefits and Impact

An example from Lockheed Martin Management

ROI Benefits and Impact

Example from Raytheon Corporation,

From personal experience

Structure of the CMMI

Example Process Area (Maturity Level 3)

Decision Analysis and Resolution

SP1.1 Establish Guidelines for Decision Analysis

Note: Other Process Areas may have more Specific

Decision Analysis and Resolution

Decision Analysis and Resolution

Generic Common Goals and Practices for All

Establish an Organizational Policy

Plan the Process

Identify and Involve Relevant Stakeholders

Monitor and Control the Process

Objectively Evaluate Adherence

Review Status with Higher-Level Management

Generic Goal 3 Institutionalize a Defined Process

Establish a Defined Process

Executive Overview of CMMI

Two representations of the model

Most like other currently accepted models

Continuous Representation (or View):

Staged is the most popular CMMI representation.

Process Areas by Maturity Level (Staged)

Process Areas by Category (Continuous)

Organizational Process Focus

Measurement & Analysis

Executive Overview of CMMI

Process Areas (22)

Process Areas (22)

Executive Overview of CMMI

Its complex, but

What is a SCAMPIsm Appraisal?

Maturity Profile End of Year 2009

Appraisal Classes A, B, and C