Practical CMM

March 2001

Scoping your improvement program based on the problems and goals of

your organization allows you to make significant headway.
by Neil Potter and Mary Sakry
The capability maturity model (CMM), developed by the Software Engineering Institute (SEI),
is a model to help software organizations improve their development capability. Acceptance
and use of the CMM for process improvement varies greatly in the industrysome use it as an
optional guideline, while others revere it as gospel. When used appropriately, CMM can help
organizations develop software with predictable cost, schedule and quality. When used
inappropriately, it can waste an entire organization's time.
Many organizations approach process improvement by simply documenting each and every
process. This process-centric approach may be amplified when an organization attempts to
adopt a more sweeping and systematic technique for improvement such as ISO9001 or the
CMM. In light of a process-centric goal stating, "Be SEI CMM Level 3 by December," the
approach of documenting all processes is reinforced and might even appear natural. A
process-centric approach can work, but it has a high risk of failure because most people
mistake documentation for progress. The improvement effort isn't integrated with the
organization's product development goals, and this usually results in a large stack of unused
process documents.
The capability maturity model (version 1.1) consists of five levels of engineering and
management process maturity, each level building on the next. CMM Level 1 has no criteria
and represents projects that typically have large amounts of rework, numerous technical
surprises, frustrated customers, and significant cost and schedule overruns. Good software can
come from a CMM Level 1 organization; however, this result isn't easy to achieve or sustain.
CMM Level 2 comprises sound project management, negotiations with the customer, version
control, vendor management, and simple process and product assurance. Organizations at
CMM Level 2 can focus on product development rather than day-to-day crises. CMM Level 3
focuses on organization-wide engineering skills, basic systems engineering, advanced project
management, and an infrastructure to support sustained improvement. Organizations at CMM
Level 3 consistently produce reliable software on time. A complete copy of the CMM can be
found at www.sei.cmu.edu/pub/documents/93.reports/pdf/tr24.93.pdf.
Goal-Problem
Approach
An alternative to the process-centric approach of improvement is to scope your improvement
program based on the problems and goals of your organization. Adopting this approach
enables you to make significant progress on real issues and make headway using the CMM.
Examples of an organization's business goals include the delivery of a product, the completion
of a software installation or the upgrade of a database. The goal could also be the desired
outcome when a critical problem has been solved. For example, an organization may be
unable to hit delivery deadlines, or it might spend 75 percent of its resources on rework. The
related goals to these problems would be meeting deadlines 100 percent of the time or
reducing rework to 25 percent.
To use the goal-problem approach, first determine your organization's business goals and
problem areas, and then compare these to the elements of the SEI CMM. Select the
appropriate elements of the CMM that address these problems and help you move towards
your goals.

During one session to help a company plan an improvement program, we learned that the
organization was about to form six teams to work on the six key process areas (KPA) of CMM
Level 2, which are requirements management (RM), software project planning (SPP), software
project tracking and oversight (SPTO), software configuration management (SCM) and
software quality assurance (SQA). The key process areas of CMM Level 3 are training
programs (TP), software product engineering (SPE), peer reviews (PR), intergroup
coordination (IC) and integrated software management (ISM). We suggested that the
developers and managers temporarily forget about reaching CMM Level 2 and instead state all
the major product development problems they currently faced. Then we asked them to list the
business goals they were trying to achieve over the next six to 18 months. These two
independent lists are shown in Table 1.
We then compared their problems and goals with the practices in SEI CMM Levels 2 and 3 and
placed the related KPA names and activities in parentheses after each item (Table 1). These
activities are small solutions that address the problems and support the goals. (If the company
had been using ISO9001 or the Malcolm Baldrige Award, we would have mapped the problems
and goals to those criteria.)
What was the scope of the improvement program? To address the problems and the goals of
the organization. As you can see, 21 out of the 23 items (91 percent) map to CMM Level 2.
When all the problems and goals have been addressed, 46 percent of the CMM Level 2
activities will have been addressed.
The essential difference between this approach and addressing the six KPAs in parallel is that
the problems and goals indicate which pieces of each KPA you should address first. Regardless
of whether your organization is using SEI CMM ISO9001 or another model or standard, the
problem-goal approach will help scope and sequence your improvement program.
Items
that
Don't
Quite
Fit
Not every problem in Table 1 exactly matches the key process areas of CMM Level 2. For
example, there isn't much in the CMM to help this organization specifically address the fifth
goal, improve performance of core software product. In this situation, you must determine
which areas are the most important for the organization to fix now. Serious problems should
be worked on first, regardless of whether they relate to the CMM. Solutions to this specific
goal will likely come from other technical sources.
Five lessons can be learned from adopting the goal-problem approach:

All process improvement can be meaningful.

The problems and goals can help identify which part of the SEI CMM to work on first.
The CMM shouldn't be seen as an all-or-nothing approach in which all parts should be
attempted at once. It should be treated as a large toolbox of little actions, ideas and
solutions, each of which is useful at different times.
Any process document developed to solve a problem will be meaningful and useful. A
process improvement team will be less tempted to gold-plate the process if its scope is
defined by a problem.
A group's motivation to work on improvement is increased when its problems and
goals are the primary focus of the improvement program.
Focusing on goals and solutions to problems prevents an organization from creating
academic process documents.

At
the
Project
Level
Here is an example of how a single project used the goal-problem approach. We first asked
the project manager for a significant project goal. From this goal, we derived areas that
needed improvement by asking: "What is preventing you from achieving the goal?" and "What
other problems do you have related to this goal?" The first question uncovered problem areas

related to the goal, and the second helped elicit further problem areas. The goal and problem
list formed the scope of the improvement program for this project.
What is your goal?

Reduce release cycle to six to nine months

What prevents you from achieving the goal?

Changing requirements.
Loss of resources. It's difficult to replace people with specialized skills who leave the
project.
Too many features are required for a six to nine month development cycle.
The poor quality of the incoming code from other groups.
Inadequate availability of test equipment

What other problems do you have related to this goal?

Lack of visibility within any life-cycle phase-it's difficult to know whether we're ahead
or behind schedule.
We don't always have the resources available to complete the planned work.
It's difficult to find defects early

We then worked through each of her answers and noted which KPA activity could significantly
help address each problem area (see Table 2). We recommended the more advanced CMM
Level 3 KPA components since her group had almost attained CMM Level 2. A selection of
these practices are defined in Table 3.
In this example, five out of the eight problems (63 percent) mapped to SEI CMM Level 2, and
100 percent mapped to SEI CMM Level 3. The problems and goal became the scope of the
improvement program, and by addressing these real issues, the project manager can also
make significant progress toward completing CMM Level 2 and starting CMM Level 3.
What questions will help you scope your improvement effort?

What one goal will you be held accountable for over the next six to 18 months?
What prevents you from achieving this goal?
What other problems do you have related to this goal?
If you are using a process improvement model (such as the SEI CMM), which items
help each of the problems listed

Covering
All
Your
Bases
One of the primary concerns that people have with the goal-problem approach is that an
organization won't achieve its initial goal of CMM Level 3 or ISO90001, since attention will be
diverted to the business goals and problems lists. When the first set of problems and goals
have been addressed, repeat the cycle to determine the next set of problems and goals. This
new set can then be compared to the remaining elements in the CMM. Over a one-to-three
year period, each section of the CMM can be matched with a problem or goal.

Figure 1.
Typical
Improvement

Phase

of

Goal-Problem
Program

Figure 1 shows the typical

phases
of
an
improvement
program,
using the goal-problem
approach. Each time a
goal or a problem is
addressed,
solution(s)
from the CMM can be
applied.
As
an
example,
the
company who owned the
problems and goals listed Each time a goal or a problem is addressed, solution(s)
in Table 1, conducted an from the CMM can be applied.
informal
process
assessment after 11 months of improvement work to monitor progress. The assessment
showed that 50 percent of CMM Level 2 practices had been adopted, and many of the initial
problems had been fixed. At the end of the assessment, they revised their problem list for the
next phase of their improvement program. The problems were:

Files delivered using CD-ROMs aren't verified.

Actual data isn't recorded from the initial project plan to determine how well we did.
We don't know the status of our testing activities and when we are ready to ship (for
example, defect closure and fine rate).
We don't know the specific differences between one software release and the previous
one.
We don't verify that the processes we have put in place are used correctly.
Each one of these items maps specifically to practices in the SEI CMM

There are situations where some of the elements of the CMM aren't used when solving a
problem or achieving a goal. These elements should be left until the end of the improvement
cycle. At that time, either the outstanding elements are put to good use, or considered not
applicable.
Elements are put to good use by asking the question, "What problem do we have where this
element could help?" For example, baseline audits are called for in the CMM under the topic of
software configuration management (SCM). A baseline audit verifies that the files that go into
a software product release are the correct ones. The audit can be a human-eye check to verify
file version numbers, file sizes and file names. This practice is usually ignored since it's not
obvious why it's needed. Asking, "What problem do we have where this element could help?"
elicits project team experiences where the wrong files have been sent to the testing group or
released to the customer. Baseline audits should be employed when these situations (or
problems) occur.
The goal-problem approach also helps an organization avoid prematurely using practices from
the CMM. Process auditing is an example of a practice that is sometimes adopted prematurely.
Process audits verify that the correct process steps have been carried out when performing
engineering activities such as schedule estimation, testing, peer reviews and SCM. Process
audits identify and eliminate engineering mistakes before they cause large unnecessary
downstream costs. In the beginning of an improvement program, there is usually little benefit
in performing process audits. However, the need for it becomes apparent once the process has
been defined, used and proven effective.
One company highlighted this with its software release management process. Before release
management had been improved, performing an audit on the related SCM activities would
have been futile. When SCM and release management were in place, one employee by-passed

the process and incorrectly released a software patch by e-mail to a customer. The software
didn't work and the customer was furious. The need for SCM auditing became apparent. After
the audits were performed, the developers and managers realized that they now had a
mechanism to verify the execution of this essential practice.
Scoping an improvement program can be difficult and frustrating. Moreover, the task becomes
daunting when a process model, such as the CMM, is adopted wholesale. However, a simple,
immediately available solution exists. The goals and problems of an organization can provide a
timeless and effective scope for any improvement program. A model or standard can then be
used as a source of ideas, solutions and actions to achieve this scope. The resulting goalproblem improvement program is compelling and practical.
Table 1. Problems and Goals List
Problems
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.

Need better requirements. Requirements tracking not in placechanges to

requirements aren't tracked; code doesn't match specifications at test time. [Level 2:
Requirements Management (RM) activities 1, 2 and 3]
Management direction unclear for version 2.3. Goals change often. [Level 2: RM
activities 1 and 3 and verification 1]
Hard to revise project planitems drop off, new things are added, plan is out of date.
[Level 2: Software Project Tracking and Oversight (SPTO) activity 2, 8 and 9]
Wrong files (for example, DLLs) get put on CD-ROMdon't know what the right ones
should be. [Level 2: Software Configuration Management (SCM) activities 4, 7, 8, 9 and
10]
Defect repairs break essential product features. [Level 2: SCM activities 5, 6, 7, 9 and
10, abilities 1, 2, 4 and 5, and verification 3 and 4]
Customers are unhappy. There are approximately 300 outstanding defects that haven't
been addressed. [Level 2: SCM verification 1, RM activity 3; Level 3: Intergroup
Coordination (IC) activity 1]
Difficult to find time to do critical activities (product development) versus crisis
activities. [Level 2: Software Project Planning (SPP) activities 4, 10 and 12]
Lack of resources and skills allocated to software design. [Level 2: SPP activity 10]
Quality department needs team training (product and test skills). [Level 2: Software
Quality Assurance (SQA) abilities 2, 3 and 4]
Changes to specifications and documentation aren't communicated effectively to
documentation and test groups. [Level 2: RM activities 1, 2 and 3; SCM activities 5, 6,
7 and 9 and ability 1]
Unreliable project schedule estimates. [Level 2: SPP activities 5, 9, 10, 12, 13 and 14
and ability 4]
Unclear status of software changes. [Level 2: SCM activities 8 and 9]
Testing doesn't necessarily comprehend things that matter to the customer. [Level 3:
SPE activities 5, 6 and 7]

Goals
1.
2.
3.
4.
5.

Orderly plans for development. [Level 2: Software Project Planning (SPP) activities 2,
5, 6, 7, 8, 13 and 14]
Understand what our capacity isdevelop one list of all the work we have to do. [Level
2: SPP activity 7 and ability 1]
Improve schedule tracking and communication of changes to impacted groups. [Level
2: SPTO activities 3 and 4]
Successfully deliver product X. [Level 2: RM activities 1, 2 and 3; SPP activities 6, 10
and 13]
Improve performance of core software product. [Level 2: SPP activity 11; SPTO activity

7]
Identify needed skills for new designers and hire, promote and train accordingly. [Level
3: Software Product Engineering (SPE) activity 3 and ability 2]
7. Identify tools to support software developers. [Level 2: SPP activity 14; Level 3: SPE
activity 1]
8. Keep making a profit. Keep customers happy. [Level 2: RM activities 1 and 2; SPP
activities 10, 12 and 13; SPTO activities 4, 6, 8 and 10; SQA activity 5. Level 3: SPE
activities 2 and 7; IC activity 1; Peer Review (PR) goal 2]
9. Identify tools to support software testers. [Level 2: SPP activity 14. Level 3: SPE
activity 1]
10. Empower Quality Department to have final say on product shipment. [Level 2: SQA
activities 6 and 7]
6.

(The lists are not in any particular order. The KPA activities in brackets are small solutions that
address the problems and goals.)
[Return to text]

Table 2. The Goal and Problems of One Project

Goal: Reduce release cycle to six to nine months
Problems

KPA component that would help this problem

Changing requirements.

Level 2: Requirements Management (RM) activity 3;

Software Configuration Management (SCM) activity 5.
Level 3: Software Product Engineering (SPE) activity 2.

Loss of resources. It's difficult to Level 2: Software Project Tracking and Oversight
replace people who leave the (SPTO)
activities
2
and
8.
project due to specialized skills.
Level 3: Training Program (TP) activity 1; SPE ability 2.
Too many features are required for Level 2: Software Project Planning (SPP) activities 4,
a six- to nine-month development 12 and 13.Level 3: SPE activity 2.
cycle.
Poor quality of incoming code from Level 3: Intergroup Coordination (IC) activities 2, 5
other groups.
and 6; Peer Review (PR) activity 2.
Inadequate
equipment.

availability

of

test Level
2:
SPP
activities
Level 3: SPE activities 6 and 7.

13

and

14.

Lack of visibility within any life cycle Level 3: Integrated Software Management
phaseit's difficult to know whether activities 4, 7 and 11 and verification 2.
we are ahead of or behind schedule.

(ISM)

We don't always have the resources Level

2:
SPP
activities
4,
12
available to complete planned work. Level 3: ISM activities 3, 5, 10 and 11.
It's difficult to find defects early.
[Return to text]

Level 3: PR activities 1 and 2 and ability 1.

and

13.

Table 3. CMM Activities for the First Two Problems of Table 2.

Key Process Area Practice (from SEI CMM Definition
1.1)
Requirements Management (RM) activity 3

Changes to the allocated requirements are

reviewed and incorporated into the software
project.

Software Configuration Management (SCM) Change requests and problem reports for all
activity 5
configuration items or units are initiated,
recorded, reviewed, approved and tracked
according to a documented process.
Software Product Engineering (SPE) activity 2 The software requirements are developed,
maintained, documented and verified by
systematically
analyzing
the
allocated
requirements according to the defined
software process.
Software Project
(SPTO) activity 2

Tracking

and

Oversight The project's software development plan is

revised
according
to
a
documented
procedure.

Software Project
(SPTO) activity 8

Tracking

and

Oversight The project's software schedule is tracked,

and corrective actions are taken as
necessary.

Training Program (TP) activity 1

Each software project develops and maintains

a training plan that specifies its training
needs.

Software Product Engineering (SPE) ability 2

Members of the software engineering

technical staff receive required training to
perform their technical assignments.

[Return to text]