You are on page 1of 56

Statistical Process Control

Version 4.0
July 2001

Statement of Confidentiality
This Training material contains information that is proprietary and confidential to
TATA Consultancy Services.
1

TCS Confidential Presentation

Course Coverage

This course on SPC covers the following topics:

Overview of basic concepts


Exploration on Process and Product Quality
Quantitative Process Management/Software Quality
Management overview, Quality Goals
Shewart/Deming Measurement Process and Site
Implementation
Computation Formulas for Individuals XmR and ucharts
Control Chart Pattern Analysis
Control Limit Drawing Guidelines
2

TCS Confidential Presentation

What is a Process?

Process applies to any sequence of steps


performed for a given purpose [IEEE-STD-610]
whether they are the software development
process or a subset within the software
development process
Process implies the way we do things. In the
software context, it implies the way we do
software.
Processes produce two outputs:

Product of service
Data
3

TCS Confidential Presentation

QPM and SQM

While QPM focuses on quantitatively


managing the process quality, Software
Quality Management (SQM) focuses on
managing the product quality.

QPM
SQM

Process Quality
Product Quality

TCS Confidential Presentation

Process and Product Quality

Process quality refers to the quantification of process


characteristics whose values indicate a degree of quality of
the underlying process
For example, we can use Review Effectiveness to quantify
the efficiency of the review process, or the Schedule
Slippage to quantify the efficiency of our process by which
we deliver our products in time
Product quality refers to the quantification of product
characteristics whose values indicate a degree of quality of
the product
For example, we can use Defect Density to quantify the
quality of our software deliverables (how many defects are
we delivering?)
5

TCS Confidential Presentation

Why Quantitative Management?

Quantitative management is about making


decisions based on sound information
Quantitative management is about
management by fact
Facts are derived out of information
Information is the interpretation of data
Data comes from processes
6

TCS Confidential Presentation

Quantitative Process Management

Quantitative Process Management (QPM) in


the CMM applies to all the applicable
production, test, and maintenance
processes and sub-processes
QPM focuses on:

Quantitatively understanding the software


processes
Quantitatively managing the software
processes
7

TCS Confidential Presentation

Quantitative Process Management

Quantitative Process Management involves:


1. Establishing goals for the projects defined software process
2. Taking measurements of process performance (project data)
3. Analyzing these measurements
4. Making adjustments to maintain process performance
(projects UCL and LCL) within acceptable limits

The organisation collects data from the software projects and


uses these data to come up with the PCB of the OSSP. PCB
describes the range of expected results from following a
software process. These PCB data are used by the projects
to analyse the performance of the projects defined software
process
TCS Confidential Presentation

Software Quality Management

The purpose of Software Quality Management is to


develop a quantitative understanding of the quality
of the projects software products and achieve
quality goals.
SQM involves:
1. Defining quality goals for the software products
2. Establishing plans to achieve these goals
3. Monitoring and adjusting software plans, work
products, activities and the quality goals to satisfy the
needs and desires of the customer and end user
9

TCS Confidential Presentation

What is a Quality Goal?

Quality Goals apply to the SQM Key Process


Area
1. A quality factor is defined for the software
product
2. A quantitative measure for the factor is
established
3. A target or desired value of the factor is
established - this constitutes the Quality Goal
10

TCS Confidential Presentation

What is a Quality Goal?


Scenario 1:
A system must be designed such that its back
end response time is not more than 5
milliseconds.

Here:

Quality factor =
Performance
(Responsiveness)

Quality goal =
< 5 milliseconds

11

TCS Confidential Presentation

Quality Goal - Example


Scenario 2:

A module that is to be delivered should not


have more than 5 defects/KLOC at the time
of delivery
Here:

Quality factor =
Reliability

Quality goal =
< 5 defects/KLOC

12

TCS Confidential Presentation

QPM and SQM - Integrated View


Quantitative Process Management
Focuses on Process Quality

Process performance is
measured

Software Quality Management


Focuses on Product Quality

Product quality is measured

Measurements on process
metrics result in changes to the
software process (OPD)

Measurements on quality
goals result in changes to
software quality plan, work
products, activities
13

TCS Confidential Presentation

QPM and SQM - Implementation

An integrated implementation of the key practices of


QPM and SQM is followed.
Process Metrics

Review Effectiveness
% Schedule slippage
% Effort slippage

% Rework Effort
% Review Effort
%Size Deviation

Product Metrics

Defect density
% Bad Fixes
TDCE
14

TCS Confidential Presentation

QPM and SQM - Implementation

SQM and QPM activities are outlined in the Metrics Section of


the Software Quality Assurance Plan or UPP.
Branch level Targets are used as performance limits for the
process metrics and quality goals for the product metrics
Other product quality goals other than the ones described may
be defined for projects:

Example:
Productivity

15

TCS Confidential Presentation

Shewart/Deming Cycle, Measurement Process and Site Implementation


Shewart/Deming Cycle
Plan

Activity in the software measurement Purpose


process
Identify scope, define procedures
Establish a purpose, plan, and
procedure for measurement tied to
organizational goals

Do

Collect data

Execute the plan to collect the data

Check

Analyze the data

Analyze the data to determine the


progress towards measurement
goals

Act

Evolve process

Make decisions based on the


progress towards goals and
recommend improvements on the
measurement process being used

16

TCS Confidential Presentation

What is Statistical Process Control?

Statistical Process Control (SPC) is a method to


define operational limits for acceptable variation in
process data, and to examine the process
performance in absolute statistical terms
SPC and its associated control charts were
developed by Walter A Shewart in the 1920s to
gain control of production costs and quality
17

TCS Confidential Presentation

What is new in SPC?

Statistical Process Control is NOT a new concept

SPC is a widely used technique in other domains of


engineering - such as manufacturing.
What is new is the application of statistical process
control techniques to Software Engineering and
Software Project Management
18

TCS Confidential Presentation

Process Capability
Branchs
LCL LST

UST

Branchs
UCL

Projects
LCL

Process
Capability
Baseline

Projects
UCL
Stretched Targets

All points within projects


fixed UCL/LCL implies
process is stable

3s

3s

All points within branchs


fixed UCL/LCL (PCB)
implies process is capable

19

TCS Confidential Presentation

Basic Definitions

UCL : Upper Control Limit

LCL : Lower Control Limit

USL : Upper Specification Limit

LSL : Lower Specification Limit

UST : Upper Stretched Target

LST : Lower Stretched Target

PCB: Process Capability Baseline

Derived from data

When PCB is
established,
the Spec Limits of the
centre are usually set as
the control limits
derived as per the PCB
Set by Centre Head as a
Long term Goal
Not derived from data

TCS Confidential Presentation

20

What is an Outlier?

In simple terms, it is a data point that falls


outside the Control Limits. It indicates
disturbance or abnormality in the process.

21

TCS Confidential Presentation

Basic Definitions

A process is said to be stable if there are no outliers wrt UCL


and LCL of the process. If points lie outside the control
limits, the process is unstable and is said to be out of
control.

A process is said to be capable if there are no outliers wrt


Specification Limits.

SEPG establishes Process Capability Baseline(PCB) when the


process is stable and Capable.This is verified at the end of
every three months
22

TCS Confidential Presentation

How do you carry out the Process


Measurement and Analysis Activities in the
projects?

23

TCS Confidential Presentation

SPC Analysis at project Level


Collect Process Performance data
Look for any outlier wrt Projects
Control limits
Analyze and Correct Process and
then Prevent it from Recurring

Now the Process is said to be


stable at the project level
Analyze and Correct outliers wrt
Spec Limits
No Outliers wrt Spec Limits,
Process is Stable and Capable

Continue for the next month

Now Fix the UCL and LCL at the


project level as Process
Performance Baseline(Fixed UCL
and LCL)

TCS Confidential Presentation

24

How are Process Measurement and Analysis

Activities carried out at the organization


level?

25

TCS Confidential Presentation

SPC Analysis at Organisation Level


SEPG collects data from projects
on a monthly basis
SEPG consolidates data

SEPG looks for any Outliers wrt


current CLs and Spec Limits of
the Organisation

Spec Limits
are
Quality Goals

SEPG along with PLs analyzes


and corrects these outliers
SEPG fixes the Control Limits
and they form Spec Limits for
the later months
Projects in turn use the Spec
Limits as their targets (USL and
LSL )
TCS Confidential Presentation

26

Process Improvement and Distribution Spread


0.04

0.035

The bell-curve becomes:


Taller
Thinner
with more process improvements

0.03

Probability

0.025

0.02

0.015

0.01

0.005

0
-150

-100

-50

50

100

150

200

0.02

250

Metric value
0.018

0.016

0.014

Probability

0.012

0.01

0.008

0.006

0.004

Process Improvements

0.002

0
-150

-100

-50

50

100

150

200

250

Metric value
0.02

0.018

0.016

0.014

Probability

0.012

0.01

0.008

0.006

0.004

More Process Improvements

0.002

0
-150

-100

-50

50

100

150

200

250

Metric value

27

TCS Confidential Presentation

Types of SPC charts

Run Charts
Control charts
X-mR chart

Attribute charts
p-chart
np-chart
c-chart
u-chart
z-chart

Not all SPC techniques may be equally applicable


for software processes
TCS Confidential Presentation

28

Computation formulae for the


Individuals X-mR chart

Characteristic:

Each individual measurement is used and


plotted
Moving range values are absolute values of
the present measurement minus the previous
measurement

29

TCS Confidential Presentation

Computation formulas for the


Individuals X-mR chart

X-bar = X-avg = X/n


R-bar = R-avg = R/(n-1)
UCL [X] = X-bar + 3*Sigma
= X-bar + 3*R-Bar/d2 where d2 = 1.128
= X-bar + 2.66 * R-bar
LCL [X] = X-bar 2.66 * R-bar
UCL [mR] = 3.267 * R-bar
LCL [mR] = 0 * R-bar = 0
30

TCS Confidential Presentation

Some control charts - Organisation


TCS, Bangalore sensitive and confidential
Level
data
X Chart Schedule Slippage
Slippage/component
Current LCL of the Project
Project's Prev (Fixed) LCL
LST (Low er Stretch Target)

Xbar
UST (Upper Stretch Target)
Upper Spec Limit

Current UCL of the Project


Project's Prev (Fixed) UCL
Low er Spec Limit

15

% Sch Slippage

10
5
0
-5

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

-10
-15
-20
Com ponents

31

TCS Confidential Presentation

Causes of Variation

Random or Common
Normally, we cannot do anything to change this
Continuously active and part of the process
Process performance noise
Special or Assignable
We can do something to change this
Not always active or part of the process
Are extraordinary events
Process non performance signals
32

TCS Confidential Presentation

System Causes of Variation

When variation is due only to system causes and


there is a sufficient volume of data, the variations
will tend to take a normal (bell shaped) distribution
centered around a middle measure
If special causes are present, the curve will be
distorted and lose its normal distribution

33

TCS Confidential Presentation

Special Causes of Variation


identifying variations in performance that are outside the bounds
of process capability; i. e., the normal range of process
performance

identifying patterns in system variations above or below the


average
Both require attention and potentially corrective action

average
34

TCS Confidential Presentation

How to Interpret?

Special causes

Is it explainable ? (PL, GL to investigate)


Can we prevent recurrence of it?

Common causes

Pareto, Fishbone
Corrective actions at Project Level and/or Branch Level

35
12/29/2014

TCS Confidential Presentation

Examples

Special causes

Shifts in the quality of raw materials


Inadequately trained people
Changes to work environments
tool failures, altered methods, failures to follow the
process, etc.,

Common causes

Problem in the process


Process is not clearly defined

36
12/29/2014

TCS Confidential Presentation

Control & Specification Limits


OK

Not OK

stable and capable

stable but not capable


37

TCS Confidential Presentation

Using control charts to correct


the process

Correcting the process involves control and/or:


removal of assignable causes

use of the chart as a dynamic picture of quality over time


psychological effect on the personnel

The whole process of using control charts is to:


gain finer and finer control

identify and correct assignable causes


perfect procedures and attitudes that ensure the maximum
utilization of the process capabilities
TCS Confidential Presentation

38

Control chart pattern analysis

Several distinct patterns can be recognized in


control charts which can be potential indicators or
signals of instability.
The reasons of the selection of patterns is based
on sound mathematical principles behind control
chart theory and probability theory.
Once a distinct unstable pattern is identified, the
search for an assignable cause must be initiated
and corrected to improve the process.
39

TCS Confidential Presentation

Basic Pathologies

The following are the basic pathologies for process


instabilities:
Points outside the 3 control limits
7 point patterns

7 points in a row above the mean


7 points in a row below the mean
7 points in a row alternating up and below the mean
7 points in a row going up
7 points in a row going down

Points between + 1

5 points in a row between + 1 from the mean


6 of 7 points in a row between + 1 from the mean
TCS Confidential Presentation

40

Basic Pathologies

Points between + 1 and + 2

Points between + 2 and + 3

3 points in a row between + 1 and + 2


4 of 5 points in a row between + 1 and + 2

2 points in a row between + 2 and + 3

10 of 11 patterns

10 of 11 points in a row above the mean


10 of 11 points in a row below the mean
10 of 11 points in a row alternating up and below the mean
10 of 11 points in a row going up
10 of 11 points in a row going down
TCS Confidential Presentation

41

Basic Pathologies

12 of 14 patterns

12 of 14 points in a row above the mean


12 of 14 points in a row below the mean
12 of 14 points in a row alternating up and below the mean
12 of 14 points in a row going up
12 of 14 points in a row going down

42

TCS Confidential Presentation

Control Limits Recalculation

The Upper and Lower Control Limits are the


boundaries of process stability.

A useful control chart can be drawn only with a


minimum of 20 points for a software process.
The general guidelines for control limit revision are
described in the following charts.

43

TCS Confidential Presentation

When to draw a control chart

If you have less than 20 points, DO NOT DRAW a control


chart.

Once you have 20 or more points

Draw a run chart


Use the branchs UCL and LCL to see whether your process is
capable or not
Draw a control chart and calculate the control limits. Treat these
limits as fixed for the further data points
Use the control limits to analyze process stability
Use the branchs UCL and LCL to analyze process capability

Whenever a new data point is available

Add it to the control chart data


Redraw the control chart BUT DO NOT RECALCULATE THE
CONTROL LIMITS - USE THE OLD (FIXED) CONTROL LIMITS
INSTEAD
TCS Confidential Presentation

44

When to draw a control chart

Continue using the old (fixed) control limits until you


bring the process under control (with respect to the
old (fixed) control limits) or a prescribed period of
time (say 3 months)
When the process is found to be within control (or
after the prescribed period of time), recalculate the
control limits and fix them as the new thresholds for
future data points
Continue this FIX-DRAW process to analyze
process stability and capability
45

TCS Confidential Presentation

Why should you fix the control limits?

You should not recalculate the control limits


whenever a new data point is added because, the
single data point may change the control limits and
alter your perspective of process stability.
This becomes important if the new data point
indicates an out-of-control process and may be
noise in the process. If you revise the control
limits, you will not get a good picture of process
stability.
46

TCS Confidential Presentation

You initially have 8 data points ...


You have less than 20 points - so draw a run
chart, use Branchs UCL and LCL
Run Chart (defecs/PD)

Defect density (defects/pd)

Branch's LCL

Branch's UCL

2.5

Defect/PD

1.5

0.5

0
1

Instance

47

TCS Confidential Presentation

12 more points are now available...


UCL
Calculated

You now have 20 data points - so draw a control chart,


calculate the control limits (process shown to be in control)

48

TCS Confidential Presentation

UCL
not calculated,
carried over

2 more data points are now available...

You now have 22 data


points - so draw a
control chart, but DO
NOT RECALCULATE
THE CONTROL
LIMITS, use the OLD
ONES INSTEAD (process shown to be
in control using old
limits)

49

TCS Confidential Presentation

If you had recalculated


the control limits...
UCL
recalculated with
the addition of 2
more points

Process is out of control because


control limits were revised with the
addition of two data points

Process is now out of control!

50

TCS Confidential Presentation

An example of recalculating control limits


at each point
Example of control limits recalculated at each data point

14

Defect Density (Defects/PD)

12

Upper control
limit recalculated
at each point

10

0
1

10

11

12

13

14

15

16

Instance

51

TCS Confidential Presentation

U Charts

U stands for per unit


Attribute Control Chart i.e. when sample reflects qualitative
characteristics such as defective/not defective, go/no go
Sample size varies
Data distribution follows Poisson distribution (discrete values)
Control Chart for non-conformities or defects
Varying UCL and LCL

52

TCS Confidential Presentation

Calculating U Charts
Ubar
= di / ni
UCLU = Ubar + 3 sqrt (Ubar / ni)
LCLU = Ubar - 3 sqrt (Ubar / ni) or 0 for less than 0
where ni = Size of ith sample
di = Non-conformities in ith sample
Using U chart for Defects/pd:
ni = Planned effort for the work item i

di = Defects detected in the work item i

53

TCS Confidential Presentation

Example of U Chart
2.5

UCL

Defects/pd

1.5

0.5

LCL
0
1

10

11

12

13

14

15

Time

54

TCS Confidential Presentation

Metrics under SPC


Project Level
Metric

Chart type

Schedule Slippage
Effort Slippage
Defects/pd - Review + UT

- Run/ XmR
- Run/ XmR
- U Chart

Organisation Level

Schedule Slippage
Effort Slippage
Defects/pd for Code & Document
Review Effectiveness
Total Defect Containment Effectiveness
Rework Effort
Review Effort

- XmR Chart
- XmR Chart
- U Chart
- Xmr Chart
- XmR Chart
- XmR Chart
- XmR Chart
55

TCS Confidential Presentation

Thank you !

56

TCS Confidential Presentation

You might also like