case study

A Plan for the Master Plan

IMProvIng Project MAnAgeMent wIth sIx sIgMA

By Thomas Dietmller, Avancis GmbH & Co. KG., and Bryan Spitler, Behr Dayton Thermal Products, USA

iproject master plan in serial development is very important. The master iplan shows which subprocess needs to be executed in the particular idevelopment phase. The plan predetermines, for example, that the failure mode effects analysis (FMEA) and other quality processes are conducted in certain phases of the development process. The project master plan is simply for developing best practices. But how is the project master plan determined? The central project management department generally devises the project master plan. The selection of preventive quality processes, such as FMEA or validation, is usually based on customer requirements, quality standards and experts experience. If the company has already agreed on a selection, this usually remains unmodified, and a critical review of the contents does not happen. The development process has a huge impact on project success. The preventive quality process targets the detection of possible failures, thereby reducing failure costs in the production process. Which preventive quality process is most cost effective and should be the focus of every development process and project master plan? Theres a databased answer to this question. To review the master plan, a case study was conducted at a supplier in the automotive industry. Although the data were collected in the automotive industry, the concept is applicable in many different industries that develop and produce serial products in product families or applications. The main difficulty for data-driven evaluations of preventive quality management is the lack of valid data throughout the life cycle. There are two reasons for this: 1. The preventive quality process is a subprocess of the slowly running development process, the output of which is difficult to measure. 2. Any measurement system for preventive quality management activities faces a long lag time between the prevention process and its effects on production. To handle these difficulties, a Six Sigma approach was chosen and an improvement project started following the define, measure, analyze, improve and control (DMAIC) phases (see Figure 1). Define The aim of the Six Sigma project was to increase the benefit of preventive quality processes. Early in discussions, it became clear that an exclusively monetary view would be too narrow for a balanced evaluation. Thus, the quality or maturity of the processes also needed to be evaluated. To keep the project focused among other quality processes, the interdis-

november 2009

WWW.ASQ.orG

A P l a n f o r t h e M a s t e r Plan

Figure 1. dMaIc map for data-improvement project

Define Project definition: increase the benefit of preventive quality processes such as FMEA and validation. Tools: cause and effect diagram. Control Verify the effectiveness of the revision. Tools: statistical tests.

Improve Revision of the preventive quality processes based on their payback.

Measure Implementation of indicator system: leading and lagging indicators for quality processes. Implementation of project classification indicators: product and process related risks.

FMeA = failure mode effects analysis

Analyze Analyze the indicators for multiple projects to identify the benefit. Tools: regression analysis, statistical tests and cost-benefit analysis.

ciplinary experts chose to start with FMEAs and validation. At the automotive supplier, FMEAs and validation were specified in design and process according to the focus on the product and the production processes. Therefore, design/process FMEA and design/process validation had been considered. In theory, these preventive quality processes affect development costs and failure costs. A cause and effect diagram helped to map this theory (see Figure 2). The FMEA process mainly focuses on avoiding failures in production and reducing internal failure costs, such as scrap or rework. The validation primarily contributes to cutting external failure costs, such as warranty costs. The goal now was to substantiate this principle with data from actual projects. This was done in the measure and analyze phases. Measure

Figure 2. cause and effect diagram for selected preventive quality processes
FMEAs Design FMEA Process FMEA Internal failure costs

Validation Design validation Process validation

Prevention costs

External failure costs

Process perspective cause and effect chain FMeA = failure mode effects analysis

Cost perspective + positive correlation negative correlation

To validate how the quality of the different processes relates to the different failure costs, detailed indica-

tors in development and production are necessary. The concept, its implementation and actual data collection were part of the measure phase. The result of this DMAIC phase was an indicator system that contained leading and lagging indicators for the preventive quality processes (see Figure 3, p. 10).

Six SiGmA forum mAGAzine

november 2009

A Plan for the Mas t e r P l a n

The leading indicators represent activities in the products development to ensure quality. The lagging indicators show the quality results in the later production and life phases. In addition, the indicators are reflective of the perspectives in the cause and effect chain: The process perspective measures the process maturity of the preventive activities in the products development. The cost perspective evaluates the prevention costs and failure costs. The process perspective checks the key success factors for each preventive quality process. Those factors are then used to calculate the process degree, timeliness and process maturity. For example, success factors for the FMEA were determined as setting up a time schedule or completeness of all disciplines in the actual FMEA meeting. Those success factors were assigned to different phases of the development process and

weighted according to importance. Organized in a checklist or quality plan, the key activities were now binding for all team members, and the quality plan provided a thorough status report of the processes for management. With those key activities accomplished, the process degree grew from phase to phase until it reached the maximum of 100%. For each process and each phase, the target of the process degree was defined in the quality plan. Before the start of production, the team should reach 100% process degree in all preventive processes. If the team fails, however, the actual process degree does not match the requirements, and the process is delayed (see Figure 4). The delta between the actual and plan process degree was used to calculate the timeliness, which is 100% minus the total delta. The bigger the delay, the bigger the delta between actual and plan, and thus the lower the timeliness becomes. The timeliness and the process degree determine the maturity of the preventive process by multiplying the two indicators.

Figure 3. Indicator system

Leading indicators Process perspective Maturity = process degree timeliness Lagging indicators Cost perspective Prevention costs Failure costs

Figure 4. controlling the process maturity with the quality plan

XY Customer Component A Project Date examination Phase Ongoing until Process System-FMEA product System-FMEA process Resp. Miller Miller 1 09.02.205 2 11.01.2005 P-16722 Project number Smith Team leader 3 02.01.2007 4 09.01.2007 09.09.2008 5 11.01.2008 Evaluation process degree Behind plan In process Closed (100%) Remark Plan Action Plan Action Plan Action Plan Action Plan Action 0 40 40 60 60 80 60 100 100 0 0 40 40 70 70 100 100 Timeliness 80 100

Maturity 8 10

Timeliness calculation Timeliness = 100% (process degree [plan] process degree [actual]) 80% = 100% ([40% 40%] + [60% 60%] + [80% 60%] + [100% 100%]) FMeA = failure mode effects analysis

Maturity calculation: Maturity = 10 process degree timeliness 8 = 10 100% 80%

10

november 2009

WWW.ASQ.orG

A P l a n f o r t h e M a s t e r Plan

According to the experts, the Figure 5. Regression analysis for FMea maturity and preventive processes needed to be prevention costs completed in time. A completed FMEA, for example, will not be successful if its begun after the start of production. Thus, the maturity indicates if all success + factors are completed in time. From the cost perspective, the costs of quality are either part of the leading or lagging indicators. The prevention costs are usually defined as costs that arise from efforts to keep defects from occurring at all1for example, N = 21 projects project-specific costs for the preR = 0e25 ventive quality processes. As typip = 0.01 cal lagging indicators, the main failure costs include scrap, rework and warranty.2 1 2 3 4 5 6 7 8 9 With the leading and lagging Low Maturity for FMEAs High indicators, the project team has established a practice-oriented Figure 6. Regression analysis for FMea maturity and indicator system to detect the internal failure costs deployment and effect of the preventive quality processes. The continuous data collection + in development improved the transparency of the preventive processes. As a result, in case of delay, corrective actions can be implemented faster than before. By analyzing the leading and lagging indicators in several projects, it is possible to statistically determine the effect of the preventive processes. For this purpose, it was essential that only similar projects High-risk projects, N = 10, R = 0.48, p = 0.03 are compared. Low-risk projects, N = 11, R = 0.67, p = 0.00 To identify similar projects, classification indicators were 1 2 3 4 5 6 7 8 9 developed. These indicators Low Maturity for FMEAs High determined the risks at the beginning of the development. The classification indicators covered product or processNow that all necessary indicators for the project life related risks, such as a new material or new manufaccycle and the project classification were defined, and turing processes, as well as project-related risks, such the projects could be measured. In this case study, all as reduced development schedule. Based on those indicators were collected for 21 automotive projects. risks, the projects were classified as either low-risk The projects were comparable in type of product and projects, such as applications, or high-risk projects, manufacturing process, as well as in development and such as innovations. production facility and production volume.
Internal failure costs
Prevention costs for FMEAs

Six SiGmA forum mAGAzine

november 2009

11

A Plan for the Mas t e r P l a n

Analyze

In the analyze phase, those indicators were evaluated with different statistical tools, such as regression or statistical tests. The following example shows the results for the FMEA. The first regression analysis focused on how FMEA maturity increases the prevention costs: Figure 5 (p. 11) plots the prevention costs vs. the maturity for FMEAs, and it shows that the projects with high maturity generally have higher prevention costs than projects with a low maturity. For a high maturity, the success factors, such as completeness of all disciplines in the actual FMEA Improve meeting, must be fulfilled. That leads to higher prevention costs. Given these findings, the project master plan was The second regression analysis focused on the modified. For example, the FMEA was highlighted in maturity and the internal failure costs. Figure 6 (p. the master plan and more capacity was given to the 11) shows that projects with higher maturity end up FMEA for all new projects. In addition, management having lower internal failure costs. re-emphasized the on-time completion. The classification into high and low-risk projects The quality plan also served as a report, providing a was necessary because high-risk projects have higher tool for management to easily review the performance failure costs than low-risk projects. This factual data, in the development projects. The quality plan became however, proved that FMEAs, if completed on time, a standard tool in project management and is now result in lower failure costs in production. Good binding in every new development project. FMEAs decrease the internal failure costs but also increase the prevention costs. Control The next step uses a cost-benefit analysis to find out whether the higher prevention costs and the higher The control phase uses the leading and lagging indimaturity finally pay off. The aim is to evaluate which cators of old and new projects to validate the improvepreventive process has the best cost-benefit ratio. For that purpose, the high and low-risk projects are classified again. Projects with a high matuFigure 7. cost-benefit analysis for FMeas rity in FMEAs, for example, are classified as high maturity projects, and vice versa. $ 280,000 The separation between the two classes Low-risk projects with: needs to be statistically significant. For the lowlow FMEA maturity N = 6 Difference risk projects, this was proven with the t-test and high FMEA maturity N = 5 $70,000 confirmed. For the high-risk projects, however, the separation in low and high maturity was not significant. This was due to the much higher risk, which leads to a higher variation Prevention Internal in the failure costs. costs failure costs For the low-risk projects, the prevention and failure costs for project classes were compared and evaluated based on the investment Difference $7,000 appraisal. The investment appraisal took into 0 Start of Start of End of account that cash invested in the development development production production (year 7) process is more expensive than cash spent on the later failure costs. The failure costs FMeA = failure mode effects analysis were calculated with the average failure rate
Cumulated prevention costs and internal failure cost for average project volume (cash value)

of the low and high-maturity projects, which was then multiplied by the total production volume. For the N = 11 low-risk projects, the cost-benefit analysis clearly showed a benefit for the project class with higher maturity in their prevention processes (see Figure 7). Following the cash-value method, a project with lower maturity initially had about $7,000 in reduced prevention costs. But those savings were quickly spent on higher failure costs. In the long run, the project class with good FMEAs ended up saving about $70,000 reduced in prevention and internal failure costs. If the savings for external failure costs, such as warranty, were considered, the profit could be even larger.

12

november 2009

WWW.ASQ.orG

A P l a n f o r t h e M a s t e r P lan

structured And dAtA-bAsed ApproAch is necessAry to improve the

benefit of the preventive quAlity mAnAgement processes requested by A mAster project plAn.
ments. The target for all new projects, Figure 8. Boxplot of the internal failure costs which was achieved, was to close out the before and after improvement development phase with a high maturity for the FMEAs. For those projects, failure costs were expected to be low. After improvement (n = 5) By comparing failure costs in projects based on the old master plan with projects according to the new master plan, Before improvement (n = 11) the control phase validates the effectiveness of the improvement. Because of the time delay between the leading Internal failure costs + and lagging indicators, the study could measure the failure costs of five new lowrisk projects. These costs were compared addition, similar development projects were clustered with the failure costs of the low-risk projects before in groups by using the rating of their respective risks. the improvement. In the case study, data from 21 similar serial developWith a p-value of 0.007, the Mood-Media test ment projects were collected and evaluated. showed a significant difference between the failure For the FMEAs, the correlation between accurately costs before and after the improvement (see Figure timed and completed processes and failure costs was sta8). This finally proved the effectiveness of the moditistically verified, and the return was calculated. Based fication. on the monetary return of individual processes, the Increased transparency means efficiency master project plan was revised. Prevention processes with more impact were highlighted for future developA structured and data-based approach is necessary to ment projects, and others became less important. improve the benefit of the preventive quality manageAs an additional benefit, the implemented indiment processes requested by a master project plan. cators increased the transparency and, as a result, Decisions about which quality processes to apply in the efficiency. This case study showed how Six Sigma development projects should be based on what return can successfully be used to drive long-term strategic those processes had in previous projects and should improvements. not be based solely on experience. Six Sigma proved to be the appropriate method to determine which REFERENCES preventive process is most effective. 1. Armand V. Feigenbaum, Total Quality Control, third edition, McGraw-Hill, This automotive case study showed how DMAIC was p. 111, 1991. used to improve the selection of preventive quality 2. Steve Pollack, Create a Simple Framework to Validate FMEA Performance, Six Sigma Forum Magazine, Vol. 4, No. 4, 2005, pp 27-34. processes in serial development projects. In the define phase, the Six Sigma project was started, and the first hypothesis about the cause and effect of individual prevention processes was developed. To verify or discard the hypothesis during the WhAt Do you thINk oF thIS ARtIClE? Please share measure phase, an indicator system was established and implemented. Throughout the entire project life your comments and thoughts with the editor by e-mailing cycle, leading and lagging indicators monitored the godfrey@asq.org. performance to cover the cause and effect chain. In
Low-risk projects

Six SiGmA forum mAGAzine

november 2009

13