How To Get Started With Design-Expert

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.
Getting Started: Stat-Ease Resources

New to Design of Experiments?
Take advantage of all the free resources available to you!
On the statease.com website:

Beginner resources: http://www.statease.com/beginner.html
Webinars: http://www.statease.com/webinar.html
Articles: http://www.statease.com/articles.html
Software Tutorials: http://www.statease.com/dx8_man.html
Getting Started: Other Resources

New to Design of Experiments?
Take advantage of all the free resources available to you!
LinkedIn Groups:
The Design of Experiment (DOE) Group great place to post
general questions about DOEs
ASQ Statistics Division more general statistics and DOE
The Stat-Ease Professional Network friends and clients of
Stat-Ease
Agenda Transition
Intro What is DOE?

Factorial Planning process
Choosing a Design
Case Study basics of analysis
7 Keys to Experimentation
What is Design of Experiments?
Controllable Factors x
DOE (Design of Experiments) is:

A systematic series of tests, in
which purposeful changes are
made to input factors,
Process
Noise Factors z
Responses y
so that you may identify
causes for significant changes
in the output responses.
It is NOT analyzing historical data!

5
Iterative Experimentation
Conjecture
Analysis
Design
Experiment
Expend no more than 25% of budget on the 1st cycle.
DOE versus OFAT

Traditional Approach to Experimentation
Study one factor at a time (OFAT), holding all other variables
constant
Simple process, but doesnt account for interactions
It is inefficient (serial processing)
Factorial Design
Study multiple factors changing at once
(parallel processing)
Accounts for interactions between variables
Maximize information with minimum runs
Agenda Transition
Intro What is DOE?
Choosing a Design
Factorial DOE Process (1 of 2)

1. Identify opportunity and define objective.
2. State objective in terms of measurable responses.

a. Define the change (Dy) that is important to detect for each
response.
b. Estimate experimental error (s) for each response.
c. Use the signal to noise ratio (Dy/s) to estimate power.
3. Select the input factors and the levels to study. (Remember
that the factor levels chosen determine the size of Dy.)
Factorial DOE Process (2 of 2)

4. Select a design and:
Evaluate aliases.
Evaluate power.
Examine the design layout to ensure all the factor
combinations are safe to run and are likely to result in
meaningful information (no disasters).
10
Factorial Design Worksheet

Use the Factorial Planning Worksheet as a guide:
Factorial Planning.pdf
11
Rev 11/13/09
Factorial Design Worksheet

Identify opportunity and define objective: __________________
________________________________________________
________________________________________________
State objective in terms of measurable responses:
Define the change (y - signal) you want to detect.
Estimate the experimental error ( - noise)

Use y/ (signal to noise) to check for adequate power.
Name
Units
y/
Power
Goal
R1:
R2:
R3:
R4:
Select the input factors and ranges to vary within the experiment:
Remember that the factor levels chosen determine the size of y.
Name
Units
Type
Low (1)
High (+1)
A:
B:
C:
D:
E:
F:
G:
H:
J:
K:
L:
M:
Choose a design: Type:____________________________________
Replicates: ____,
Blocks: _____,
1-7
Center points: ____
Agenda Transition
Intro What is DOE?
Choosing a Design
12
Choosing a Design
Opening screen (after selecting File, New Design)
Find more guidance
in the Help System.
13
Choosing a Design
The Help System,
along with the screen
tips button,
provide excellent
guidance.
Click on them when
you have questions!
14
Choosing a Design (the stoplight)

Resolution III (Red Stop and Think)
Main effects aliased (confused) with two

factor interactions.
Can be misleading when significant twofactor interactions affect the response.
Resolution IV (Yellow Proceed with Caution)

Main effects aliased with three-factor interactions.
Two-factor interactions may be aliased with other two-factor
interactions.
Good choice for a screening design because the main effects
can be estimated cleanly, clear of two-factor interactions.
15
Choosing a Design (the stoplight)

Resolution V (Green Go Ahead)
All the main effects and two-factor

interactions can be estimated (assuming
three-factor interactions are negligible)
Full Factorials (White No aliasing)
All factor effects are estimated no

confounding at all
16
Choosing a Design Min Run Options

Opening screen (after selecting File, New Design)
17
Choosing a Design: Minimum-Run Designs

Screening
Characterization
Factors
Std Res V
MR5*
Factors
Std Res IV MR4**
32
22
32
18
64
30
10
32
20
64
38
11
32
22
128
46
12
32
24
10
128
56
13
32
26
11
128
68
14
32
28
12
256
80
15
32
24
13
256
92
16
32
26
14
256
106
17
64
28
* Oehlert & Whitcomb, Small, Efficient, Equireplicated Resolution V Fractions of 2k designs ,

Fall Technical Conference, 2002: www.statease.com/pubs/small5.pdf
** Anderson & Whitcomb, Screening Process Factors In the Presence of Interactions, Annual Quality
Congress, American Society of Quality, Toronto, 2004: www.statease.com/pubs/aqc2004.pdf
18
Choosing a Design - Strategy

Resolution IV (yellow)
fractional factorials are
appropriate when:
screening for unknown
but important factors
runs must be limited.

Consider using the
Minimum-Run Res IV
designs.
19

Resolution V (green)
fractional factorials or full
factorials are appropriate
when:
you need to uncover
interactions
runs are not so limited.
Add center points to detect
curvature.
Consider using the
Minimum Run Res V
designs!
20

Response surface method
(RSM) designs are
appropriate when:
the goal is optimization
vital factors are known
they create non-linear
effects, that is,
significant curvature
ranges are defined.
The central composite
design (CCD) serves well.
21
10
Choosing a Design Wrap Up
Always confirm your results!

Do so with a number of runs
at the optimal conditions and
assess the outcome against
the appropriate prediction
interval.
If you see discrepancies, go
back and investigate what
may have changed in your
system.
22
11
23
24
We will analyze the first response Taste together and then you will analyze the second
response UPKs (weight of un-popped kernels, technically called UPKs by popcorn
manufacturers). Then together we will look for the best operating conditions.
See reference for complete report. You can find it on the www.statease.com website.
25
Note that in this array, listed in randomized run order, the actual levels of factors are
shown. We call this an uncoded or actual representation of factors. Its useful for the
technician (or cook) who runs the experiment, but not for calculations.
26
Better coded (pun off butter coated) factor levels, run order and standard order.
27
Enter factor information.
28
When we have to type in the data to match a classroom exercise, standard order ensures
that everyone enters the data in the same order.
29
Set up the popcorn experiment in Design-Expert software.
30
This is the expanded design matrix in standard order. It includes columns for all eight of
the effects we can evaluate with the eight runs: the overall mean (column labeled I for
identity) plus seven effects -- three main effects (MEs), three 2-factor interactions (2FIs)
and one 3-factor interaction (3FI).
How are the signs for the interaction columns computed? Answer by multiplying the
main effect columns.
31
The software does all the effect calculations using the expanded design matrix.
The % contribution was added at the request of a major client who likes to highlight the
heavy hitters on a sum of squares basis. This can be somewhat confusing when
interactions are significant.
32
Heres the basic way of calculating effects: the average of A at its high setting minus the
average of A at its low setting.
33
Design-Expert ranks the absolute values of the effects from low to high and constructs a
half-normal probability plot. The significant effects fall to the right on this plot. Starting on
the right select the largest effects. Look for a definite gap between the keepers and the
trivial many effects near zero.
34
The effects must be split into the significant keepers and the non-significant ones that
are used to estimate noise.
35
After using the half-normal plot to pick effects, look at the Pareto Chart to reinforce your
selection. This is a good way to communicate to others because its simply an ordered
bar-chart that anyone can figure out, unlike the half-normal plot, which most people
havent ever seen before.
Heres the guidelines for assessing effects on this Pareto Chart designed especially for
this purpose:
(Big) effects above the Bonferroni Limit (a conservative statistical correction for
multiple comparisons*) are almost certainly significant.
(Intermediate) effects that are above the lower t-Value Limit are possibly
significant.
(Small) effects below the t-Value limit are probably not significant.
*(For details, see DOE Simplified, 2nd Ed, Chapter 3 How to Make a More Useful
Pareto Chart.)
36
Model Sum of Squares (SS): Total of the sum of squares for B, C, and BC (the selected factors).
SSModel = 840.50 + 578.00 + 924.50 = 2343.00
Model DF (Degrees of Freedom): Number of model parameters, not including the intercept.
dfModel = 3
Model Mean Square (MS): Estimate of model variance.
MSModel = SSModel/dfModel = 2343.0/3 = 781.0
Residual Sum of Squares: Total SS for the terms estimating experiment error, those that fall on
the normal probability line.
SSResidual = 2.00 + 0.50 + 72.00 + 24.50 = 99.00
Residual df (Degrees of Freedom)
dfResidual = Corrected Total df - Model df = 7 - 3 = 4
Residual Mean Square (MS): Estimate of error variance.
MSResidual = SSResidual/dfResidual = 99.00/4 = 24.75
F Value: Test for comparing model variance with residual variance.
= MSModel/MSResidual = 781.00/24.75 = 31.56
Prob > F: Probability of observed F value if the null hypothesis is true. The probability equals the
tail area of the F-distribution (with 3 and 4 DF) beyond the observed F-value. Small probability
values call for rejection of the null hypothesis.
Cor Total: Total sum of squares corrected for the mean, SS = 2442.00 and df = 7.
37
The F-distributions and associated statistics are named after Sir Ronald Fisher. He
developed the technique for application to agricultural experiments. In fact, Fishers
landmark paper is entitled The Differential Effect of Manures on Potatoes.
38
Std. Dev.: Square root of the Residual mean square (sometimes referred to as Root MSE).
= SqRt(24.75) = 4.97
Mean: Overall average of the response.
C.V. (Coefficient of Variation): The standard deviation as a percentage of the mean.
= 100 x (Std. Dev.)/Mean = 100 x 4.97/66.50 = 7.48%
R-squared: The multiple correlation coefficient.
= 1 - [SSResidual/(SSModel + SSResidual)]
= 1 - [99.00/(2343.00+99.00)] = 0.9595
Adj R-Squared: R-Squared adjusted for the number of model parameters relative to the number of runs.
= 1 - {[SSResidual/DFResidual]/[(SSModel + SSResidual)/(DFModel + DFResidual)]}
= 1 - {[99.00/4]/[(2343.001+ 99.00)/(3 + 4)]} = 0.9291
Pred R-Squared: Predicted R-Squared. A measure of the predictive capability of the model.
= 1 - (PRESS/SSCor Total) = 1 - (396.00/2442.00) = 0.8378
Adeq Precision: Compares the range of predicted values at design points to the average prediction error.
Ratios greater than four indicate adequate model discrimination.
PRESS (Predicted Residual Sum of Squares): A measure of how this particular model fits each point in the
design. The coefficients are calculated without the first point. This model is then used to estimate the first
point and calculate the residual for point one. This is done for each data point and the squared residuals
are summed. Used to calculate the Pred R-Squared.
39
Coefficient Estimate: The coefficients listed in the factorial post-ANOVA section are
based on coded (low level = -1, high level = +1) units. The graph shows how the
coefficients relate to the effects. Get into the habit of using coded units, because this
makes interpretation much easier.
Standard Error: The standard deviation associated with the coefficient estimate.
95% CI: 95% confidence interval on the coefficient estimate. An interval calculated to
bracket the true coefficient 95% of the time. These intervals exclude 0 when significant.
They convey the uncertainty that comes from variability in the sample data.
VIF (variance inflation factor): Measures how much the variance of the coefficient is
inflated by the lack of orthogonality in the design. If the coefficient is orthogonal to all the
other coefficients in the model, the VIF is one.
40
Design-Expert provides equations to predict response using coded or actual (original)

units. Coding makes direct comparisons between coefficients possible. Otherwise they
change depending on the unit of measure.
Lets see how the coded prediction model works by plugging in the values B=-1 and
C=-1:
y
= 66.50 -10.25(-1) -8.50(-1) -10.75(-1)(-1)

= 66.50 +10.25 +8.50 -10.75
= 74.50
41
Heres the equation in terms of actual factor levels. The coefficients are quite different
from those in the coded equation. They depend on the units of measure. In some cases
even the sign changes.
Lets see how the actual prediction model works by plugging in the values B=4, and C=75.
y
= -199.00 +65.00(4) +3.62(75) -0.86(4)(75)

= -199.00 +260.00 + 271.50 258.00
= 74.50
Another problem with use of actual measures is round-off error. Notice the additional
decimal places that the software lists to avoid this problem.
42
This slide recaps the two forms of predictive models. We recommend you work only with
the coded equation. Then you can do a fair comparison of effects.
Remember, regardless of the form of model, do not extrapolate except to guess at
conditions for the next set of experiments. The model is only an approximation, not the
real truth. It's good enough to help you move in proper direction, but not to make exact
predictions, particularly outside the actual experimental region.
43
Examine the residuals to look for patterns that indicate something other than noise is
present.
If the residual is pure noise (it contains no signal), then the analysis is complete.
44
Design-Expert software produces a table of case statistics on Diagnostics button,

Influence and Report; see Diagnostics Report Formulas & Definitions in your
Handbook for Experimenters.
Plots of the case statistics are used to validate our model.
45
The diagnostic plots of the case statistics are used to check these assumptions.
Note: Studentized residuals are the raw residual normalized by dividing it by its
standard error. In the case of a non-orthogonal or unbalanced design the raw
residuals are not members of the same normal distribution because they have
different standard errors. Studentizing them maps them all to the standard normal
distribution, puts them all on an equal basis and allows them to go on the same
plots. The studentized residuals have a standard error of 1 and therefore the three
sigma limits are just 3.
46
The normality plot of residuals is used to confirm the normality assumption:

If all is okay, residuals follow a straight line, i.e., are normal.
Some scatter is expected, look for definite patterns, e.g., "S" shape.
NOTE: Although there are statistical tests available for checking normality (i.e. AndersonDarling), they are not appropriate in this case because the residuals are
correlated with each other, violating a fundamental assumption of the statistical
test for normality. So, we have to make do with a simple visual evaluation.
The residuals vs predicted plot is used to confirm the constant variance assumption:
All the points should be within the three sigma limits.

Variance (scatter) should be approximately constant over the range of predictions.
Look for definite patterns, e.g., megaphone <" shape.
47
The residuals vs run chart (yours may be different due to the differing random run order)
is used to confirm the independence assumption:
All the points should be within the three sigma limits.
Should exhibit approximately random scatter, i.e. no trends, with time.
Use the predicted vs actual plot to see how the model predicts over the range of data:
Plot should exhibit random scatter about the 450 line.
Clusters of points above or below the line indicate problems of over or under
predicting.
48
The Box-Cox plot tells you whether a transformation of the data may help. Without
getting into the details just yet, notice that it says none for recommended
transformation. Thats all you need to know for now.
49
Note: your graphs are likely different due to the differing random run order
The externally studentized residuals plot is used to identify model and data problems:
Look for values outside the red limits. A high value indicates a particular run does
not agree well with the rest of the data, when compared using the current model.
The DFFITS is not as much of a diagnostic plot as a problem solving tool. Use it when
the other graphics are indicating problems. DFFITS stands for Difference in Fits.
Look for values outside the blue limits. A high value indicates the predicted
response for a particular run changes when that particular run is removed form the
regression.
50
In this case, the software warns you that factor B is involved in an interaction. You might
say its a parent term. Do not make plots of any main effect that is a "parent".
Remember The significant BC interaction means that the effect of factor B depends on
the level of factor C. The one-factor plot for B averages over the two levels of factor C.
51
Now you get the whole story about factor B. Note the difference in the time effect at the
two levels of concentration. At 75% power there is no difference between 4 and 6
minutes. At 100% power there is a time effect that is twice that seen on the previous
(incorrect) slide of the main effect of time.
The LSD bars are visual aids in helping to interpret effect plots. If the LSD bars for two
means overlap, the difference in those means is not large enough to be declared
significant using a t-test. Note that the LSD bars for all the means except the one at 6
minutes and 100% power overlap (cover the same Taste range.)
52
The contour plot of the BC interaction clearly dispels the myth that two-level designs can
only fit linear (1st order) models. Remember that 2FIs are second order terms.
3D plot of BC interaction show that 2FIs allow for twisting the plane, but do not allow for
hills or depressions; squared terms (e.g. B2 and C2) are required for that.
53
When looked at end-on, the 3D graph shows where the interaction plot comes from.
Since the surface is a twisted plane the interaction graph can capture all the curvature.
54
Now it is time for you to exercise your new knowledge.
55
How would you set up the microwave for best popcorn taste and yield? (Answer: high
power and low time.)
This slide was made by using the pop-out view feature in Design-Expert software,
available as a button on the floating Graphs Tool:
56
Agenda Transition
Intro What is DOE?

Choosing a Design

57
1.
2.
3.
4.
5.
6.
7.
Set good objectives

Invest in measurement
Assess power of the design
Randomize the runs
Know which effects (if any) will be aliased
Do a sequential series of experiments
Always confirm critical findings
Reference: Excepted from article posted at http://www.statease.com/pubs/doe-keys.pdf
58
Key #1: Set good objectives

To avoid getting swamped with useless data, establish a clear
purpose for collecting it:
Why are you doing the DOE?
What data will you collect?
How will it help you improve quality, reduce costs or decrease

cycle time?
59
Key #2: Invest in measurement

Must be quantitative stats require numbers!
Precision provides power to reduce costly runs

Option: do repeated testing
Accuracy is ideal, but consistency suffices to achieve relative
improvement
To assess quality: Continuous > count > pass/fail
Rating scale will do, such as 1, 2, 3, 4, 5
60
Key #3: Assess Power of the Design

1.
Quantify overall variation in your system via:

Historical data
Control charts (r-bar over d2)
Process capability study (gage R&R)
Analysis of variance (ANOVA) from prior DOE
Guess based on experience (SWAG)
2.
Determine signal (change in response) of minimum

importance
3.
Confirm design has enough runs needed for

adequate power (>80 %)
61
Key #4: Randomize the runs*

Provides a valid basis for statistical inference and counteracts
confounding from time-related lurking variables, such as
Temperature rising
People tiring and machines wearing
Stuff happening (Murphys law, gremlins, bugs, etc.)
Never run all Low and then all High!
*Refer to George Box, Must We Randomize Our Experiment?

Quality Engineering, 1990, V2, # 4, pp. 497-502, based on Report 47 at
www.engr.wisc.edu/centers/cqpi/reports.html
62
Key #5: Know which effects (if any) will

be aliased
(Aliasing was only reiewed briefly in this webinar
but is VERY Important to understand.)
A problem that comes with any fractional DOE
If runs cheap, play it safe via Resolution V or better
Resolution IV designs OK for screening
Consider Min-Run Res IV
Only do this when you can do a follow-up experiment
Resolution III designs dangerous: ME=2FI

o
Be very wary of: Taguchi, Plackett-Burman

Foldover needed to get Res III back to Res IV
63
Key #6: Experiment sequentially
Invest only 25% in first DOE
Screen first with Res IV to discover previously unknown

factors that may be of vital importance
Add center points at next stage of characterization (Res V)
Look for droops or bumps (Cuthbert Daniel)
Stage now set for response surface methods (RSM)
64
Key #7: Always confirm critical findings
This is the true test of science do not assume
Use prediction intervals to manage expectations on results

from single follow-up runs
Do not expect miracles: Stuff will continue to happen
It is easy to be fooled in science,

and the easiest one to fool is yourself.
- Richard Feynman
65
Practical Paperbacks on DOE*

by Mark Anderson and Pat Whitcomb
User Review of DOE Simplified:

As an engineer (just beginning self study on the topic of DOE) I found this book
very useful. The authors provide practical insight that I was unable to find in other
DOE or statistics books. This is not a book for advanced statisticians, however, it is
a great book for someone trying to understand and apply the principles of DOE.
* Published by Productivity Press, New York.
66
Statistics Made Easy

For all the new features in v8 of Design-Expert software, see
www.statease.com/dx8descr.html
Best of luck for your experimenting!

Thanks for listening!
-- Shari
Shari Kraber, MS, Applied Stats
Stat-Ease, Inc.
Shari@statease.com
*Pdf of this Powerpoint presentation posted at www.statease.com/webinar.html.
For future webinars, subscribe to DOE FAQ Alert at www.statease.com/doealert.html.
67

How To Get Started With Design-Expert

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

How To Get Started With Design-Expert

Uploaded by

Copyright:

Available Formats

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Getting Started: Stat-Ease Resources

On the statease.com website:

Getting Started: Other Resources

Take advantage of all the free resources available to you!

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Intro What is DOE?

What is Design of Experiments?

DOE (Design of Experiments) is:

It is NOT analyzing historical data!

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

DOE versus OFAT

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Factorial DOE Process (1 of 2)

2. State objective in terms of measurable responses.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Factorial DOE Process (2 of 2)

Factorial Design Worksheet

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Factorial Design Worksheet

Estimate the experimental error ( - noise)

Center points: ____

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Choosing a Design (the stoplight)

Main effects aliased (confused) with two

Can be misleading when significant twofactor interactions affect the response.

Resolution IV (Yellow Proceed with Caution)

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Choosing a Design (the stoplight)

All the main effects and two-factor

Full Factorials (White No aliasing)

All factor effects are estimated no

Choosing a Design Min Run Options

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Choosing a Design: Minimum-Run Designs

Std Res IV MR4**

* Oehlert & Whitcomb, Small, Efficient, Equireplicated Resolution V Fractions of 2k designs ,

Choosing a Design - Strategy

runs must be limited.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Choosing a Design - Strategy

Choosing a Design - Strategy

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Choosing a Design Wrap Up

Always confirm your results!

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Enter factor information.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Set up the popcorn experiment in Design-Expert software.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.

Copyright 2013 Stat-Ease, Inc. Do not copy or redistribute in any form.