28/05/13

5.5.3.1.4. Single response: Optimization when there is adequate quadratic fit

5. Process Improvement 5.5. Advanced topics 5.5.3. How do you optimize a process? 5.5.3.1. Single response case

5.5.3.1.4. Single response: Optimization when there is adequate quadratic fit

Regions where quadratic models or even cubic models are needed occur in many instances in industry After a few steepest ascent (or descent) searches, a first-order model will eventually lead to no further improvement or it will exhibit lack of fit. The latter case typically occurs when operating conditions have been changed to a region where there are quadratic (second-order) effects present in the response. A second-order polynomial can be used as a local approximation of the response in a small region where, hopefully, optimal operating conditions exist. However, while a quadratic fit is appropriate in most of the cases in industry, there will be a few times when a quadratic fit will not be sufficiently flexible to explain a given response. In such cases, the analyst generally does one of the following: 1. Uses a transformation of Y or the Xis to improve the fit. 2. Limits use of the model to a smaller region in which the model does fit. 3. Adds other terms to the model. Procedure: obtaining the estimated optimal operating conditions Secondorder polynomial model Once a linear model exhibits lack of fit or when significant curvature is detected, the experimental design used in Phase I (recall that a 2k -p factorial experiment might be used) should be augmented with axial runs on each factor to form what is called a central composite design. This experimental design allows estimation of a second-order polynomial of the form

Steps to find optimal operating conditions

If the corresponding analysis of variance table indicates no lack of fit for this model, the engineer can proceed to determine the estimated optimal operating conditions. 1. Using some graphics software, obtain a contour plot of the fitted response. If the number of factors (k) is greater than 2, then plot contours in all planes corresponding to all the possible pairs of factors. For k greater than, say, 5, this could be too cumbersome (unless the graphic software plots all pairs automatically). In such a case, a "canonical analysis" of the surface is recommended (see Technical Appendix 5D). 2. Use an optimization solver to maximize or minimize (as desired) the estimated response 3. Perform a confirmation experiment at the estimated optimal operating conditions given by the solver in step 2. .

Chemical We illustrate these steps using the chemical experiment discussed previously. For a technical experiment description of a formula that provides the coordinates of the stationary point of the surface, see example Technical Appendix 5C. www.itl.nist.gov/div898/handbook/pri/section5/pri5314.htm#Technical Appendix

1/7

28/05/13

5.5.3.1.4. Single response: Optimization when there is adequate quadratic fit

example

Technical Appendix 5C. Example: Second Phase Optimization of Chemical Process

Experimental results for axial runs

Recall that in the chemical experiment, the ANOVA table, obtained from using an experiment run around the coordinates X1 = 189.5, X2 = 350, indicated significant curvature effects. Augmenting the 22 factorial experiment with axial runs at to achieve a rotatable central composite experimental design, the following experimental results were obtained: x1 -1.414 +1.414 0 0 x2 0 0 -1.414 +1.414 X1 147.08 231.92 189.5 189.5 X2 350 350 279.3 420.7 Y (= yield) 72.58 37.42 54.63 54.18

ANOVA table

The ANOVA table corresponding to a cubic model with an interaction term (contained in the quadratic sum-of-squares partition) is
S O U R C E M E A N L i n e a r Q u a d r a t i c C u b i c R E S I D U A L T O T A L S U MO F S Q U A R E S 5 1 4 1 8 . 2 1 1 1 3 . 7 7 6 8 . 1 9 . 9 2 2 3 . 1 5 3 5 3 3 . 0 D F 1 2 3 2 5 1 3 M E A N S Q U A R E 5 1 4 1 8 . 2 5 5 6 . 8 2 5 6 . 0 5 . 0 4 4 . 6 F V A L U E P R O B>F

5 . 5 6 7 . 6 9 0 . 1 1

0 . 0 2 4 0 . 0 1 3 0 . 8 9 7

Lack-of-fit tests and auxillary diagnostic statistics

From the ANOVA table, the linear and quadratic effects are significant. The lack-of-fit tests and auxiliary diagnostic statistics for linear, quadratic, and cubic models are:
M O D E L L i n e a r Q u a d r a t i c C u b i c P U R EE R R O R S U MO F S Q U A R E S 8 2 7 . 9 5 9 . 9 4 9 . 9 1 7 3 . 2 R O O T M S E 1 0 . 0 1 5 . 7 7 6 . 6 8 D F 6 3 1 4 M E A N S Q U A R E 1 3 8 . 0 2 0 . 0 4 9 . 9 4 3 . 3 A D J R S Q R 0 . 4 3 1 9 0 . 8 1 1 1 0 . 7 4 6 8 P R E D R S Q R 0 . 2 4 2 5 0 . 6 7 0 8 0 . 6 3 9 3 F V A L U E P R O B>F 3 . 1 9 0 . 4 6 1 . 1 5 0 . 1 4 1 0 . 7 2 5 0 . 3 4 3

M O D E L L i n e a r Q u a d r a t i c C u b i c

R S Q R 0 . 5 2 6 6 0 . 8 8 9 8 0 . 8 9 4 5

P R E S S 1 6 0 2 . 0 2 6 9 6 . 2 5 3 4 6 6 . 7 1

The quadratic model has a larger p-value for the lack of fit test, higher adjusted R2, and a lower PRESS statistic; thus it should provide a reliable model. The fitted quadratic equation, in coded units, is

Step 1: Contour plot A contour plot of this function (Figure 5.5) shows that it appears to have a single optimum point in
2/7

www.itl.nist.gov/div898/handbook/pri/section5/pri5314.htm#Technical Appendix

28/05/13

Contour plot of the fitted response function

5.5.3.1.4. Single response: Optimization when there is adequate quadratic fit A contour plot of this function (Figure 5.5) shows that it appears to have a single optimum point in the region of the experiment (this optimum is calculated below to be (-0.9285, 0.3472), in coded x1 , x2 units, with a predicted response value of 77.59).

FIGURE 5.5: Contour Plot of the Fitted Response in the Example 3D plot of the fitted response function Since there are only two factors in this example, we can also obtain a 3D plot of the fitted response against the two factors (Figure 5.6).

www.itl.nist.gov/div898/handbook/pri/section5/pri5314.htm#Technical Appendix

3/7

28/05/13

5.5.3.1.4. Single response: Optimization when there is adequate quadratic fit

FIGURE 5.6: 3D Plot of the Fitted Response in the Example Step 2: Optimization point An optimization routine was used to maximize . The results are = 161.64oC, (X* ) = 77.59 %. = 367.32

minutes. The estimated yield at the optimal point is Step 3:

Confirmation experiment

A confirmation experiment was conducted by the process engineer at settings X1 = 161.64, X2 = 367.32. The observed response was (X* ) = 76.5 %, which is satisfactorily close to the estimated optimum. ================================================================== Technical Appendix 5C: Finding the Factor Settings for the Stationary Point of a Quadratic Response

How to find the maximum or minimum point for a quadratic response

1. Rewrite the fitted equation using matrix notation as

where b' = (b1 , b2 , ..., bk ) denotes a vector of first-order parameter estimates,

is a matrix of second-order parameter estimates and x' = (x1 , x2 , ..., xk ) is the vector of controllable factors. Notice that the off-diagonal elements of B are equal to half the twofactor interaction coefficients. 2. Equating the partial derivatives of with respect to x to zeroes and solving the resulting system of equations, the coordinates of the stationary point of the response are given by

roots or eigenvalues of B to see whether B is "positive definite" (so x* is a point of minimum response) or "negative definite" (the case in which x* is a point of maximum response). This task is easier if the two-factor interactions are "eliminated" from the fitted equation as is described in www.itl.nist.gov/div898/handbook/pri/section5/pri5314.htm#Technical Appendix

Nature of the stationary point is determined by B

The nature of the stationary point (whether it is a point of maximum response, minimum response, or a saddle point) is determined by the matrix B. The two-factor interactions do not, in general, let us "see" what type of point x* is. One thing that can be said is that if the diagonal elements of B (bii) have mixed signs, x* is a saddle point. Otherwise, it is necessary to look at the characteristic

4/7

28/05/13

5.5.3.1.4. Single response: Optimization when there is adequate quadratic fit

easier if the two-factor interactions are "eliminated" from the fitted equation as is described in Technical Appendix 5D. Example: computing the stationary point, Chemical Process experiment Example of computing the stationary point The fitted quadratic equation in the chemical experiment discussed in Section 5.5.3.1.1 is, in coded units,

from which we obtain b' = (-11.78, 0.74),

and

Transforming back to the original units of measurement, the coordinates of the stationary point are

. The predicted response at the stationary point is (X* ) = 77.59 %.

Technical Appendix 5D: "Canonical Analysis" of Quadratic Responses Case for a single controllable response Whether the stationary point X* represents a point of maximum or minimum response, or is just a saddle point, is determined by the matrix of second-order coefficients, B. In the simpler case of just a single controllable factor (k=1), B is a scalar proportional to the second derivative of (x) with respect to x. If d2 /dx2 is positive, recall from calculus that the function (x) is convex ("bowl shaped") and x* is a point of minimum response. Unfortunately, the multiple factor case (k>1) is not so easy since the two-factor interactions (the off-diagonal elements of B) obscure the picture of what is going on. A recommended procedure for analyzing whether B is "positive definite" (we have a minimum) or "negative definite" (we have a maximum) is to rotate the axes x1 , x2 , ..., xk so that the two-factor interactions disappear. It is also customary (Box and Draper, 1987; Khuri and Cornell, 1987; Myers and Montgomery, 1995) to translate the origin of coordinates to the stationary point so that the intercept term is eliminated from the equation of (x). This procedure is called the canonical analysis of (x). Procedure: Canonical Analysis Steps for performing the canonical analysis 1. Define a new axis z = x - x* (translation step). The fitted equation becomes . 2. Define a new axis w = E'z, with E'BE = D and D a diagonal matrix to be defined (rotation step). The fitted equation becomes .
www.itl.nist.gov/div898/handbook/pri/section5/pri5314.htm#Technical Appendix 5/7

Case for multiple controllable responses not so easy

28/05/13

5.5.3.1.4. Single response: Optimization when there is adequate quadratic fit

This is the so-called canonical form of the model. The elements on the diagonal of D, i (i = 1, 2, ..., k) are the eigenvalues of B. The columns of E', ei, are the orthonormal eigenvectors of B, which means that the ei satisfy (B - i)ei = 0, = 1.0. = 0 for i j, and

3. If all the i are negative, x* is a point of maximum response. If all the i are positive, x* is a point of minimum response. Finally, if the i are of mixed signs, the response is a saddle function and x* is the saddle point. Eigenvalues that are approximately zero If some i 0, the fitted ellipsoid

is elongated (i.e., it is flat) along the direction of the wi axis. Points along the wi axis will have an estimated response close to optimal; thus the process engineer has flexibility in choosing "good" operating conditions. If two eigenvalues (say i and j) are close to zero, a plane in the (wi, wj) coordinates will have close to optimal operating conditions, etc. Canonical analysis typically performed by software Software is available to compute the eigenvalues i and the orthonormal eigenvectors ei; thus there is no need to do a canonical analysis by hand.

Example: Canonical Analysis of Yield Response in Chemical Experiment B matrix for this example Let us return to the chemical experiment example to illustrate the method. Keep in mind that when the number of factors is small (e.g., k=2 as in this example) canonical analysis is not recommended in practice since simple contour plotting will provide sufficient information. The fitted equation of the model yields

Compute the eigenvalues and find the orthonormal eigenvectors Canonical analysis results

To compute the eigenvalues i, we have to find all roots of the expression that results from equating the determinant of B - iI to zero. Since B is symmetric and has real coefficients, there will be k real roots i, i = 1, 2, ..., k. To find the orthonormal eigenvectors, solve the simultaneous equations (B - iI)ei = 0 and = 1. The results of the canonical analysis are as follows:
E i g e n v e c t o r s E i g e n v a l u e s 4 . 9 7 3 1 8 7 9 . 8 2 7 3 1 7 X 1 0 . 7 2 8 4 6 0 0 . 6 8 5 0 8 9 X 2 0 . 6 8 5 0 8 9 0 . 7 2 8 4 6 0

Notice that the eigenvalues are the two roots of

www.itl.nist.gov/div898/handbook/pri/section5/pri5314.htm#Technical Appendix 6/7

28/05/13

5.5.3.1.4. Single response: Optimization when there is adequate quadratic fit

As mentioned previously, the stationary point is (x* )' = (-0.9278, 0.3468), which corresponds to X* ' = (161.64, 367.36). Since both eigenvalues are negative, x* is a point of maximum response. To obtain the directions of the axis of the fitted ellipsoid, compute w1 = 0.7285(x1 + 0.9278) - 0.6851(x2 - 0.3468) = 0.9143 + 0.7285x1 - 0.6851x2 and w2 = 0.6851(x1 + 0.9278) - 0.7285(x2 - 0.3468) = 0.8830 + 0.6851x1 + 0.7285x2 Since | 1 | < | 2 |, there is somewhat more elongation in the wi direction. However, since both eigenvalues are quite far from zero, there is not much flexibility in choosing operating conditions. It can be seen from Figure 5.5 that the fitted ellipses do not have a great elongation in the w1 direction, the direction of the major axis. It is important to emphasize that confirmation experiments at x* should be performed to check the validity of the estimated optimal solution.

www.itl.nist.gov/div898/handbook/pri/section5/pri5314.htm#Technical Appendix

7/7