View Article Online / Journal Homepage / Table of Contents for this issue

Analyst, December 1997, Vol. 122 (1521–1529) 1521

Multilevel Multifactor Designs for Multivariate

Calibration

Richard G. Brereton
School of Chemistry, University of Bristol, Cantock’s Close, Bristol, UK BS8 1TS

Multilevel experiments are common in chemistry,
especially in calibration and mixture problems. This
paper presents designs for l = 2–5 different concentration
levels and l2 corresponding experiments. The importance
of orthogonality between successive factors is discussed. It
Published on 01 January 1997. Downloaded on 13/07/2014 10:06:21.
and x2 are the two factors. Fig. 2(a) illustrates this graph (which
relates to confidence bands), which is completely circular, fairly
flat and has a value of 0.5 in the four extreme corner points
(±2,±2). In contrast, the equation for the design of Fig. 2(b) is h
= 0.15625x21 + 0.15625x22 + 0.1875x1x2. This makes a dramatic

is shown that up to 12 mutually orthogonal factors can be difference to the graph of leverage, as shown in Fig. 2(b). The
generated for a five-level design. Methods of generating graph is now asymmetrical and the values of leverage for
designs are generalised. The designs are restricted to (22,22) and (+2,+2) are now equal to 2, whereas for (22,+2)
first-order (linear) models, typical of most instrumental and (+2,22) they remain at 0.5. This implies low confidence
calibration experiments. and poor modelling ability for certain regions of the mixture
domain, which can have substantial practical implications.
Keywords: Experimental design; chemometrics; calibration;
Suppose, for example, acetone and methanol are part of a
mixtures
mixture experiment, and their high and low concentrations are
0.1 and 0.5 mm. If the second design is employed, this may
imply that, whereas good predictions can be achieved if one
Despite substantial interest in mixture experiments in spectro- compound is at a high concentration and the other at a low
scopy and chromatography and the analysis of such multivariate concentration, the case where both are at high or low
datasets by chemometrics methods such as multivariate calibra- concentrations cannot be predicted with much confidence. If
tion1–3 and factor analysis,4,5 there is limited literature on how methanol and acetone happen to be at 0.15 mm concentrations,
to design such experiments. A typical multivariate calibration no matter what the concentrations of the other components in
experiment involves spectra of several compounds, recorded at the mixture are, they will not be predicted well, even though a
several concentration levels. For one factor, there is literature on concentration of 0.15 mm methanol might be predicted very
how to design calibration experiments.6–8 Although some well in the presence of 0.45 mm acetone and vice versa. In
designs such as the central composite design9,10 exist for three practical cases, a series of mixtures may be made in the
or four factors, these are not really aimed at calibration laboratory and a model developed using a multivariate approach
experiments; the main purpose is often optimisation or simply such as PLS to predict the concentrations of compounds from
to see whether certain factors are significant. spectra using experimental mixtures. However, if the laboratory
In many cases where multivariate calibration is employed, experiments do not adequately cover the mixture space,
great effort is made to ensure instrumental responses are linear, predictions could be dramatically in error under certain
so that linear models are appropriate for analysis of the data in situations.
hand. In this paper we restrict designs to such linear models;
alternative strategies would be required if non-linearities and
interactions are expected.
In many real-world situations there are a large number of
components in a mixture. For example, we have reported the
study of polyaromatic hydrocarbons in airborne emissions:11 at
least 16 can be identified from standard EPA samples; a further
30 or 40 may remain unidentified. In order to build up models
in the laboratory or even using computers, experiments
involving mixing together compounds in different proportions
and recording the resultant spectra are commonly employed.
However, in order to obtain sensible calibration models, several
Fig. 1 Designs consisting of two factors, and five experiments. The levels
concentration levels (at least four or five per compound) are are coded so that the centre is 0.
required.
In addition, it is essential that the compounds are uniformly
distributed over mixture space. This latter condition is often
overlooked. An unbalanced design may, for example, result in
poor predictions for unexpected and fairly subtle combinations
of component concentrations. Fig. 1 illustrates two factor
designs each consisting of five experiments. The first design is
completely symmetrical and rotatable, whereas the second
contains some asymmetry about the diagonal. Confidence in
predictions can be estimated using leverage;12,13 the higher the
value the worse the prediction. Leverage provides a method for
the estimation of the shape of confidence bands in a model: if
leverage is symmetrical, so is confidence and the related Fig. 2 Leverage corresponding to the designs of Fig. 1. (a) Contour levels
parameter of uncertainty. For the symmetric design of Fig. 1, the 0.1, 0.2, 0.3 and 0.4 from centre; (b) contour levels 0.4, 0.8, 1.2, and 1.6
equation for leverage is h = 0.0625x12 + 0.0625x22, where x1 from centre.

1522 Analyst, December 1997, Vol. 122
For most methods of multivariate calibration, several concen-
tration levels are necessary to obtain an adequate model.
Consider a mixture containing 10 compounds. If five levels are
to be studied, then, in order to obtain all possible combinations
of levels, 510 or nearly 107 experiments would be required,
which is clearly impracticable. Hence the number of experi-
ments must be reduced, but certain key properties must be
retained in the designs.
For two level designs, there is an extensive literature about
how to reduce the number of experiments. Fractional factori-
als14–16 and Plackett–Burman17,18 designs are well known. The
original paper of Plackett and Burman17 proposes a possible
extension to more than two-level designs, but this is not well
known and has not been generalised or used by chemists. The
purpose of this paper is to propose methods for generalised
multifactor, multilevel designs that can safely be used by
Published on 01 January 1997. Downloaded on 13/07/2014 10:06:21.
experimentalists for models of experimental mixtures employed
in multivariate calibration.
In this paper, we concentrate exclusively on designs for linear
I constructed in this way. The maximum number of factors that
calibration where the model is of the form yˆ = Â
i =1
ai xi .
Interactions and quadratic terms would make the analysis much
more complex.
Theory
A key concept used in much of the experimental design
literature is that of orthogonality. Two factors are said to be
orthogonal if they have a correlation coefficient of zero. This is
equivalent to stating that the factors span each otherAs mixture
space evenly. In order to have predictions that depend only on
the distance from the centre of the design, and to have circular
and symmetric leverage plots, it is necessary that any combina-
tion of factors is orthogonal. Below the emphasis will be on
developing orthogonal designs for linear models.
Two-level Designs
The two-level, three-factor fractional factorial design, as given
in Table 1, is one of the simplest possible multifactor designs.
Each column represents a factor and each row an experiment,
and is often identified as a 23–1 factorial design. This design also
relates to Plackett–Burman designs (strictly, such designs are
used when there are a large number of factors, but many of the
simplest designs have been widely used as a basis for more
sophisticated cases) and has various properties. The first is that
each level occurs the same number of times in each column, so
that +1 and 21 occur twice. The second is that each column is
orthogonal or uncorrelated. This is an essential property to
ensure that the experimental space is spanned adequately. If two
factors are not orthogonal the problem of asymmetric predic-
tions will occur, as described above.
There are several ways of setting up the design of Table 1. A
simple rule is as follows.
Table 1 Two-level, three-factor fractional factorial design
View Article Online
(a) Pick one level (+1 or 21). This is the ‘odd level out’. The
first row simply consists of this level repeated.
(b) Rows 2 to 4 consist of a ‘cyclic permuter’. Each column
is shifted cyclically and vertically against the previous column,
so that row 4 of column i equals row 2 of column i 2 1. This
permuter is given by rows 2 to 4 of the first column, and in the
case of Table 1 is (21 1 1). Hence row 2, column 1 equals 21,
as does row 4, column 2. These permutations are illustrated in
Table 1.
(c) The nature of the permuter is crucial to the design. It must
contain two 1s and one 21 in this case, to ensure that each
column is balanced, having an even number of high and low
levels.
For this design, the construction is very straightforward.
Plackett and Burman extended these designs to cases where the
number of factors is much larger. It can be shown that, to obtain
orthogonal two-level designs, N = k 3 22 experiments must be
performed,17 i.e., 4, 8, 12 , etc., where k is any integer. Other
related saturated fractional factorial designs can also be
can be studied is no more than N 2 1. Hence, to study 11 factors
at two levels, 12 experiments must be performed.
A number of two-level cyclic generators can be produced for
more than three factors, with interesting properties. The
mathematical derivation of these generators is outside the scope
of this paper, except to point out that only very specific
sequences are allowed. The aim of this paper is to extend the
reasoning to multilevel designs.
Possible Orthogonal Designs
It can be shown that, in order to produce an orthogonal design,19
the number of experiments must be a multiple of a power of the
number of levels l, so that N = k lr, where r is at least 2. Setting
k equal to 1 and r to the minimum value, this implies that N =
l2. Hence for a five-level design, 25 experiments should be
performed.
In this paper, we will consider only experiments where the
number of experiments is equal to the square of the number of
levels. The maximum number of factors permitted is N 2 1.
Hence for five levels, 24 factors are permitted. More elaborate
generators can be proposed for more factors. Hence a 40-factor,
five-level design, requires 50 rather than 25 experiments. For
many typical multivariate calibration experiments, five levels is
sufficient, as there will rarely be more than 24 compounds of
interest.
It can also be shown that these designs are based on Latin
squares,20,21 and mutually orthogonal Latin squares exist only if
the number of levels is a prime number or a power of a prime
number, meaning that, whereas such designs exist for two,
three, four and five levels, they do not exist for six levels, an
important point to note when performing laboratory experi-
ments. Another theorem is that a maximum of (N 2 1)/(l 2 1)
mutually orthogonal factors are possible, i.e., six for five levels,
five for four levels, and so on. This relates to the nature of the
cyclical permuter. Although this is true for two- to four-level
designs, special circumstances allow up to 12 factors to be
mutually orthogonal for five-level designs, as is shown below.
However, for a four concentration level, 16 experiment, design,
it is possible only to have five compounds that are mutually
orthogonal, and so span each other’s mixture space completely;
for five levels and 25 experiments, up to 12 mutually orthogonal
factors are possible—this will be shown later.
An important feature of all the designs described in this paper
is the relationship between successive columns or factors. The
first experiment is always performed at the same level for all
factors. For example, if there are three factors, and five levels,
then the first experiment will be performed at the same
(generally the middle level) for each of the factors. The

remaining experiments are related by the cyclical permuter. If
there are N experiments, then experiment 2 of factor 2 will be at
the same level as experiment 3 of factor 1; experiment 3 of
factor 2 will be at the same level as experiment 4 of factor 1;
finally, experiment N of factor 2 will be at the same level as
experiment 2 of factor 1. Note that only experiment 1 is left out
of these permutations. This can be extended so that experiment
2 of factor 3 is at the same level as experiment 4 of factor 1, for
example. Hence rows 2 to N 2 1 can be generated from
knowing column 1. The design, therefore, is uniquely charac-
terised by the first column.
Development of a Five-level Design
Several steps are required. A generalised treatment is given
below, to allow the reader to construct further designs beyond
Published on 01 January 1997. Downloaded on 13/07/2014 10:06:21.
those presented in this paper. The five levels will be indicated
by 22, 21, 0, 1 and 2, respectively, representing, for example,
five equally spaced concentrations, the central one coded by the
number 0. Twenty-five experiments are necessary and up to 24
possible factors can be included in the design.
The first step is to pick an ‘odd level out’, which will be
referred to as the ‘repeater’ level below. In the original paper by
Plackett and Burman,17 this is the lowest level (22). As we
show below, it is possible to have up to six orthogonal factors
using any such repeater level, but it is essential that this level is
0 (the central level) for more than six successive mutually
orthogonal factors. This is the level at which experiment 1 is
performed for all of the factors.
The next step is to produce a cyclic generator for the
remaining four levels. A possible generator is of the form 22 ?
21 ? 2 ? 1 ? 22, if the repeater level is 0, as is illustrated
in Fig. 3. The forward cyclical difference between levels can be
calculated easily. For example, the cyclical difference between
levels 22 and 21 is 1 whereas the cyclical difference between
21 and 22 is 3 and between 1 and 2 is likewise 3. Note that
clockwise differences are calculated, and must be in one
direction only. The significance of this cyclic generator will be
discussed below.
The first two steps can be generalised by denoting the
repeater by x and the other four levels a–d, where the cyclic
generator converts a to b, b to c, c to d and d to a. In the case
illustrated, x = 0, a = 22, b = 21, and so on.
The key is to generate the first column of the design matrix,
after which all other columns can be generated. The 25
experiments in the first column can be divided into four sets of
five experiments and five unique experiments as illustrated in
Fig. 4. The first experiment is a unique experiment at level x, as
is the second, then comes a block of five experiments, a unique
experiment, and so on. Rows 1 and 2 + j(l + 1) where j = 0–3
(i.e., 2, 8, 14, and 20) are at level x, the repeater. The remaining
four blocks of five experiments (shaded vertically) are at levels
a–d.
Fig. 3 Possible cyclic generator for a five-level design.
Fig. 4 Arrangement of experiments in the first column of a five-level
design.
View Article Online
Analyst, December 1997, Vol. 122 1523
The four (l 2 1) blocks of five ( = l) experiments are
determined by the first block of five experiments as follows.
The levels in each block are cyclically shifted from the previous
block by one, using the cyclic generator introduced above.
Hence, if the first block (experiments 3–7) is at the levels acdcb,
the second block will be at levels bdada, since b is shifted by
one from a, d by one from c, a by one from d, and so on. Hence,
for the generator 22 ? 21 ? 2 ? 1 ? 22, if experiment 3 is
at level 22, experiment 9 will be at level 21, experiment 15 at
level 2 and experiment 21 at level 1. This property ensures that
each level is represented five times over the 25 experiments, so
that each individual compound is measured at each of the five
concentration levels five times, an important prerequisite for a
balanced mixture design. Knowing x, the cyclical generator, and
the levels of experiments 3–7, it is possible to determine the
overall design.
The next, and most difficult, aspect is to establish which
combinations of levels are possible for the first block of five
experiments in column 1, and here it is essential to understand
the motivations of calibration designs. An ideal design will
match experiments at each level for one factor with experiments
at all other levels for another factor. For example, the five
experiments at level 1 for the first factor should correspond to
one experiment each at levels 22, 21, 0, 1 and 2 for the second
factor, to give a design as illustrated in Fig. 5. Because column
1 is shifted vertically against column 2, and so on, vertical
differences in column 1 become horizontal differences between
columns 1 and 2. Hence if, for example, experiment 7 in column
1 is at level 21 and experiment 8 at level +1, this also implies
that experiment 7 is at level 21 for factor 1 and +1 for factor
2.
Using a generator, it is possible to determine how successive
levels are related vertically in column 1; as explained above, this
then translates into a horizontal relationship between successive
columns or factors or compound concentrations. For example, if
the first sequence of five levels (experiments 3–7) is given by
aacbc, then the cyclical difference between the first two levels
is 0. The same difference will be repeated in the second block
(experiments 9–13), which will become bbdcd, and so on. The
design can be understood by considering forward cyclical
differences between successive members of each block. In the
case cited these can be denoted by a difference vector [0 2 3 1]:
the difference between a and a is 0, between a and c is 2,
between c and b is 3 and between c and d is 1. This means that
factor 2 will be shifted by these corresponding amounts against
factor 1, so that for experiments 3, 9, 15 and 21 factors 1 and 2
are at the same levels, since the first value of the difference
vector is 0. It should be evident that only cyclic difference
vectors containing all four numbers 0 to 3 (or 0 to l 2 2, where
l is the number of levels) will generate designs of the form of
Fig. 5, since each level must be matched with all other possible
levels only once. Note that level x will take care of itself, owing
to the nature of the generator. Hence each possible cyclic
difference vector generates a design. For example, a cyclic
difference vector of [1 3 0 2] generates a design of the form
Fig. 5 An ideal design, where each axis represents the levels of two
different factors.

1524 Analyst, December 1997, Vol. 122
abaac. There are 24 [ = (l 2 1)!] possible cyclic difference
vectors.
The next step is to determine which of these vectors are
acceptable. In order to do this, it is important to consider not
only one factor forward cyclical differences but two and more
factor differences; these again can be computed simply by
examining differences between successive experiments for a
single factor. For example, if experiments 7 and 9 are performed
at levels +1 and 0 for factor 17, then experiment 7 will be
performed at levels +1 for factors 17 and level 0 for factor 19.
A forward difference matrix can be computed, as follows. If the
generator is [0 2 3 1], then the cyclical forward difference
between the first and third experiment is 0 + 2 = 2, between the
second and fourth is 2 + 3 = 5, which because of the cyclicity
becomes 1, and between the third and fifth is 3 + 1 = 4,
becoming 0 because of cyclicity. Hence the first three
Published on 01 January 1997. Downloaded on 13/07/2014 10:06:21.
differences are given by (gi + gi + 1 mod l 2 1), where gi is the
ith term of the generator, i.e., g1 = 0, g2 = 2 and so on. The final
difference is more complex to determine. The first and fifth
members of each block will always have a difference of 2,
because (0 + 1 + 2 + 3 mod 4) = 2. Hence if the first member
of a block is at level b the last will be at level d, for example. The
final difference will, however, be increased by a further 1,
because it is between the last member of the first block and the
first member of the second block, which is cyclically shifted by
a further 1, owing to the nature of the generator. This is easy to
understand. If the first member of a block is at level b, the first
member of the next block must be at level c; the forward
cyclical difference between levels d (the last member of the first
block—see above) and c is 3.
Similar arguments apply to three, four and five experiment
shifts. A difference matrix can be obtained and is given in Table
2. The diagonals relate to the differences with the repeater level.
The columns relate to differences between experiments sepa-
rated by j rows; so column 4 represents the differences between
experiments 3 and 7, 4 and 8, and so on. The rows simply
represent the five experiments in each block. Hence row 3 and
column 5 represent the forward cyclic difference between
experiments 5 (row 3 corresponds to experiment 5 since row 1
corresponds to experiment 3 or the first experiment in the first
block) and 10 (five experiments later) and equals 3. For the
sequence of levels aacbc, experiment 5 is level c (since a
corresponds to experiment 3), so experiment 10 will be level b.
This also implies that if experiment 5 of factor 1 is at level c,
experiment 5 of factor 6 will likewise be at level b. For the
cyclic generator 22 ? 21 ? 2 ? 1 ? 22, if experiment 5 for
factor 1 is at level 2 ( = c), then experiment 5 for factor 6 is at
the same level as experiment 10 of factor 1 and is at level 21 ( =
b).
Extending this reasoning, it is evident that only designs for
which the five columns of the difference matrix contain the
numbers 0, 1, 2 and 3 will produce designs which have the
property that the first six ( = l + 1) columns are mutually
orthogonal and result in all possible combinations of factor
levels as illustrated in Fig. 5. Table 3 illustrates a generator that
does not result in an orthogonal design. For a five-level design,
only the difference vectors namely [0 2 3 1], [1 3 2 0], [2 0 1 3]
and [3 1 0 2] have the appropriate properties as can be verified
by computing the relevant difference matrices.
Table 2 Difference matrix for a five-level design and the cyclical difference
vector [0 2 3 1]
0 2 1 2 —
2 1 2 — 1
3 0 — 3 3
1 — 0 0 2 Fig. 6 Graph of levels of factor 1 versus levels of factor 7 for a five-level
— 3 3 1 0 correlated design with a cyclic generator of the form 22 ? 21 ? 2 ? 1
View Article Online
The final consideration relates to the nature of the cyclic
generator. For up to and including six factors, any cyclic
generator, including any repeater, can be employed. However, it
is desirable that factors 1 and 7, factors 2 and 8, factors 3 and 9,
and so on, are also uncorrelated. Whether this is possible
depends on the nature of the repeater and cyclic generator.
Every seven factors the repeater level (recommended to be 0)
translates onto itself, and the other levels are shifted cyclically
by 1, because experiments 9–13 of column 1 contain the same
sequence of levels as experiments 3–7, simply shifted cyclically
by one level. Hence if the cyclic generator is 22 ? 21 ? 2 ?
1 ? 22, and experiment 3, factor 1, is at level 22, then
experiment 3 for factor 7 will be at level 21. A graph of levels
of factor 1 against levels of factor 7 will consist of the points
(22,21), (21,2), (2,1) and (1,22) in addition to the centre
point (0,0) and is given in Fig. 6. In all cases a level of 22 in
column (factor 1) will correspond to a level of 21 in factor 7,
hence each point in Fig. 6 represents five experiments. These
experiments, however, are still completely uncorrelated. There
are only two possible uncorrelated cyclical generators, namely
the one described above and the variant 22 ? 1 ? 2 ? 21 ?
22. Note that all four levels must be included in the generator
for a five-level design, so a graph of factor 1 versus factor 2
containing the points (22,22), (22,2), (2,22) and (2,2),
although uncorrelated, is an impossible solution as levels 1 and
21 must be used, otherwise there will be only three levels (22,
0 and 2) in the design.
The equation of leverage for factors 1 versus 2, assuming all
other factors are at level 0, is given by h = 0.02x12 + 0.02x22,
provided that the designs have been generated as above, using
the correct difference vector. Identical equations for leverage
for factors 1 versus 7 are also obtained if one of the two cyclic
generators above is employed. Such equations are symmetrical
in the two factors and hence result in circular and symmetric
confidence limits. Hence the entire mixture space is spanned
evenly.
A cyclic generator of the form 22 ? 1 ? 21 ? 2 ? 22
does not have this property. Although leverage for factors 1 and
2 is the same as for the other cyclic generator, for factors 1 and
7 this becomes h = 0.1053x12 + 0.1053x22 + 0.1895x1x2 as is
illustrated in Fig. 7. This design clearly behaves in an
unsatisfactory manner for seven factors, but works perfectly
acceptably when there are only six factors.
The importance of a correct cyclical generator for more than
six factors cannot be overestimated. Consider, for example, a
Table 3 Difference matrix for a five-level design and the cyclical difference
vector [0 3 2 1]
0 3 1 2 –
3 1 2 – 1
2 3 – 2 2
1 — 0 0 3
— 3 3 2 0
? 22.

mixture experiment where factors 1–9 are methanol, acetone,
benzene, toluene, tetrahydrofuran, ethanol, hexane, cyclohex-
ane and diethyl ether, respectively. Since factors 1 and 7, 2 and
8, and 3 and 9 are partially correlated, it will happen that at
certain combinations of concentrations of methanol and hexane,
acetone and cyclohexane, and benzene and diethyl ether, the
model predicts poorly, whereas for all other possible combina-
tions, the quality of predictions is simply related to distance
from the centre of the design. The practical implications are
strong, especially in situations such as process control where it
may be relied upon to produce a good predictive model over the
entire experimental domain. Unexpectedly poor predictions are
often reported as ‘poor performance of algorithms,’ yet they
depend in fact in part on the design of the calibration
experiments.
Unfortunately, after 12 factors, the two feasible cyclical
Published on 01 January 1997. Downloaded on 13/07/2014 10:06:21.
generators become correlated, as two steps of the cycle result in
22 ? ? 2 ? ? 22 and 21 ? ? 1 ? ? 1 for the first of the
two possible generators, producing the points (0,0) (the
repeater), (22,2), (2,22), (21,1) and (1,21), with a correlation
coefficient of 21, so it is impossible to distinguish the influence
of increasing the concentration of factor j from that due to a
decrease in the concentration of factor j + 12. There is no
obvious solution to this problem, apart from doubling the
number of experiments to 50, which is likely to be impracticable
in most situations. A possible answer is to change the nature of
the cyclic generator so that factors n and n + 6 are correlated, but
with a correlation coefficient of < 1. A correlation coefficient
> 0 but < 1 will then also exist between factors j and j + 12.
Summary of Construction of a Five-level Design
The considerations above provide information on how to set up
a five-level design, with 25 experiments and up to 24 factors (or
compounds):
(i) Select a repeater level, recommended to be 0. For between
7 and 12 factors it is essential that this is 0.
(ii) Select a cyclical generator for the remaining levels. Only
two possible cyclical generators, namely 22 ? 21 ? 2 ? 1 ?
22 and 22 ? 1 ? 2 ? 21 ? 22 have the property that
factors j and j + l + 1 are uncorrelated. For less than seven
factors, the order of this generator is not relevant. Note that for
more than 12 factors, other considerations may need to be taken
into account.
(iii) Select a difference vector. Only the four vectors
[0 2 3 1], [1 3 2 0], [2 0 1 3] and [3 1 0 2] guarantee that six
successive factors are mutually orthogonal and also each level
of each factor has a corresponding experiment with each level of
every other factor, for six successive factors.
(iv) Then generate the design. The entire first experiment is at
the repeater level for each factor. Experiments 2, 8, 14 and 20
are at the repeater level for factor 1. The blocks corresponding
to experiments 3–7, 9–13, 15–19 and 21–25 for factor 1 are
related by the cyclical generator and within each block related
Fig. 7 (a) Graph of levels of factors 1 and 7 for a five-level design using
the cyclic generator 22 ? 1 ? 21 ? 2 ? 22. (b) Graph of leverage
corresponding to (a), contour levels 0.4, 0.8 and 1.2.
View Article Online
Analyst, December 1997, Vol. 122 1525
by the difference vector. Factor 2 (column 2) is obtained by
shifting experiments (rows) 2 to 25 by 1, so that row 2 of factor
2 is equivalent to row 3 of factor 1. The 25th experiment of
factor 2 corresponds to the second of factor 1. Continue shifting
to generate 24 possible factors. The 25th factor will be the same
as the first, so different designs are required if more than l2 2 1
factors are needed. Hence the designs in this paper involve a
maximum of 24 possible compounds at five concentration
levels.
Note that it does not matter whether 22, 21, 1 or 2 is the
level of the third experiment for factor 1. This is because each
of the four blocks is related cyclically to each other. It is
possible to change the order of each of the blocks, and there is
no physical difference if experiments 3–7 are exchanged with
experiments 9–13. For simplicity, however, it is recommended
that experiment 3 is set at the lowest remaining level for factor
1, meaning that the first three experiments will normally be at
levels 0, 0 and 22 for this factor.
Corresponding Designs for Two to Four Levels
Two levels
It is easy to see how the design of Table 1 can be generated using
the reasoning above. Only two levels, 21 and +1, are possible.
Selecting the repeater as level 21 generates the first row.
Experiment 2, factor 1, is also set at the level of the repeater.
There is only one block which consists of l + 1 ( = 2)
experiments. The only remaining level is +1, and the only
possible difference vector is [0], hence the first column, which
is simply shifted against the other columns.
It is possible only to study only 22 2 1 = 3 factors by this
means, all of which are mutually orthogonal and span each
other’s mixture space, as can readily be verified.
Three levels
In this case, if the levels are denoted 21, 0 and +1, the most
appropriate repeater is 0. There will be nine experiments and
hence up to eight possible factors. In the first column, rows 1, 2
and 6 will consist of the repeater (0). There is only one possible
cyclic generator, namely 21 ? 1 ? 21, which is identical
with 1 ? 21 ? 1. There are two blocks, each consisting of
three experiments.
There are two possible difference vectors, namely [0 1] and
[1 0]. Both vectors generate acceptable mutually orthogonal
design matrices only for four factors.
Unfortunately, it is not possible to obtain an uncorrelated
cyclic generator. Hence factors 1 and 5, 2 and 6, 3 and 7, and 4
and 8 will be completely correlated. This implies, that using a
three-level design, it is impossible in nine experiments to
distinguish the influence of these pairs of factors or compounds
on a system. The solution is to perform more experiments (e.g.,
18) at three levels, or to increase the number of levels. Note that
for four levels only 16 experiments are necessary.
Four levels
In this case, there are 16 experiments and a maximum of 15
factors are allowed. For the first factor, the first experiment will
consist of the repeater, as will experiments 2, 7 and 12. The
remaining experiments will be divided into blocks of four.
The levels will be denoted by 22, 21, +1 and +2 (note that
it is normally recommended that the middle levels are evenly
spaced, although provided that the spacing is symmetric above
and below the centre of the design, circular leverage and
confidence plots will be obtained). One level must be picked as
a repeater. In this paper, we pick level 22, but any other
possibility can be proposed.
The difference vectors [0 2 1] and [1 2 0] are both acceptable
for five successive factors to be mutually orthogonal. There are
 View Article Online

1526 Analyst, December 1997, Vol. 122

3! = 6 possible difference vectors, and the remaining four are
not acceptable.
There is no cyclic generator for the remaining three levels
after the repeater has been removed that will lead to factors 1
and 6, 2 and 7, etc. being orthogonal, and either of the possible
generators 21 ? 1 ? 2 ? 21 or 21 ? 2 ? 1 ? 21 is equally
as good. However, factors j and j + 5 do not have a correlation
coefficient of !1, so some information to distinguish these
factors will be available. Nevertheless, if it is desired to study
more than five factors, it is probably preferable to change to a
five-level design, where up to 24 factors can be studied.
properties, the designs generated simply by replacing relevant
numbers in the table, but maintaining the difference vector. One
interesting feature of the four-level designs is that the non-
orthogonal columns are not completely correlated, unlike in the
case of the three-level designs, the correlation coefficient
between such columns being < 1. This implies that it is possible
to distinguish partially the effect of all compounds from each
other, even if more than five components are studied in a
mixture, and represents a substantial improvement from a three-
level design. However, the user should take the confounding

Results and Discussion Table 4 Three-level calibration designs

This paper contains some design tables for three- to five-level (a) Difference vector [1 0]
calibration experiments. 0 0 0 0 0 0 0 0
Published on 01 January 1997. Downloaded on 13/07/2014 10:06:21.

Table 4 presents the two possible three-level calibration 21 1 1 0 1 21 21 0

designs. Note that any four successive columns are orthogonal 1 1 0 1 21 21 0 2
1 0 1 21 21 0 21 1
and completely span the calibration space, whereas columns 1
0 1 21 21 0 21 1 1
and 5, 2 and 6, etc., are correlated with correlation coefficients 1 21 21 0 21 1 1 0
of 21. It is, therefore, impossible to distinguish the effects of 21 21 0 21 1 1 0 1
these factors from each other using nine experiments. Extension 21 0 21 1 1 0 1 21
to 18 experiments (not reported here) would seem excessive, 0 21 1 1 0 1 21 21
and it is preferable to increase the number of levels. It is possible (b) Difference vector [0 1]
to have any numbers as the repeater or within the blocks, the 0 0 0 0 0 0 0 0
only constraint being the sequence as dictated by the difference 21 21 1 0 1 1 21 0
vector, so further designs can be generating simply by replacing 21 1 0 1 1 21 0 21
each unique level in the table by a different unique level. 1 0 1 1 21 0 21 21
Four-level designs are given in Table 5 for the two possible 0 1 1 21 0 21 21 1
difference vectors. Note, again, that columns 1 and 6, 2 and 7 1 1 21 0 21 21 1 0
1 21 0 21 21 1 0 1
etc. are no longer uncorrelated. In Table 4, the repeater is set at
21 0 21 21 1 0 1 1
22 and the cyclic generator at 21 ? 1 ? 2 ? 21. However, 0 21 21 1 0 1 1 21
all and every other possibility can be proposed without loss of

Table 5 Four-level calibration designs

(a) Difference vector [0 2 1]

22 22 22 22 22 22 22 22 22 22 22 22 22 22 22
22 21 21 2 21 22 1 1 21 1 22 2 2 1 2
21 21 2 21 22 1 1 21 1 22 2 2 1 2 22
21 2 21 22 1 1 21 1 22 2 2 1 2 22 21
2 21 22 1 1 21 1 22 2 2 1 2 22 21 21
21 22 1 1 21 1 22 2 2 1 2 22 21 21 2
22 1 1 21 1 22 2 2 1 2 22 21 21 2 21
1 1 21 1 22 2 2 1 2 22 21 21 2 21 22
1 21 1 22 2 2 1 2 22 2 21 2 21 22 1
21 1 22 2 2 1 2 22 21 21 2 21 22 1 1
1 22 2 2 1 2 22 21 21 2 21 22 1 1 21
22 2 2 1 2 22 21 21 2 21 22 1 1 21 1
2 2 1 2 22 21 21 2 21 22 1 1 21 1 22
2 1 2 22 21 21 2 21 22 1 1 21 1 22 2
1 2 22 21 21 2 21 22 1 1 21 1 22 2 2
2 22 21 21 2 21 22 1 1 21 1 22 2 2 1
(b) Difference vector [1 2 0]
22 22 22 22 22 22 22 22 22 22 22 22 22 22 22
22 21 1 21 21 22 1 2 1 1 22 2 21 2 2
21 1 21 21 22 1 2 1 1 22 2 21 2 2 22
1 21 21 22 1 2 1 1 22 2 21 2 2 22 21
21 21 22 1 2 1 1 22 2 21 2 2 22 21 1
21 22 1 2 1 1 22 2 21 2 2 22 21 1 21
22 1 2 1 1 22 2 21 2 2 22 21 1 21 21
1 2 1 1 22 2 21 2 2 22 21 1 21 21 22
2 1 1 22 2 21 2 2 22 21 1 21 21 22 1
1 1 22 2 21 2 2 22 21 1 21 21 22 1 2
1 22 2 21 2 2 22 21 1 21 21 22 1 2 1
22 2 21 2 2 22 21 1 21 21 22 1 2 1 1
2 21 2 2 22 21 1 21 21 22 1 2 1 1 22
21 2 2 22 21 1 21 21 22 1 2 1 1 22 2
2 2 22 21 1 21 21 22 1 2 1 1 22 2 21
2 22 21 1 21 21 22 1 2 1 1 22 2 21 2
 View Article Online

Analyst, December 1997, Vol. 122 1527

Table 6 Five-level calibration designs, using cyclic generator 22 ? 1 ? 2 ? 1 ? 22 and repeater of 0

(a) Difference vector [0 2 3 1]

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 22 22 2 21 2 0 21 21 1 2 1 0 2 2 22 1 22 0 1 1 21 22 21
22 22 2 21 2 0 21 21 1 2 1 0 2 2 22 1 22 0 1 1 21 22 21 0
22 2 21 2 0 21 21 1 2 1 0 2 2 22 1 22 0 1 1 21 22 21 0 22
2 21 2 0 21 21 1 2 1 0 2 2 22 1 22 0 1 1 21 22 21 0 22 22
21 2 0 21 21 1 2 1 0 2 2 22 1 22 0 1 1 21 22 21 0 22 22 2
2 0 21 21 1 2 1 0 2 2 22 1 22 0 1 1 21 22 21 0 22 22 2 21
0 21 21 1 2 1 0 2 2 22 1 22 0 1 1 21 22 21 0 22 22 2 21 2
21 21 1 2 1 0 2 2 22 1 22 0 1 1 21 22 21 0 22 22 2 21 2 0
21 1 2 1 0 2 2 22 1 22 0 1 1 21 22 21 0 22 22 2 21 2 0 21
1 2 1 0 2 2 22 1 22 0 1 1 21 22 21 0 22 22 2 21 2 0 21 21
2 1 0 2 2 22 1 22 0 1 1 21 22 21 0 22 22 2 21 2 0 21 21 1
1 0 2 2 22 1 22 0 1 1 21 22 21 0 22 22 2 21 2 0 21 21 1 2
0 2 2 22 1 22 0 1 1 21 22 21 0 22 22 2 21 2 0 21 21 1 2 1
2 2 22 1 22 0 1 1 21 22 21 0 22 22 2 21 2 0 21 21 1 2 1 0
Published on 01 January 1997. Downloaded on 13/07/2014 10:06:21.

2 22 1 22 0 1 1 21 22 21 0 22 22 2 21 2 0 21 21 1 2 1 0 2
22 1 22 0 1 1 21 22 21 0 22 22 2 21 2 0 21 21 1 2 1 0 2 2
1 22 0 1 1 21 22 21 0 22 22 2 21 2 0 21 21 1 2 1 0 2 2 22
22 0 1 1 21 22 21 0 22 22 2 21 2 0 21 21 1 2 1 0 2 2 22 1
0 1 1 21 22 21 0 22 22 2 21 2 0 21 21 1 2 1 0 2 2 22 1 22
1 1 21 22 21 0 22 22 2 21 2 0 21 21 1 2 1 0 2 2 22 1 22 0
1 21 22 21 0 22 22 2 21 2 0 21 21 1 2 1 0 2 2 22 1 22 0 1
21 22 21 0 22 22 2 21 2 0 21 21 1 2 1 0 2 2 22 1 22 0 1 1
22 21 0 22 22 2 21 2 0 21 21 1 2 1 0 2 2 22 1 22 0 1 1 21
21 0 22 22 2 21 2 0 21 21 1 2 1 0 2 2 22 1 22 0 1 1 21 22
(b) Difference vector [1 3 2 0]
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 22 21 22 2 2 0 21 2 21 1 1 0 2 1 2 22 22 0 1 22 1 21 21
22 21 22 2 2 0 21 2 21 1 1 0 2 1 2 22 22 0 1 22 1 21 21 0
21 22 2 2 0 21 2 21 1 1 0 2 1 2 22 22 0 1 22 1 21 21 0 22
22 2 2 0 21 2 21 1 1 0 2 1 2 22 22 0 1 22 1 21 21 20 22 21
2 2 0 21 2 21 1 1 0 2 1 2 22 22 0 1 22 1 21 21 0 22 21 22
2 0 21 2 21 1 1 0 2 1 2 22 22 0 1 22 1 21 21 0 22 21 22 2
0 21 2 21 1 1 0 2 1 2 22 22 0 1 22 1 21 21 0 22 21 22 2 2
21 2 21 1 1 0 2 1 2 22 22 0 1 22 1 21 21 0 22 21 22 2 2 0
2 21 1 1 0 2 1 2 22 22 0 1 22 1 21 21 0 22 21 22 2 2 0 21
21 1 1 0 2 1 2 22 22 0 1 22 1 21 21 0 22 21 22 2 2 0 21 2
1 1 0 2 1 2 22 22 0 1 22 1 21 21 0 22 21 22 2 2 0 21 2 21
1 0 2 1 2 22 22 0 1 22 1 21 21 0 22 21 22 2 2 0 21 2 21 1
0 2 1 2 22 22 0 1 22 1 21 21 0 22 21 22 2 2 0 21 2 21 1 1
2 1 2 22 22 0 1 22 1 21 21 0 22 21 22 2 2 0 21 2 21 1 1 0
1 2 22 22 0 1 22 1 21 21 0 22 21 22 2 2 0 21 2 21 1 1 0 2
2 22 22 0 1 22 1 21 21 0 22 21 22 2 2 0 21 2 21 1 1 0 2 1
22 22 0 1 22 1 21 21 0 22 21 22 2 2 0 21 2 21 1 1 0 2 1 2
22 0 1 22 1 21 21 0 22 21 22 2 2 0 21 2 21 1 1 0 2 1 2 22
0 1 22 1 21 21 0 22 21 22 2 2 0 21 2 21 1 1 0 2 1 2 22 22
1 22 1 21 21 0 22 21 22 2 2 0 21 2 21 1 1 0 2 1 2 22 22 0
22 1 21 21 0 22 21 22 2 2 0 21 2 21 1 1 0 2 1 2 22 22 0 1
1 21 21 0 22 21 22 2 2 0 21 2 21 1 1 0 2 1 2 22 22 0 1 22
21 21 0 22 21 22 2 2 0 21 2 21 1 1 0 2 1 2 22 22 0 1 22 1
21 0 22 21 22 2 2 0 21 2 21 1 1 0 2 1 2 22 22 0 1 22 1 21
(c) Difference vector [2 0 1 3]
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 22 2 2 1 2 0 21 1 1 22 1 0 2 22 22 21 22 0 1 21 21 2 21
22 2 2 1 2 0 21 1 1 22 1 0 2 22 22 21 22 0 1 21 21 2 21 0
2 2 1 2 0 21 1 1 22 1 0 2 22 22 21 22 0 1 21 21 2 21 0 22
2 1 2 0 21 1 1 22 1 0 2 22 22 21 22 0 1 21 21 2 21 0 22 2
1 2 0 21 1 1 22 1 0 2 22 22 21 22 0 1 21 21 2 21 0 22 2 2
2 0 21 1 1 22 1 0 2 22 22 21 22 0 1 21 21 2 21 0 22 2 2 1
0 21 1 1 22 1 0 2 22 22 21 22 0 1 21 21 2 21 0 22 2 2 1 2
21 1 1 22 1 0 2 22 22 21 22 0 1 21 21 2 21 0 22 2 2 1 2 0
1 1 22 1 0 2 22 22 21 22 0 1 21 21 2 21 0 22 2 2 1 2 0 21
1 22 1 0 2 22 22 21 22 0 1 21 21 2 21 0 22 2 2 1 2 0 21 1
22 1 0 2 22 22 21 22 0 1 21 21 2 21 0 22 2 2 1 2 0 21 1 1
1 0 2 22 22 21 22 0 1 21 21 2 21 0 22 2 2 1 2 0 21 1 1 22
0 2 22 22 21 22 0 1 21 21 2 21 0 22 2 2 1 2 0 21 1 1 22 1
2 22 22 21 22 0 1 21 21 2 21 0 22 2 2 1 2 0 21 1 1 22 1 0
22 22 21 22 0 1 21 21 2 21 0 22 2 2 1 2 0 21 1 1 22 1 0 2
22 21 22 0 1 21 21 2 21 0 22 2 2 1 2 0 21 1 1 22 1 0 2 22
21 22 0 1 21 21 2 21 0 22 2 2 1 2 0 21 1 1 22 1 0 2 22 22
22 0 1 21 21 2 21 0 22 2 2 1 2 0 21 1 1 22 1 0 2 22 22 21
continued over—
 View Article Online

1528 Analyst, December 1997, Vol. 122

Table 6—continued

0 1 21 21 2 21 0 22 2 2 1 2 0 21 1 1 22 1 0 2 22 22 21 22
1 21 21 2 21 0 22 2 2 1 2 0 21 1 1 22 1 0 2 22 22 21 22 0
21 21 2 21 0 22 2 2 1 2 0 21 1 1 22 1 0 2 22 22 21 22 0 1
21 2 21 0 22 2 2 1 2 0 21 1 1 22 1 0 2 22 22 21 22 0 1 21
2 21 0 22 2 2 1 2 0 21 1 1 22 1 0 2 22 22 21 22 0 1 21 21
21 0 22 2 2 1 2 0 21 1 1 22 1 0 2 22 22 21 22 0 1 21 21 2
(d) Difference vector [3 1 0 2]
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 22 1 22 22 2 0 21 22 21 21 1 0 2 21 2 2 22 0 1 2 1 1 21
22 1 22 22 2 0 21 22 21 21 1 0 2 21 2 2 22 0 1 2 1 1 21 0
1 22 22 2 0 21 22 21 21 1 0 2 21 2 2 22 0 1 2 1 1 21 0 22
22 22 2 0 21 22 21 21 1 0 2 21 2 2 22 0 1 2 1 1 21 0 22 1
22 2 0 21 22 21 21 1 0 2 21 2 2 22 0 1 2 1 1 21 0 22 1 22
2 0 21 22 21 21 1 0 2 21 2 2 22 0 1 2 1 1 21 0 22 1 22 22
0 21 22 21 21 1 0 2 21 2 2 22 0 1 2 1 1 21 0 22 1 22 22 2
21 22 21 21 1 0 2 21 2 2 22 0 1 2 1 1 21 0 22 1 22 22 2 0
Published on 01 January 1997. Downloaded on 13/07/2014 10:06:21.

22 21 21 1 0 2 21 2 2 22 0 1 2 1 1 21 0 22 1 22 22 2 0 21
21 21 1 0 2 21 2 2 22 0 1 2 1 1 21 0 22 1 22 22 2 0 21 22
21 1 0 2 21 2 2 22 0 1 2 1 1 21 0 22 1 22 22 2 0 21 22 21
1 0 2 21 2 2 22 0 1 2 1 1 21 0 22 1 22 22 2 0 21 22 21 21
0 2 21 2 2 22 0 1 2 1 1 21 0 22 1 22 22 2 0 21 22 21 21 1
2 21 2 2 22 0 1 2 1 1 21 0 22 1 22 22 2 0 21 22 21 21 1 0
21 2 2 22 0 1 2 1 1 21 0 22 1 22 22 2 0 21 22 21 21 1 0 2
2 2 22 0 1 2 1 1 21 0 22 1 22 22 2 0 21 22 21 21 1 0 2 21
2 22 0 1 2 1 1 21 0 22 1 22 22 2 0 21 22 21 21 1 0 2 21 2
22 0 1 2 1 1 21 0 22 1 22 22 2 0 21 22 21 21 1 0 2 21 2 2
0 1 2 1 1 21 0 22 1 22 22 2 0 21 22 21 21 1 0 2 21 2 2 22
1 2 1 1 21 0 22 1 22 22 2 0 21 22 21 21 1 0 2 21 2 2 22 0
2 1 1 21 0 22 1 22 22 2 0 21 22 21 21 1 0 2 21 2 2 22 0 1
1 1 21 0 22 1 22 22 2 0 21 22 21 21 1 0 2 21 2 2 22 0 1 2
1 21 0 22 1 22 22 2 0 21 22 21 21 1 0 2 21 2 2 22 0 1 2 1
21 0 22 1 22 22 2 0 21 22 21 21 1 0 2 21 2 2 22 0 1 2 1 1

scheme into consideration when interpreting results from these
partially confounded designs.
The four possible five-level designs are given in Table 6
using the cyclic generator 22 ? 21 ? 2 ? 1 ? 22, and the
four possible difference vectors. Note that very similar designs
can be obtained using the cyclic generator 22 ? 1 ? 2 ? 21
? 22, simply by replacing 21 by +1 throughout the tables. It
is important to recognise that the order of levels does, in this
case, have some significance, and an inferior design may be
obtained by simply swapping the levels around. However, it is
essential to recognise that only 12 successive factors are
mutually orthogonal, and span each other’s calibration space
symmetrically. The 13th factor has a correlation of 21 with the
first factor, and so on for pairs of factors j and j + 12. The only
way to overcome this and still perform 25 experiments is by
changing the cyclic generator, which will result in some
correlation and asymmetry in predictions between factors j and
j + 6, but factors j and j + 12 will no longer be indistinguishable.
For example, a cyclic generator of the form 22 ? 2 ? 1 ? 21
? 22 results in a correlation coefficient between factors 1 and
7 of 20.1 and between factors 1 and 13 of 20.8, in contrast to
the corresponding correlations of 0 and 21.0 for the cyclic
generators of Table 6.
Conclusion
Despite an enormous interest in calibration experiments within
the chemometrics literature, there has been an extremely limited
discussion of the design of such experiments. Chemometricians
have tended to borrow designs from other disciplines, yet it is
only in chemistry that there is such an emphasis on multilevel
experiments, primarily for calibration, where several concentra-
tions are necessary for meaningful models. There is much
comparison and discussion of algorithms and many empirical
studies in which laboratory mixture experiments are employed
to develope multivariate calibration models, whose predictive
ability is then explored. However, the nature of the calibration
set used to produce the model is essential to the resultant
confidence in the predictions.
The tables in this paper provide guidance as to the use of a
useful class of multivariate calibration designs. It is recom-
mended that up to 12 factors, five level mixture designs are
employed. Beyond 12 factors, either the number of levels or the
number of experiments must be increased. Theory suggests that
it is impossible to obtain basic properties unless the number of
levels is either a prime number or a power of a prime number,
so that a six-level design is not feasible. Extension to a seven-
level design results in at least 49 experiments; extending the
five-level design to 50 experiments is complex and not reported
in this paper, but is an alternative.
D. A. Cirovic is thanked for helping in the initial literature
survey.
References
1 Martens, H., and Næs, T., Multivariate Calibration, Wiley, New
York, 1989.
2 Höskuldsson, A., J. Chemom., 1988, 2, 211.
3 Wold, S., Geladi, P., Esbensen, K., and Ohman, J., J. Chemom., 1987,
1, 41.
4 Brereton, R. G., Analyst, 1995, 120, 2313.
5 Malinowski, E. R., Factor Analysis in Chemistry, Wiley, New York,
2nd edn., 1991.
6 Araujo, P. W., Brereton, R. G., Analyst, 1995, 120, 2497.
7 Analytical Methods Committee, Analyst, 1994, 119, 2363.
8 Krutchkoff, R. G., Technometrics, 1967, 9, 425.
9 Brereton, R. G., Chemometrics: Applications of Mathematics and
Statistics to Laboratory Systems, Ellis Horwood, Chichester, 1993.
10 Araujo P. W., and Brereton R. G., Trends Anal. Chem., 1996, 15,
63.
11 Cirovic, D. A., Brereton, R. G., Walsh, P. T., Ellwood, J. A., and
Scobbie, E., Analyst, 1995, 121, 575.
 View Article Online

Analyst, December 1997, Vol. 122 1529

12 Araujo P. W., and Brereton R. G., Analyst, 1997, 122, 621.
13 Allus M. A., Brereton R. G., and Nickless G., Chemom. Intell. Lab.
Syst., 1989, 6, 65.
14 Morgan E., Chemometrics: Experimental Design, Wiley, Chichester,
1995.
15 Deming S. N., and Morgan S. L., Experimental Design: A
Chemometric Approach, Elsevier, Amsterdam, 2nd edn., 1993.
16 Box G. E. P., Hunter W. G., and Hunter J. S., Statistics for
Experimenters, Wiley, New York, 1978.
17 Plackett R. L., and Burman J. P., Biometrika, 1946, 33, 305.
18 Stevens W. L., Ann. Eugenics, 1939, 9, 82.
Published on 01 January 1997. Downloaded on 13/07/2014 10:06:21.
19 Euler L., Memoir presented to Academy of Science of St. Petersburg
on 8th March 1779, published as Leonardi Euleri Opera Omnia, Sér.
1, 1932, 7, 291.
20 Dénes J., and Mullen G. L., Discrete Math., 1993, 111, 157.
21 Logothetis, N., and Wynn, H. P., Quality Through Design: Experi-
mental Design, Off-line Quality Control, and Taguchi’s Contribution,
Clarendon Press, Oxford, 1989
Paper 7/03654J
Received May 27, 1997
Accepted September 18, 1997