Journal of Applied Statistics

This article was downloaded by: [201.147.144.
194]
On: 26 June 2012, At: 15:49
Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Applied Statistics

Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/cjas20
Why is not design of experiments

widely used by engineers in Europe?
a a a
Martín Tanco , Elisabeth Viles , Maria Jesus Álvarez & Laura
a
Ilzarbe
a
Department of Industrial Management, TECNUN, University of
Navarra, Paseo Manuel Lardizabal 13, 20018, San Sebastián, Spain
Available online: 19 Nov 2010
To cite this article: Martín Tanco, Elisabeth Viles, Maria Jesus Álvarez & Laura Ilzarbe (2010): Why
is not design of experiments widely used by engineers in Europe?, Journal of Applied Statistics,
37:12, 1961-1977
To link to this article: http://dx.doi.org/10.1080/02664760903207308
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-

conditions
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation
that the contents will be complete or accurate or up to date. The accuracy of any
instructions, formulae, and drug doses should be independently verified with primary
sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand, or costs or damages whatsoever or howsoever caused arising directly or
indirectly in connection with or arising out of the use of this material.
Vol. 37, No. 12, December 2010, 1961–1977
Why is not design of experiments widely used

by engineers in Europe?
Martín Tanco*, Elisabeth Viles, Maria Jesus Álvarez and Laura Ilzarbe
Department of Industrial Management, TECNUN, University of Navarra, Paseo Manuel Lardizabal 13,
Downloaded by [201.147.144.194] at 15:49 26 June 2012
20018 San Sebastián, Spain
(Received 5 February 2008; final version received 24 July 2009)
An extensive literature review was carried out to detect why design of experiments (DoE) is not widely used
among engineers in Europe. Once 16 main barriers were identified, a survey was carried out to obtain first-
hand information about the significance of each. We obtained 101 responses from academics, consultants
and practitioners interested in DoE. A statistical analysis of the survey is introduced, including: (a) a ranking
of the barriers, (b) grouping of barriers using factorial analysis, (c) differences between characteristics of
respondents. This exploratory analysis showed that the main barriers that hinder the widespread use of
DoE are low managerial commitment and engineers’ general weakness in statistics. Once the barriers were
classified, the most important resultant group was that related to business barriers.
Keywords: barriers; design of experiments; engineers; industry; survey
1. Introduction
Lye [18] defined the design of experiments (DoE) as a methodology for systematically applying
statistics to the experimentation process. More precisely, it can be defined as a series of tests in
which purposeful changes are made to the input variables of a process or system so that one may
observe and identify the reasons for these changes in the output response [20]. Based on this
definition, DoE is defined throughout this article as a broad subject, encompassing methods such
as response surface methodology, robust parameter methodology (RPM) or Taguchi methods.
Engineers perform experiments and analyze data as an integral part of their job. Regardless of
their background in statistics, engineers will use statistics in their experimentation [4]. However,
there is still a significant gap between theoretical development of DoE and its effective application
in industry.
Although these techniques are commonly found in statistics and quality literature, they are
hardly used in European industry. Gremyr et al. [12] first focused on the use and knowledge of
robust parameter design in Sweden, revealing that this technique is only used by 18% of Swedish
companies. These results correspond to a larger European project in which the use of RPM in
*Corresponding author. Email: mtanco@tecnun.es
ISSN 0266-4763 print/ISSN 1360-0532 online

© 2010 Taylor & Francis
DOI: 10.1080/02664760903207308
http://www.informaworld.com
1962 M. Tanco et al.
companies from five European countries (Netherlands, Germany, Ireland, Spain and Sweden) was
studied [9]. More recently, Bergquist and Albing [3] analyzed the use of statistical methods in the
companies where alumni students of Lulea University of Technology (Sweden) had worked. The
survey showed that 18% had used DoE, although only 3% of companies apply it to the most relevant
processes.
A survey among manufacturers in the Basque country with more than 50 employees was also
carried out by our department [24]. Results showed that 94% of companies undertake experimen-
tation; most of them use the one factor at a time (OFAT) strategy and only 20% of those follow
a pre-established statistical methodology. Furthermore, results show that a lack of knowledge
of general statistics is commonplace, and only 31% of companies claim to be knowledgeable
about DoE. The generalization of these results were presented in [23].
Since this is not a new problem, an extensive literary review was conducted to compile the
barriers, which hinder the application of DoE in industries.Afterwards, a survey among academics,
consultants and practitioners from Europe was designed in order to evaluate the significance of
each barrier.
Consequently, the purpose of this exploratory research is to pinpoint which barriers most
significantly hinder the application of DoE, and how to best deal with those barriers. This article
is intended to help identify pitfalls and initiate discussion of DoE in the European engineering
community.
In Section 2, the identification and explanation of the barriers are presented. In Section 3,
the details of the survey methodology are given. In Section 4, a characterization of the responses
received is given and in Section 5, the main results obtained from different analyses are presented.
Finally, conclusions are intended to encourage the scientific community to focus on overcoming
the most significant barriers.
2. Barriers which hinder widespread use of DoE

2.1 Barrier identification
In order to identify the barriers that engineers may come up against when applying DoE, a
brainstorming session was carried out among our group of researchers. Afterwards, an extensive
literary review of publications helped to identify a significant number of barriers that affect the
widespread use of DoE. Since our search yielded massive amounts of information about these
barriers, we were forced to narrow the field to 16 barriers by using affinity diagrams.
However, we still had a long list of barriers. In order to identify the most common pitfalls, it
was necessary to carry out a ranking among them. We conducted a survey among DoE experts
and practitioners to obtain the desired ranking.
2.2 Short definition

As mentioned earlier, 16 barriers that hinder the application of DoE were hypothesized, as listed
in Table 1. Next, a brief explanation of each barrier identified is included in random order, as in
the questionnaire. Since we could not mention all 80 references, we decided to include at least
one reference per barrier. Further details of the literature review may be found in Tanco et al. [25].
2.2.1 B1: resistance to change

Since engineers often do OFAT experiments, you must be able to convince practicing engineers
that what they have been doing for years can be improved upon [8]. Moreover, most engineers
believe that they must do additional work to prove what they already knew [1].
Journal of Applied Statistics 1963
Table 1. List of barriers that hinder widespread use of DoE.
B1 Resistance to change
B2 Bad image of statistics
B3 Low commitment of managers
B4 Previous bad experience with DoE
B5 Absence of teamwork skills
B6 Not enough software aid
B7 Lack of methodologies to guide users through experimentation
B8 Insufficient resources
B9 Poor statistical background
B10 Absence of theoretical developments to solve real industrial problems
B11 DoE is not taught to engineers
B12 Poor statistical consultancy
B13 Statistical jargon is used to explain DoE
B14 DoE is taught badly
B15 Publications do not reach engineers
B16 DoE is not widely used because it is a complex tool
2.2.2 B2: bad image of statistics

Many engineers have a negative image of statistics [15]. The word “statistics” often invokes fear
and resistance in engineers.
2.2.3 B3: low commitment of managers

As for any quality improvement initiative, it is essential to have strong managerial commitment
while applying DoE [3]. Most managers are unaware of the importance of statistical techniques
for process and product improvement.
2.2.4 B4: previous bad experiences with DoE

Negative experiences with DoE may discourage and limit its use in the future [6]. Experiments
may have failed because the appropriate design or analyses were not used. In addition, non-
technical issues such as poor planning or the failure to identify all the factors are sometimes not
taken into account [13].
2.2.5 B5: absence of teamwork skills

Since experimentation is a team process, its success depends on involving the necessary people
who will work as a team [17]. Poor interpersonal relationships and lack of inner communication
may cause the project to fail.
2.2.6 B6: not enough software aid

Although, nowadays, there are many commercial software products and expert systems to aid
in the experimentation, they are not enough to satisfy business needs. Sometimes, they lead to
erroneous application of statistical methods, are poor at handling technical features and do not
cover all the steps necessary for a DoE project [26].
2.2.7 B7: lack of methodologies to guide users through experimentation

The complete methodology needed to implement DoE may be unclear since so much time is
devoted to explaining data analysis. Often there are so many unstructured, unorganized and uneven
elements that a thorough reorganization is needed for efficient management [21].
2.2.8 B8: insufficient resources

Even though many engineers appreciate the power of DoE, they believe that more resources such
as time, money and materials are needed to use it [19].
2.2.9 B9: poor statistical background

Statistical courses generally focus too much on probability theory and hypothesis testing instead
of on problem solving through statistical thinking. Consequently, statistical methods needed for
DoE are generally misunderstood and wrongly applied [4].
2.2.10 B10: absence of theoretical developments to solve real industrial problems1

Some industrial problems are so complicated, and have so many restrictions that they cannot be
solved by using current methods of DoE application.
2.2.11 B11: DoE is not taught to engineers

Engineers and scientists receive little or no training in designed experiments [10]. Most statistics
professors believe that one course is not enough to teach DoE concepts.
2.2.12 B12: poor statistical consultancy

Consultation often results in creating unrealistic expectations, meaning that problems are not
analyzed deeply enough [15]. This is a common complaint about statistical consultation. More-
over, many small and medium enterprises (SME) are unable to pay the high costs of thorough
consultation.
2.2.13 B13: statistical jargon is used to explain DoE

Many references and books are written in statistical terminology or jargon that is obscure to
engineers [22]. Obscure explanations make the application of DoE more difficult.
2.2.14 B14: DoE is taught badly

DoE is not taught well. Many professors lack practical experience with DoE and do not present
real case studies in class [5]. Furthermore, students are not encouraged to conduct practical
experiments. Moreover, DoE courses are generally incomplete; more than 80% of course content
is dedicated to analysis [2].
2.2.15 B15: publications do not reach engineers

Engineers, mainly those from SME, do not have access to books and articles that explain details of
the technique [11]. Moreover, publications are generally focused on technical problems rather than
on the whole experimentation process. Also, few reports of practical experiments are published,
and failed experiments are rarely mentioned.
2.2.16 B16: DoE is not widely used because it is a complex tool

DoE is seen as a difficult technique because of the inherent complexity of the tools, so unless
engineers are well educated in statistical methods including DoE, the technique will never become
widespread [7].
3. Survey methodology
Once the barriers were detected and grouped, a short questionnaire was developed using rec-
ommendations from the survey literature. Useful comments from experts were received before
launching the survey. The online survey ran from April to June 2007 and was available at
http://examinador.tecnun.es/mtanco/encuesta.asp. This exploratory study was mainly directed
toward members of the European Network for Business and Industrial Statistics (ENBIS),2 which
allowed access not only to academics but also practitioners interested in DoE. Four different
initiatives were carried out in April 2007 to obtain responses from ENBIS members.
• Print copies of the survey were distributed among participants of the ENBIS-DEINDE
conference held at Turin, Italy from 11–13 April 2007.
• An announcement was included in the April quarterly newsletter from ENBIS (ISSN: 1871-
8132).
• The survey was announced on the ENBIS news page of the ENBIS web page (www.enbis.org).
• The results were presented to the ENBIS council and included in its monthly report.
Since we did not have enough responses to fulfill our quota, we decided to send the URL link,
via e-mail, to mainly Spanish colleagues in business and industry. We are aware of the bias in
these responses, and have dedicated a section to analyze differences introduced by the bias.
The questionnaire had two parts. In the first part, the respondents were asked to rate each
barrier using a scale from 1 (insignificant) to 10 (very significant). Each barrier acted as a link,
which, when clicked, opened up a brief explanation of each. The second part consisted of seven
questions, consisting mostly of multiple choice questions, which were included to characterize
the respondents. These questions sought out the following information:
• How often had they used DoE technique?

• Which was their profession: academic, consultant or practitioner?
• In which types of industries were their clients or their company involved?
• What was their highest completed level of education?
• In what field had they majored?
• In which country were they working?
• How old were they?
The survey was in English and was designed to be completed in approximately 5 min, although
the actual average response time was 8 min. All participants were assured of anonymity, since
there were no personal questions in the survey.
Many statistical tools were used to analyze the results. First, a two-way ANOVA was used
to observe if significant differences existed between the ratings of each barrier. Next, factorial
analysis was used to see if it was possible to classify the barriers into fewer groups. Finally,
ANOVA tests and graphics such as multi-vary charts or radial diagrams were used to observe
differences in the ratings caused by respondent characteristics.
4. Response characterization
A total of 101 completed questionnaires were received from April to June 2007. Thirty-four per
cent of responses came from academics, 25% from consultants, 19% from employees of small
or medium companies (< 250 workers) and 35% from employees of large companies (> 250
workers). The total comes out to more than 100%, since respondents were able to select more
than one category. 8% of respondents described themselves as both academics and consultants,
Table 2. Company type.
Industry type Respondents
Automotive 35
Chemical 24
Metals 22
Machinery 19
Pharmaceutical 18
Food 11
Others 43
Do not work for industries 9
Total 101
4% as both consultants and employees of large companies and finally 2% described themselves
as both consultants and employees of small companies.
Respondents were asked to describe which field their clients or their companies were involved
in by using one of eight pre-tabulated options. As shown in Table 2, the most common were the
automotive and chemical industries, although 43 respondents claimed to be working in another
field.
Regarding educational background, 24% of respondents were graduates, 42% were post-
graduates and 33% had earned a PhD. Most of the respondents were engineers (49%), followed
by statisticians and mathematicians (19%) and chemists (16%). Moreover, the majority of respon-
dents were familiar with DoE, since 37% had applied this technique more than 10 times, 17%
more than five times and 38% at least one time. The average and median age of respondents was
42 years old. Twenty-six per cent of respondents were younger than 35, 38% were between 35
and 45, and the remaining 36% were older than 45.
Finally, respondents were asked to name the country in which they work. Table 3 shows that 89%
of responses were from 10 different European countries. However, there is a significant bias due to
the high number of responses received from Spain. This was an expected bias caused by responses
from colleagues in Spain (most of them working for companies). A table of contingency states
with a confidence level of 99% that there is no independence (they are correlated) between the
respondent’s profession (academic, consultant or practitioner) and their corresponding country.
Luckily, there is no other significant correlation between respondent characteristics. Consequently,
the ranking will be analyzed by stratifying respondent category and country.
Table 3. Countries of respondents.
Country Respondents Academics Consultants Small industry Large industry
Spain 54 11 10 15 22
Italy 11 9 3 0 1
Germany 5 3 2 0 1
Denmark 4 1 1 1 2
England 4 1 2 1 1
France 4 2 0 0 2
Netherlands 3 1 1 0 2
Portugal 2 1 0 0 1
Belgium 2 1 0 1 0
Liechtenstein 1 0 1 0 1
Non-European countries 11 4 5 1 3
Total 101 34 25 19 36
5. Results
5.1 General ranking of barriers
Respondents were asked to rate each barrier using a scale from 1 (insignificant) to 10 (very
significant). The average results obtained for each of the 16 barriers are presented graphically
in Figure 1. Most of the barriers appear to be significant due to the high rate they received.
Furthermore, nine of the barriers were rated higher than average (5.5).
A two-way ANOVA considering the respondents as a blocking factor was used to detect if the
rating given to each barrier was significantly different from the others. The analysis carried out
allowed us to divide the barriers into four main groups, which are displayed in different colors
in Figure 1. According to the survey, “low commitment of managers” (B3) and “poor statistical
background” (B9) are the barriers most responsible for hindering the widespread use of DoE.
Statistical tests show that both barriers, which make up group one, are significantly greater than
the others. The second group is formed by “resistance to change” (B1), “DoE is not taught to
engineers at universities” (B11) and “lack of methodologies to guide users through the whole
experimentation process” (B7). This group was formed by those who are at least significantly
greater than more than two barriers. The third group is formed by the largest group of barriers,
including those from B10 to B2 as shown in Figure 1, ranked number 6 and 14, respectively.
The ANOVA demonstrated that each barrier of this group is significantly greater than B6 and
B4. Consequently, the final group includes “not enough software aid” (B6) and “previous bad
experiences with DoE” (B4). These barriers appear to affect the application of DoE to some degree.
5.2 Grouping of barriers

We believed once barriers were ranked it would be useful to narrow them down into a few groups.
The full components analysis, a kind of factorial analysis, was the most adequate statistical method
to complete this task.
Factorial analysis is a generic name given to a class of statistical multivariate methods with the
goal of representing the interrelationships among a set of variables V by a number of underlying,
Figure 1. General rating of the barriers from 1 (insignificant) to 10 (very significant). Four different color
groups were identified according to significance in the ANOVA table.
linearly independent reference variables called factors F, with F < V. The main goals of factor
analytic techniques are: (1) to reduce the number of variables and (2) to detect structures in the
relationships between variables [14].
Academic literature classifies factorial analysis as one of the following groups:
(1) Exploratory analysis: The factors are unknown a priori and are determined by the factorial
analysis.
(2) Confirmatory analysis: The analysis is used to confirm a proposed a priori model in which
some factors represent the original variables.
We conducted an exploratory analysis by choosing the most used method for extracting the factors’
principal component.3 One of the first steps of the factorial analysis was to check the hypothesis,
although most are conceptual rather than statistical. Basically, as the researcher must ascertain
whether or not the variables are independent, the data matrix must have enough correlations to
justify the application of the factorial analysis. Consequently, the correlation matrix (Table 4)
was calculated and analyzed to determine the suitability of the method. The determinant of 0.056,
the p-value of the Bartlett test of 0.00 and a Kaiser–Meyer–Olkin index of 0.540 ( >0.5 [16]),
indicated that this technique could be adequately applied.
There is no lack of criteria to determine the number of factors to keep. Most of them use, in
some way, the resulting “scree plot” shown in Figure 2. We use the Kaiser rule, which selects
only the principal components with an “eigenvalue” greater than unity. Consequently, as can
be seen in the graph, there are only six components with “eigenvalues” greater than 1. Conse-
quently, the barriers can be represented by six principal components, which make up 61% of the
total variance.4 Moreover, the resulting communality index for each barrier (variable) describes
which proportion of the variance is explained by the principal components selected. Since val-
ues of communalities for all barriers are greater than 0.5, the continuation of our analysis was
justified [14].
The next step, after having selected the number of factors to keep, was to choose a method
for the rotation of the components in order to obtain a better interpretation of the groups. We
decided to make an orthogonal rotation, which guarantees that components remain orthogonal
(not correlated) to other components. The technique used was the “vari-max”, which is the most
common among those that make orthogonal rotations. This result in a rotated components matrix,
which represents the relationship between the variables (barriers) and the rotated components,
called factorial loads.
5.2.1 Group interpretation

To finalize the factorial analysis, each of the principal components was analyzed by studying
which variables saturate and contribute to each factor. To accomplish this task, we used rotated
components matrix coefficients, shown in Table 5. Values lesser than 0.3 have been omitted in
order to facilitate interpretation [14].5
Next, the resulting six components are presented with subsequent names and explanations. The
order in which they are presented is due to the number of components. The lower the number
of components, the higher the amount of variation it explains. However, this ranking does not
accurately reflect the significance of each barrier.
(G1) Low user support: For the widespread application of DoE to take hold, it is necessary to
have quality software assistance and complete methodologies to guide users through the
whole experimentation process. Users must have good teamwork skills and instruction must
be explained clearly and without statistical jargon.
Table 4. Correlation matrix.
B1 B2 B3 B4 B5 B6 B7 B8 B9 B10 B11 B12 B13 B14 B15 B16

B1 1 0.16 0.31 0.01 − 0.08 − 0.13 − 0.01 0.05 0.14 0.08 0.16 0.09 0.06 − 0.04 0.05 − 0.13
B2 1 0.01 0.28 − 0.08 0.00 0.01 − 0.06 0.27 − 0.04 0.17 0.08 0.05 0.27 0.03 − 0.01
B3 1 0.10 0.06 − 0.03 0.07 0.11 − 0.02 − 0.14 0.15 − 0.03 0.01 0.10 0.01 − 0.17
B4 1 0.03 − 0.01 0.17 0.16 0.20 0.19 0.03 0.10 0.21 0.29 0.09 0.18
B5 1 0.26 0.22 0.11 0.01 0.28 0.03 − 0.14 0.02 0.14 0.05 0.19
B6 1 0.51 0.16 0.05 0.16 0.01 0.21 0.20 0.14 0.13 0.30
B7 1 0.08 0.17 0.10 0.27 0.12 0.22 0.36 0.15 0.20
B8 1 − 0.02 0.11 − 0.09 0.03 − 0.10 0.07 0.10 0.39
B9 1 0.19 0.33 0.21 0.12 0.28 0.03 0.08
B10 Symmetrical 1 − 0.02 0.13 − 0.07 0.09 0.05 0.26
B11 1 0.25 0.14 0.24 0.21 − 0.14
B12 1 0.27 0.19 0.16 0.08
B13 1 0.27 0.09 0.09
B14 1 0.21 0.04
B15 1 0.30
B16 1
1969
Figure 2. Scree plot of the factorial analysis. This plot is useful in determining the number of factors to keep.
Table 5. Rotated component matrix (vari-max method).
Components
1 2 3 4 5 6
B1 0.718
B2 0.699
B3 0.792
B4 0.752 0.308
B5 0.595 − 0.377 0.377
B6 0.697
B7 0.775
B8 0.771
B9 0.466 0.307 0.481
B10 0.805
B11 − 0.361 0.466 0.386
B12 0.768
B13 0.346 0.391 − 0.324
B14 0.446 0.595
B15 0.368 0.486
B16 0.723
(G2) Negative image of statistics: Companies have a negative image of statistics, which hinders
the use of many statistical methods. This worsens when negative DoE experiences take
place. Furthermore, a weak background in statistics and the way DoE is explained may
dissuade users from applying statistics in companies.
(G3) Unknown method: Since the DoE technique is generally not explained at universities and
publications do not reach practitioners like other statistical methods, it is primarily consid-
ered to be an overly complex tool. Consequently, practitioners believe they have insufficient
resources to apply DoE. Unfortunately, those who know little about it and attempt to apply
it often fail, and consequently refuse to use the technique in the future.
(G4) Poor DoE promotion: It is imperative that the promotion of DoE among companies becomes
widespread. Education in DoE at universities and quality statistics consultation for busi-
nesses is also of great importance. Moreover, the content and the way in which publications
reach practitioners is another way to increase awareness of DoE.
(G5) Inherent to business: The resistance that most companies have to change makes the introduc-
tion of new techniques difficult. Moreover, low managerial commitment to these initiatives,
based on the lack of training and familiarity with the method, increases the resistance to
change at companies.
(G6) Development needed: There is still an absence of theoretical development needed to solve
real industrial problems. A weak background in statistics, the absence of teamwork skills
and the way DoE is explained make some recent developments still too complex to apply
in business.
Attempting to validate the results is worthwhile. The ideal way to do this is to make a confir-
matory factorial analysis with new data (new survey). Since this was too resource demanding,
we used a common practice to validate it: sample division. We divided our sample into two ran-
dom groups of equal size, after which we applied the factorial analysis to each of these groups.
We carried out two different random divisions. Similar results were obtained from the different
factorial analyses carried out on each of the samples.
5.2.2 Group ranking

Once the groups were named and explained, they were ranked: the barriers that most stifled the
usage of DoE among engineers in Europe were ranked first, and so on. Figure 3 shows the overall
ranking, considering the general barrier ratings with factorial punctuations obtained from the
rotated components matrix.
The strongest barriers are those “inherent to business”, reflecting that companies are reluctant
to use DoE. The second most formidable barrier is “development needed” which demonstrates
that it is necessary to continue developing the technique in order to make DoE easier (more
understandable and practical) and more widely used. The third and fourth barriers are “poor DoE
promotion” and “unknown method”, respectively. Finally, “low user support” and “negative image
of statistics” are the groups which seem less influential.
5.3 Barrier importance stratified by respondent characteristics

As previously explained, questions were included in the survey to determine respondent char-
acteristics. The influence of these characteristics on barrier ranking is evaluated. Each of the
barriers was analyzed with a one-way ANOVA test to detect significant differences between
the respondent characteristics. Only significant differences, p-value < 0.05, between groups are
presented.
Figure 3. Barrier group ratings with factorial punctuations.

5.3.1 Stratification by respondent category

In this section, we tried to measure the correlation between respondent’s profession (academics,
consultants and practitioners) and barrier rating. Figure 4 shows a radial diagram reflecting the
rating of barriers by profession.
• Academics–practitioners: Academics believe that not teaching DoE to engineers at universities

is a significant barrier (B11), while practitioners disagree. Furthermore, academics criticize
consultants; they believe poor statistical consultancy (B12) is also a significant barrier, while
practitioners rank it 14th. On the other hand, while practitioners believe that the complexity
of DoE most hinders the widespread use of DoE (B16), while academics rank this barrier
as 15th. These differences offer further evidence of the existing gap between academics and
practitioners when it comes to DoE.
• Consultants–practitioners: Practitioners rank the absence of teamwork skills in business as the
largest barrier (B5), while consultants rank it 15th. The same thing occurs with software aid
(B6); practitioners believe it is a greater barrier than consultants. On the other hand, there is
once again a great difference in the importance given to teaching DoE at universities (B11)
since consultants believe it is a significant barrier while practitioners do not. Furthermore,
consultants ranked resistance to change (B1) fairly high at 7.7, while practitioners ranked it at
a lower 6.1. Once again we can conclude that a communication problem exists, widening the
gap between consultants and practitioners.
• Academics–consultants: There are no significant differences between the ratings of both groups,
in part due to the fact that there are some respondents who classified themselves as both
categories. However, the biggest differences between these groups lie in the rating of “absence
of teamwork skills” (B5), “not enough software aid” (B6) and “lack of methodologies to guide
users through the whole experimentation process” (B7), which are more significant barriers for
academics than consultants.
Finally, we would like to stress the fact that there are no differences between these categories
when it comes to the two most important barriers detected in Section 5.1. Therefore, the low
commitment of management (B3) and the poor statistical background (B9) are really influential
to all categories.
5.3.2 Stratification by country of respondent

Afterwards, we analyzed how survey responses varied according to a respondent’s country. As
previously mentioned, due to the high volume of responses from Spain we have created two
groups; those from Spain and those from outside Spain. Furthermore, to interpret the results, we
have to bear in mind the high correlation between respondent’s profession (academics, consultants
or practitioners) and the country where they work.
Once again, there is no difference between the two most significant barriers (B3, B9). As shown
in Figure 5, the significant differences in the barriers rating are in order of importance: “DoE
is not taught to engineers at universities” (B11), “absence of teamwork skills” (B5), “DoE is
not used widely because it is a complex tool” (B16), “resistance to change” (B1), “publications
do not reach engineers” (B15) and “absence of theoretical development to solve real industrial
problems” (B10). The rating of the first four barriers was influenced by the existing correlation of
the respondent’s profession , which was explained in the previous section. However, we believe
that the presence of B15 and B10 is due to the late introduction of theoretical developments in
Spanish companies, because of the language barrier and the lack of access that engineers have to
publications.
Figure 4. Radial diagram showing the barrier rating stratified by respondent’s profession.
Figure 5. Radial diagram showing the barrier rating stratified by respondent’s countries.
5.3.3 Others
We analyzed correlations between barrier rating and characteristics of respondents. Education
level and profession do not cause any significant difference between the ratings. However, we
found that respondent’s age strongly affected the ratings of “insufficient resources” (B8), with
older people downplaying its importance. This barrier is also affected by the number of times
that respondents have applied DoE. A multi-vary chart (Figure 6), which is a tool that graphically
displays patterns or families of variation to compare the variability among at most four factors, was
Figure 6. Multi-vary chart for B8 – insufficient resources. This chart provides a visual display of the means
of each family, to show the differences among them.
Figure 7. Multi-vary chart for B5 – absence of teamwork skills.
used to show the differences between characteristics of both groups of respondents. Interaction
between these characteristics can also be seen, since older, more experienced respondents rank
the barrier as unimportant.
Moreover, differences also exist between the number of times that respondents have applied
DoE in “absence of teamwork skills” (B5) and “DoE is not used widely because it is a complex
tool” (B16). The multi-vary chart presented in Figure 7 shows that those who applied DoE often
believe that the “absence of teamwork skills” (B5) is not an important barrier. Secondly, as shown
in Figure 8, those who had not applied DoE often believe that the complexity of DoE hinders its
widespread use (B16).
The last three graphs show that those with experience in DoE (more than five applications)
believe these three barriers to be less important. This could be extended, since there is significant
difference of 0.4 in the average of all values given to each barrier. Does this mean that experience
eases the application of DoE? Or on the other hand, do those who have experience in DoE
underestimate the barriers that hinder its widespread use?
Figure 8. Multi-vary chart for B16 – DoE is not widely used because it is a complex tool.
6. Conclusions
The on-line survey carried out among European academics, consultants and practitioners about
the significance of the barriers that hinder the widespread use of DoE among engineers have been
analyzed. Those barriers were identified in an exhaustive literary review.
The survey analysis shows that the main barriers that hinder the widespread use of DoE are the
low commitment of managers and the poor background of engineers in statistics. These barriers
present significant differences in the rating over the others; however, there are no differences
between the characteristics of survey respondents.
We carried out a factorial analysis in order to narrow down the group of barriers, taking into
account the existing correlation between them. It was possible to group all the barriers into six main
groups: business barriers, development barriers, DoE promotion, unknown method, user support
and negative image of statistics. Business barriers that include the low commitment of managers
and high resistance to change are the hardest to overcome. Finally, differences in barrier rating
caused by respondent characteristics were analyzed. Slight differences exist between different
ages and experience using DoE. However, greater differences exist in the significance given to
each barrier by academics, consultants and practitioners, confirming that a gap exists between
DoE experts and engineers.
Our research aim was to get first-hand information about just how much each barrier hinders
the application of DoE among engineers. This exploratory study identified the most significant
barriers to help overcome the low usage of DoE. Besides the limitations of this study, conclusions
are intended to encourage the scientific community to focus on overcoming the most troublesome
barriers. We hope this article proves to be useful in identifying pitfalls and generating a realm of
discussion of DoE in Europe.
Although this paper aims to analyze the reasons why most companies do not apply DoE to their
projects, future research could focus on developing guidelines to help companies find the right
mix of DoE and other alternatives (OFAT) to carry out all company projects.
Acknowledgements
We are grateful to the three anonymous referees for helpful comments that improved this paper significantly. Moreover,
the authors would like to thank Enrique del Castillo and Tony Greenfield for their valuable comments and suggestions
regarding survey elaboration. Finally, we would like to thank Ron Kennett, past president of ENBIS, for his support and
effort in distributing the survey among ENBIS members.
Notes
1. B10 was identified only during the brainstorming session.
2. ENBIS Mission: “(a) Foster and facilitate the application and understanding of statistical methods to the benefit of
European business and industry, (b) provide a forum for the dynamic exchange of ideas and facilitate networking
among statistical practitioners (a statistical practitioner is any person using statistical methods whether formally
trained or not) and (c) nurture interactions and professional development of statistical practitioners regionally and
internationally”.
3. Although some proclaim the superiority of common factorial analysis over the full components analysis, Velicer’s
results [27] show that they usually obtain similar results.
4. Sixty per cent is sufficient in social sciences [14].
5. Since coefficients are the correlation between the variable and the factor, the square of the coefficients is the amount
of total variance of the variable explained by the factor. Therefore, loads of 0.30, 0.50 and 0.70 explain approximately
10%, 25% and 50% of the variable.
References
[1] R.P. Amstong, Commentary: Is there really a communications gap between statisticians and engineers, Technomet-
rics 32(3) (1990), pp. 249–250.

[2] J. Antony, Ten useful and practical tips for making your industrial experiments successful, TQM Mag. 11(4) (1999),
pp. 252–256.
[3] B. Bergquist and M. Albing, Statistical methods: Does anyone really use them? Total Qual. Manag. Bus. Excel.
17(8) (2006), pp. 961–972.
[4] S. Bisgaard, Teaching statistics to engineers, Amer. Statist. 45(4) (1991), pp. 274–283.
[5] S. Coleman and J. Antony, Teaching fractional factorial experiments via course delegate designed experiments,
Qual. Assur. 7(1) (2000), pp. 37–48.
[6] D.E. Coleman and D.C. Montgomery, A systematic approach to planning for a designed industrial experiment,
Technometrics 35(1) (1993), pp. 1–12.
[7] N.R.P. Costa, A.R. Pires, and C.O. Ribeiro, Guidelines to help practitioners of design of experiments, TQM Mag.
18(4) (2006), pp. 386–399.
[8] V. Czitrom, One factor at a time versus designed experiments, Amer. Statist. 53(2) (1999), pp. 126–131.
[9] EURobust, Use and knowledge of robust design methodology: A survey of European industry, Chalmers University,
Sweden, 2003, pp. 1–15.
[10] P.D. Funkenbusch, Practical Guide to Designed Experiments. A Unified Modular Approach, Marcel Dekker,
New York, 2005.
[11] T. Greenfield, Past Issues? Sci. Comput. World 90 (2006), p. 45.
[12] I. Gremyr, M. Arvidsson, and P. Johansson, Robust design methodology: A status in the Swedish manufacturing
industry, Qual. Reliab. Eng. Int. 19(4) (2003), pp. 285–293.
[13] B.H. Gunter, Discussion: A systematic approach to planning for a designed industrial experiment, Technometrics
35(1) (1993), pp. 13–14.
[14] J.F. Hair, R. Anderson, R.L. Tatham, and W. Black, Multivariate Data Analysis, 5th ed., Prentice Hall International,
New Jersey, 1999.
[15] B.A. Hoadley and J.R. Kettering, Communications between statisticians and engineers/physical scientist, Techno-
metrics 32(3) (1990), pp. 243–247.
[16] H.F. Kaiser, An index of factorial simplicity, Psychometrika 39(1) (1974), pp. 31–36.
[17] J. Knowlton and R. Keppinger, The experimentation process, Qual. Prog. 6(2) (1993), pp. 43–47.
[18] L.M. Lye, Tools and toys for teaching design of experiments methodology, 33rd Annual General Conference of the
Canadian Society for Civil Engineering, Toronto, Ontario, Canada, 2005.
[19] J. MacKay, Commentary: Is there really a communications gap betweem statisticians and engineers, Technometrics
32(3) (1990), pp. 263–264.
[20] D.C. Montgomery, Design and Analysis of Experiments, John Wiley & Sons, Inc., New York, 2005.
[21] J.L. Romeu, Teaching engineering statistics to practicing engineers, ICOTS-7, Salvador, Brazil, 2006.
[22] G. Schneider, Process analysis with design of experiments, 2006; software available at http://www.queng.com.
[23] M. Tanco, E. Viles, I. Ilearbe, and M.J. Alvarez, How is experimentation carried out by companies? A survey of three
European regions, Qual. Reliab. Eng. Int. 24(8) (2008), pp. 973–981.
[24] M. Tanco, E. Viles, I. Ilearbe, and M.J. Alvarez, Is Design of Experiments really used?: A survey of Basque industries,
Journal of Engineering Design 19(5) (2008), pp. 447–460.
[25] M. Tanco, E. Viles, I. Ilearbe, and M.J. Alvarez, Barriers faced by engineers when applying design of experiments,
The TQM Journal 21(6) (2009), pp. 565–575.
[26] K.-M. Tay and C. Butler, Methodologies for experimental design: A survey, comparison and future predictions, Qual.
Eng. 11(3) (1999), pp. 343–356.
[27] W.F. Velicer and D.N. Jackson, Component analysis versus common factor analysis: Some issues in selecting an
appropriate procedure, Multivariate Behav. Res. 25(1) (1990), pp. 1–28.

Journal of Applied Statistics

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Journal of Applied Statistics

Uploaded by

Copyright:

Available Formats

This article was downloaded by: [201.147.144.

Journal of Applied Statistics

Why is not design of experiments

Available online: 19 Nov 2010

To link to this article: http://dx.doi.org/10.1080/02664760903207308

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-

Why is not design of experiments widely used

20018 San Sebastián, Spain

(Received 5 February 2008; final version received 24 July 2009)

Keywords: barriers; design of experiments; engineers; industry; survey

*Corresponding author. Email: mtanco@tecnun.es

ISSN 0266-4763 print/ISSN 1360-0532 online

2. Barriers which hinder widespread use of DoE

2.2 Short definition

2.2.1 B1: resistance to change

2.2.2 B2: bad image of statistics

2.2.3 B3: low commitment of managers

2.2.4 B4: previous bad experiences with DoE

2.2.5 B5: absence of teamwork skills

2.2.6 B6: not enough software aid

2.2.7 B7: lack of methodologies to guide users through experimentation

2.2.8 B8: insufficient resources

2.2.9 B9: poor statistical background

2.2.10 B10: absence of theoretical developments to solve real industrial problems1

2.2.11 B11: DoE is not taught to engineers

2.2.12 B12: poor statistical consultancy

2.2.13 B13: statistical jargon is used to explain DoE

2.2.14 B14: DoE is taught badly

2.2.15 B15: publications do not reach engineers

2.2.16 B16: DoE is not widely used because it is a complex tool

• How often had they used DoE technique?

Industry type Respondents

Table 3. Countries of respondents.

Country Respondents Academics Consultants Small industry Large industry

5.2 Grouping of barriers

5.2.1 Group interpretation

Table 4. Correlation matrix.

B1 B2 B3 B4 B5 B6 B7 B8 B9 B10 B11 B12 B13 B14 B15 B16

Journal of Applied Statistics

Table 5. Rotated component matrix (vari-max method).

5.2.2 Group ranking

5.3 Barrier importance stratified by respondent characteristics

Figure 3. Barrier group ratings with factorial punctuations.

5.3.1 Stratification by respondent category

• Academics–practitioners: Academics believe that not teaching DoE to engineers at universities

5.3.2 Stratification by country of respondent

Figure 7. Multi-vary chart for B5 – absence of teamwork skills.

rics 32(3) (1990), pp. 249–250.

You might also like