University of Jordan

0904709
Research Methodology
Dr. Moudar Zgoul

Important Experimental Designs

Student Name Student Number

Salam Albaradie 8170761

Dec 4, 2018
Abstract

This report introduces the basic concepts of experimental design, data collection, and
data analysis The following steps summarize the Important Experimental Designs and
their classified.
 Table of Contents
Abstract ...........................................................................................................................2
1 Introduction ...............................................................................................................6
1.1 Experimental unit .................................................................................................6
1.2 Experiment ..........................................................................................................6
1.3 Treatment ...........................................................................................................6
1.4 Factor .................................................................................................................6
1.5 Replication ..........................................................................................................7
1.6 Experimental error ...............................................................................................7
1.7 Treatment design ..................................................................................................7
2 Important Experimental Designs ...................................................................................7
2.1 Informal experimental design .................................................................................8
2.1.1 Before and after without control design. ...........................................................8
2.1.2 After only with control design ..........................................................................8
2.1.3 Before and after with control design .................................................................9
2.2 Formal experimental design ................................................................................. 11
2.2.1 Completely randomized design ....................................................................... 11
2.2.2 Randomized block design ............................................................................... 14
2.3 Latin Square Designs .......................................................................................... 15
2.4 Factorial designs ................................................................................................ 16
3 Bibliography ............................................................................................................. 18
 List of Figure
Figure 1: An example of an after-only design .......................................................................9
 List of Table

Table 1: An example of a before-after with control group design .............................................9

Table 2 random numbers are ranked from the smallest to the larges. .................................... 12
Table 3 : Group Number are Ranks in the Group. .............................................................. 13
Table 4: A sample layout of a completely randomized design with four treatments (A, B, C and
D) each replicated five times............................................................................................. 13
1 Introduction
Design of experiment means how to design an experiment in the sense that how the observations
or measurements should be obtained to answer a query in a valid, efficient and economical way.
The designing of experiment and the analysis of obtained data are inseparable. If the experiment
is designed properly keeping in mind the question, then the data generated is valid and proper
analysis of data provides the valid statistical inferences. If the experiment is not well designed,
the validity of the statistical inferences is questionable and may be invalid. It is important to
understand first the basic terminologies used in the experimental design.

1.1 Experimental unit

The first step in detailing the data collection protocol is to define the experimental unit. An
experimental or sampling unit is the person or object that will be studied by the researcher. This
is the smallest unit of analysis in the experiment from which data will be collected. For example,
depending on the objectives, experimental or sampling units can be individual persons, students
in a classroom, the classroom itself, an animal or a litter of animals, a plot of land, patients from
a doctor's office, and so on.

1.2 Experiment
A way of getting an answer to a question which the experimenter wants to know.

1.3 Treatment
The treatment structure consists of factors that the researcher wants to study and about which
the researcher will make inferences.

1.4 Factor
A factor is a general type or category of treatments. Different treatments constitute different
levels of a factor. For example, three different groups of runners are subjected to different training
methods. The runners are the experimental units, the training methods, the treatments, where the
three types of training methods constitute three levels of the factor 'type of training'. (McColl's,
1997)
1.5 Replication
repetition of the experimental situation. Usually, experiments must be repeated enough times for
the results to be analyzed statistically.

1.6 Experimental error

The unexplained random part of variation in any experiment is termed as experimental error.
An estimate of experimental error can be obtained by replication.

1.7 Treatment design

A treatment design is the manner in which the levels of treatments are arranged in an
experiment.

2 Important Experimental Designs

Experimental design refers to the framework or structure of the experiment and as such there are
several such experimental design. Experimental design can be classified into two broad categories.
Informal experimental design and Formal experimental design.
Informal experimental design, are those design that normally uses a less sophisticated form of
analysis based on differences in magnitude, whereas formal experimental design offers relatively
more control and use precise statistical procedures for analysis.

Important statically designs are as follows

1. Informal experimental design 2. Formal experimental design

• Before and after without control design. • Completely randomized design (C.R.

• After only with control design. design).

• Before and after with control design. • Randomized block design (R. B. design).

• Latin square design (L.S. design).

• Factorial design.
2.1 Informal experimental design

2.1.1 Before and after without control design.

In design terminology, “before” refers to a measurement being made before an intervention is
introduced to a group and “after” refers to a measurement being made after its introduction.
Equivalent terms for “before” and “after” are “pre” and “post”.
The before-and-after design is the most important non-experimental design, since it is a reasonable
option for an evaluation. Although it suffers from many threats to internal validity, it can, in
many cases, provide preliminary evidence for intervention effectiveness, especially when
supplemented with complementary information.
An example of an after-only design is where you measure safety knowledge in a test only after you
train a work group. The weakness of this approach is that you cannot be sure whether the test
score would have been any different without training. Even if the average score after training was
90%, the training might actually have been ineffective, since the group might also have scored 90%
on the test if it had been given before the training. Thus, the only acceptable use for this type of
design is to ascertain if a certain standard has been met. It is not useful for effectiveness
evaluation.

2.1.2 After only with control design

As the name suggests, with after-only experimental designs measures of the independent variable
are only taken after the experimental subjects have been exposed to the independent variable. This
is a common approach in advertising research where a sample of target customers are interviewed
following exposure to an advertisement and their recall of the product, brand, or sales features is
measured. The advertisement could be one appearing on national television and/or radio or may
appear in magazines, newspapers or some other publication. The amount of information recalled
by the sample is taken as an indication of the effectiveness of the advertisement. Figure 1 An
example of an after-only design.
 Figure 1: An example of an after-only design

The chief problem with after-only designs is that they do not afford any control over extraneous
factors that could have influenced the post-exposure measurements. After-only designs are not
true experiments since little or no control is exercised over any of the variables by the researcher.
However, its inclusion here serves to underline the need for more complex designs.

2.1.3 Before and after with control design

This design involves establishing two samples or groups of respondents: an experimental group
that would be exposed to the marketing variable and a control group which would not be subjected
to the marketing variable under study. The two groups would be matched. That is, the two samples
would be identical in all important respects. The idea is that any confounding factors would impact
equally on both groups and therefore any differences in the data drawn from the two groups can
be attributed to the experimental variable.
Study table 1 which depicts how an experiment involving the measurement of the impact of a sugar
beet seed promotional campaign on brand awareness might be configured with a control group.

Table 1: An example of a before-after with control group design

Experimental Control
Group Group
Before1 measure: % recalling Brand X sugar beet seed 25.5% 25.5%
Exposed to promotional campaign Yes No
'After' measure: % recalling Brand X sugar beet seed 34.5% 24,5%
First, the two groups would be matched: attributes such as age distribution of group members,
spread of sizes of farms operated, types of farms operated, ratio of dependence on hand tools,
animal drawn tools and tractor mounted equipment, etc. would be matched within each group so
that the groups are interchangeable for the purposes of the test. As table 1 conveys, the initial
level of awareness of the sugar beet brand would be recorded within each group. Only the
experimental group would see the test promotional campaign. After the campaign, a second
measure of brand awareness would be taken from each group. Any difference between the 'after'
and 'before' measurements of the control group (C2- C1) would be due to uncontrolled variables.
Differences between the 'after' and 'before' measurements in the experimental group (E2 - E1)
would be the result of the experimental variable plus the same uncontrolled variables affecting the
control group. Isolating the effect of the experimental variable is simply a matter of subtracting
the difference in the two measurements of the control group from the difference in the two
measures taken from the experimental group. To illustrate the computation, consider the following
hypothetical table.
Awareness of the brand within the experimental group has increased by 9 percent. At the same
time, the awareness level, within the control group, appears to have fallen by 1 percent. This could
be due to random fluctuations or a real lowering of awareness due to some respondents forgetting
the brand in the absence of any supporting advertisements/promotions. Thus, the effects of the
test campaign would seem to have been:

Effect of experimental variable = (34.5 - 25.5) - (24.5 - 25.5)

= (9%) - (-1%) = 10%

If a "before and after with control group" experiment is properly designed and executed then the
effects of maturation, pretesting and measurement variability should be the same for the
experimental group as for the control group. In this case. these factors appear to have had a
negative effect on awareness of one percent. Had it not been for the experimental variable, the
experimental group would have shown a similar fall in awareness over the period of the test.
Instead of recording a fall in the level of awareness of the sugar beet brand, the experimental
group actually showed a nine percent increase in brand awareness. However, the design is not
guaranteed to be unflawed. The accurate matching of the two groups is a difficult, some would say
impossible, task. Moreover, over time the rate and extent of mortality, or drop out, is likely to
vary between the groups and create additional problems in maintaining a close match between
groups.

2.2 Formal experimental design

2.2.1 Completely randomized design

In a completely randomized design, treatment levels or combinations are assigned to
experimental units at random. This is typically done by listing the treatment levels or
treatment combinations and assigning a random number to each. By sorting on the random
number, we produce a random order for application of the treatments to experimental units.
The step-by-step procedure for randomization and layout of a CRD are given here for a pot culture
experiment with four treatments A, B, C and D, each replicated five times.

Step 1. Determine the total number of experimental plots (n) as the product of the number of
treatments (t) and the number of replications (r); that is, n = rt.
For our example, n = 5 x 4 = 20. Here, one pot with a single plant in it may be called a plot. In
case the number of replications is not the same for all the treatments, the total number of
experimental pots is to be obtained as the sum of the replications for each treatment. i.e.,
t
n = r
i =1
i where ri is the number of times the ith treatment replicated.

Step 2. Assign a plot number to each experimental plot in any convenient manner; for example,
consecutively from 1 to n.

Step 3. Assign the treatments to the experimental plots randomly using a table of random numbers
as follows. Locate a starting point in a table of random numbers (Appendix 6) by closing your
eyes and pointing a finger to any position in a page. For our example, the starting point is taken
at the intersection of the sixth row and the twelfth (single) column of two-digit numbers. Using
the starting point obtained, read downward vertically to obtain n = 20 distinct two-digit random
numbers. For our example, starting at the intersection of the sixth row and the twelfth column,
the 20 distinct two-digit random numbers are as shown here together with their corresponding
sequence of appearance.
Random number: 37, 80, 76, 02, 65, 27, 54, 77, 48, 73,
Sequence: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
Random number: 86, 30, 67, 05, 50, 31, 04, 18, 41, 89
Sequence: 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
Rank the n random numbers obtained in ascending or descending order. For our example, the 20
random numbers are ranked from the smallest to the largest, as shown in the following:

Table 2 random numbers are ranked from the smallest to the larges.

Random Sequence Rank Random Sequence Rank

Number Number

37 1 8 86 11 19

80 2 18 30 12 6

76 3 16 67 13 14

02 4 1 05 14 3

65 5 13 50 15 11

27 6 5 31 16 7
54 7 12 04 17 2

77 8 17 18 18 4

48 9 10 41 19 9

73 10 15 89 20 20

Divide the n ranks derived into t groups, each consisting of r numbers, according to the sequence
in which the random numbers appeared. For our example, the 20 ranks are divided into four
groups, each consisting of five numbers, as follows:
 Table 3 : Group Number are Ranks in the Group.

Group Number Ranks in the Group

1 8 13 10 14 2

2 18 5 15 3 4

3 16 12 19 11 9

4 1 17 6 7 20

Assign the t treatments to the n experimental plots, by using the group number as the treatment
number and the corresponding ranks in each group as the plot number in which the corresponding
treatment is to be assigned. For our example, the first group is assigned to treatment A and plots
numbered 8, 13, 10, 14 and 2 are assigned to receive this treatment; the second group is assigned
to treatment B with plots numbered 18, 5, 15, 3 and 4; the third group is assigned to treatment C
with plots numbered 16, 12, 19, 11 and 9; and the fourth group to treatment D with plots numbered
1, 17, 6, 7 and 20. The final layout of the experiment is shown Table 4.

Table 4: A sample layout of a completely randomized design with four treatments (A, B, C and D) each replicated
five times

Plot no 1 2 3 4
Treatment D A B B

5 6 7 8
B D D A

9 10 11 12
C A C C

13 14 15 16
A A B C

17 18 19 20
D B C D
2.2.2 Randomized block design
It’s an improvement over the C.R. design. In the R.B design the principle of local content can be
applied along with the other two principles of experimental designs. In the R.B. design, subjects
are first divided into groups, known as blocks, such that within each group the subjects are
relatively homogeneous in respect to some selected variable. The variable selected for grouping the
subjects is one that is believed to be related to the measures to be obtained in respect of the
dependent variable. The number of subjects in a given block would be equal to the number of
treatments and one subject in each block would be randomly assigned to each treatment. In general,
blocks are levels at which we hold the extraneous factor fixed, so that its contribution to the total
variability of data can be measured. The main feature of the R.B design is that in this each
treatment appears the same number of times in each block. The R.B design is analyzed by the
two-way analysis of variance (two-way ANOVA) technique.
Let us illustrate the R.B. design with the help of an example. Suppose four different forms of a
standardized test in statistics were given to each of five students (selected one from each of the
five I.Q. blocks) and following are the scores which they obtained.

Very low IQ Low IQ Average IQ High IQ Very High IQ

Student A Student B Student C Student D Student E

Form 1 82 67 57 71 73

Form 2 90 68 54 70 81

Form 3 86 73 51 69 84

Form 4 93 77 60 65 71

If each student separately randomized the order in which he or she took the four tests (by using
random numbers or some similar device), we refer to the design of this experiment as a R.B.
design. The purpose of this randomization is to take care of such possible extraneous factors (say
as fatigue) or perhaps the experience gained from repeatedly taking the test.
2.3 Latin Square Designs

Latin Square designs are similar to randomized block designs, except that instead of the removal
of one blocking variable, these designs are carefully constructed to allow the removal of two blocking
factors. They accomplish this while reducing the number of experimental units needed to conduct
the experiment. Following is an example of a four treatment Latin Square. The experimental
layout is as follows:

Rows Column 1 Column 2 Column 3 Column 4

Rows 1 A B C D

Rows 2 B C D A

Rows 3 C D A B

Rows 4 D A B C

In the above table, the four treatments are represented by the four letters: A, B, C, and D. The
letters are arranged so that each letter occurs only once within each row and each column. Notice
that a simple random design would require 4 x 4 x 4 = 64 experimental units. This Latin Square
need only 16 experimental units—a reduction of 75%! The influence of a fourth factor may also
be removed from the design by introducing a second set of letters, this time lower case. This
design is known as the Graeco-Latin Square.

Rows Column 1 Column 2 Column 3 Column 4

Rows 1 Aa Bb Cc Dd

Rows 2 Bb Ca Db Ac

Rows 3 Cb Dc Ad Ba

Rows 4 Dc Ad Ba Cb

Four factors at four levels each would normally require 256 experimental units, but this design
only requires 16— a reduction in experimental units of almost 94%! The Graeco-Latin Square is
formed by combining to orthogonal Latin Squares. Graeco-Latin Squares are available for all
numbers of treatments except six.
2.4 Factorial designs

Factorial designs are used in experiments where the effects of varying more than one factor are
to be determined. They are especially important in several economic and social phenomena where
usually a large number of factors affect a particular problem. Factorial designs can be of two
types: (i) simple factorial designs and (ii) complex factorial designs
Simple factorial designs: In case of simple factorial designs, we consider the effects of varying two
factors on the dependent variable, but when an experiment is done with more than two factors,
we use complex factorial designs. Simple factorial design is also termed as a ‘two-factor-factorial
design,’ whereas complex factorial design is known as ‘multi-factor-factorial design.’ Simple
factorial design may either be a 2 x 2 simple factorial design, or it may be, say 3 x 4 or 5 x 3 or
the like type of simple factorial design.

Illustration: (4* 3 simple factorial design) The 4*3 simplex factorial design will usually include
four treatments of the experimental variable and three levels of the control variable. Graphically
it may take following form:

Experimental Variable
CONTROL TREATMENT TREATMENT TREATMENT TREATMENT
VARIABLE A B C D
Level 1 Cell 1 Cell 4 Cell 7 Cell 10

Level 2 Cell 2 Cell 5 Cell 8 Cell 11

Level 3 Cell 3 Cell 6 Cell 9 Cell 12

This model of a simplex factorial design includes four treatments viz. A, B, C and D of the
experimental variable and three levels viz (I), (II), and (III) of the control variable and has 12
different cells as shown above. This shows that a 2 * 2 simple factorial design can be generalized
to any number of treatments and levels. In such a design the means for the columns provide the
researcher with an estimate of the main effects for the levels. Such a design also enables the
researcher to determine the interaction between treatments and levels
Complex factorial designs: experiments with more than two factors at a time involve the use of
complex factorial designs. A design which considers three or more independent variables
simultaneously is called a complex factorial design. In case of three actors with one experimental
variable having two levels, the design used will be termed 2 * 2 * 2 * comple factorial design which
will contain a total of eight cells as shown below:

Experimental Variable

Treatment A Treatment B

Control Control Control Control

Variable 2 Variable 2 Level Variable 2 Variable 2
Level I II level I Level II

Level I Cell 1 Cell 3 Cell 5 Cell 7

Control
Variable 1
Cell 2 Cell 4 Cell 6 Cell 8
Level II

To obtain the first order interaction say, for EV * CV 1in the above stated design, the researcher
must necessarily ignore control variable 2 for which purpose he may develop 2 * 2 design from
the 2 * 2* 2 design by combining the data of the relevant cells of the latter design as has been
shown below:

Experimental Variable
Treatment A Treatment B
Control Level 1 Cells 1, 3 Cells 5, 7
Level 2 Cells 2, 4 Cells 6, 8

Similarly, the researcher can determine other first order interactions analysis of the first order
interaction, here essentially a simple factorial analysis as only two variables are considered at a
time and the remaining on eis ignored.
3 Bibliography
JAYARAMAN, K. (n.d.). A Statistical Manual For Forestry Research. Retrieved from
http://www.fao.org/docrep/003/x6831e/X6831E00.htm#TOC
Lentner, M., & Bishop, T. (1993). Experimental design and analysis (Second ed.). Valley Book
Company.
McColl's, V. J. (1997). Statistics Glossary v1.1.
Software, N. S. (n.d.). 263-1. Retrieved from https://ncss-wpengine.netdna-ssl.com/wp-
content/themes/ncss/pdf/Procedures/NCSS/Latin_Square_Designs.pdf
Statistics, N. e.-H. (2008, OCt 4). NIST/SEMATECH e-Handbook of Statistical Methods.
Retrieved from https://www.itl.nist.gov/div898/handbook/cd.htm
University, T. P. (n.d.). Analysis of Variance and Design of Experiments. Retrieved from
https://onlinecourses.science.psu.edu/stat502/node/175/
University, W. S. (2000, August 16 ). Retrieved from
http://www.tfrec.wsu.edu/ANOVA/Latin.html