You are on page 1of 49

Chapter Seven Causal Research Design: Experimentation

7-2

Concept of Causality
A statement such as "X causes Y " will have the following meaning to an ordinary person and to a scientist.
____________________________________________________ Ordinary Meaning Scientific Meaning ____________________________________________________ X is the only cause of Y. X is only one of a number of possible causes of Y.

X must always lead to Y (X is a deterministic cause of Y).


It is possible to prove that X is a cause of Y.

The occurrence of X makes the occurrence of Y more probable (X is a probabilistic cause of Y). We can never prove that X is a cause of Y. At best, we can infer that X is a cause of Y.

Types of Evidence That Supports a Causal Inference

7-3

Concomitant variation is the extent to which a cause, X, and an effect, Y, occur together or vary together in the way predicted by the hypothesis under consideration. The Time order of occurrence condition states that the causing event must occur either before or simultaneously with the effect; it cannot occur afterwards e.g. in-store service & sales The Absence of other possible causal factors means that the factor or variable being investigated should be the only possible causal explanation e.g pricing, advertising, quality, competition etc. Xthe presumed cause (independent variable) Ythe presumed effect (dependent variable)

Evidence of Concomitant Variation between Purchase of Fashion Clothing influenced by Education


Purchase of Fashion Clothing, Y High Education, X High 363 (73%) Low 137 (27%) 500 (100%)

7-4

Low

322 (64%)

178 (36%)

500 (100%)

Evidence of Concomitant Variation between Purchase of Fashion Clothing influenced by Education

7-5

Based on evidence can we say high education causes high purchase of fashion clothing? Certainly not! We can only say that the association makes the hypothesis more tenable, it does not prove it There may be other causal factors e.g. income Next example we have kept effect of income constant We see that difference between high & low education respondents reduced considerably Evidence of concomitant variation, time order of occurrence, elimination of other causal factors even if combined still do not demonstrate conclusively that a causal relationship exists However if all evidence is strong & consistent, it may be reasonable to conclude that there is a causal relationship

Purchase of Fashion Clothing By Income and Education

7-6

Low Income Purchase High Education High 122 (61%) Low 200 (100%) 300 (100%) Education 78 (39%) High

High Income Purchase High 241 (80%) Low 59 (20%) 300 200

Low

171 (57%)

129 (43%)

Low

151 (76%)

49 (24%)

7-7

Definitions and Concepts

Independent variables or treatments are variables or alternatives that are manipulated and whose effects are measured and compared e.g. price levels. Test units are individuals, organizations, or other entities whose response to the independent variables or treatments is being examined, e.g. consumers or stores. Dependent variables are the variables which measure the effect of the independent variables on the test units, e.g. sales, profits, and market shares. Extraneous variables are all variables other than the independent variables that affect the response of the test units, e.g. store size, store location, and competitive effort.

7-8

Experiment

A research investigation in which conditions are controlled One independent variable is manipulated Its effect on a dependent variable is measured To test a hypothesis

7-9

Types of Experiments
Laboratory Experiment
Research investigation in which Investigator creates a situation with exact conditions so as to control some, and manipulate other, variables

Experiment
Scientific investigation in which an investigator manipulates and controls one or more independent variables and observes the degree to which the dependent variable or variables change

Field Experiment Research study in a realistic situation in which one or more independent variables are manipulated by the experimenter under as carefully controlled conditions as the situation will permit

7-10

Laboratory Experiment

Field Experiment

Artificial-Low Realism Few Extraneous Variables High control Low Cost Short Duration Subjects Aware of Participation

Natural-High Realism Many Extraneous Variables Low control High Cost Long Duration Subjects Unaware of Participation

7-11

Basic Issues of Experimental Design

Manipulation of the Independent Variable Selection of Dependent Variable Assignment of Subjects (or other Test Units) Control Over Extraneous Variables

7-12

Independent Variable: The experimenter has some degree of control over the independent variable. The variable is independent because its value can be manipulated by the experimenter to whatever he or she wishes it to be.

7-13

Experiment Treatment

Alternative manipulations of the independent variable being investigated

7-14

Dependent Variable

Its value is expected to be dependent on the experimenters manipulation Criterion or standard by which the results are judged e.g... sales volume,
awareness, recall etc

7-15

Test Units

Subjects or entities whose response to the experimental treatment are measured or observed.

7-16

Symbols

X = independent variable, treatment, or event, the effects of which are to be determined O = observation or measurement of the dependent variable on the test units R = the random assignment of test units to separate treatments Movement from left to right indicates movement through time E.g. X O1 O2 Means test units were exposed to treatment/independent variable (X) & response was measured at two different pts of time O1 & O2 Horizontal alignment of symbols implies that all those symbols refer to a specific treatment group Vertical alignment of symbols implies that those symbols refer to activities or events that occur simultaneously E.g. R X1 O1 R X2 O2 Means that two groups of test units were randomly assigned to two different treatment groups at the same time & the dependent variable was measured in the two groups simultaneously

7-17

Validity in Experimentation

Two goals in experimentation: Draw valid conclusions about the effects of independent variables on the study group (internal validity) Make generalizations to a larger population of interest (external validity)

7-18

Validity in Experimentation

Internal validity refers to whether the manipulation of the independent variables or treatments actually caused the observed effects on the dependent variables. Control of extraneous variables is a necessary condition for establishing internal validity. External validity refers to whether the causeand-effect relationships found in the experiment can be generalized. To what populations, settings, times, independent variables and dependent variables can the results be projected?

7-19

Factors Influencing Internal Validity


History Maturation Testing Instrumentation Selection Mortality

Type of Extraneous Variable

7-20

History (H)- Specific events that are external to the experiment but occur at the same time as the experiment & may affect the dependent variable e.g. A major employer closes its plant in test market area Maturation (MA)- Subjects/Test units change during the course of the experiment e.g. Subjects become tired, stores change in terms of layout, dcor, composition etc Testing - The Before measure alerts or sensitizes subject to nature of experiment or second measure. - Main testing effect (MT), when a prior observation affects a latter observation e.g. Measure effect of advt on attitudes towards a brand, respondents given a questionnaire measuring background info & attitude, then exposed to ad & again asked qs, may be no change in their attitude as they may try to maintain consistency or may show change just because it is being measured - Interactive testing effect (IT), a prior measurement affects the test units response to the independent variable e.g. When people are asked to indicate their attitudes towards a brand they become sensitized /aware of the brand & pay more attention to the test commercial than they normally would

Type of Extraneous Variable

7-21

Instrument (I)- Changes in instrument result in response bias e.g. in advt experiment if newly designed questionnaire is used to measure post treatment attitudes. Statistical Regression(SR) when test units with extreme scores move closer to the average score during the experiment e.g. in advt experiment if some respondents had either very favorable of very unfavorable attitudes, post treatment their attitudes might have moved towards average. People with extreme attitudes have more room for change, so variation more likely Selection Bias(SB) improper assignment of test units to treatment conditions e.g. different store sizes Mortality(MO)- loss of test units while the experiment is in progress

What are the Different Basic Experimental Designs?

7-23

A Classification of Experimental Designs

Pre-experimental designs do not employ randomization procedures to control for extraneous factors: the one-shot case study, the one-group pretest-posttest design, and the static-group. In true experimental designs, the researcher can randomly assign test units to experimental groups and treatments to experimental groups: the pretest-posttest control group design, the posttestonly control group design, and the Solomon fourgroup design.

7-24

A Classification of Experimental Designs

Quasi-experimental designs result when the researcher is unable to achieve full manipulation of scheduling or allocation of treatments to test units but can still apply part of the apparatus of true experimentation: time series and multiple time series designs. A statistical design is a series of basic experiments that allows for statistical control and analysis of external variables: randomized block design, Latin square design, and factorial designs.

7-25

A Classification of Experimental Designs


Experimental Designs

Pre-experimental One-Shot Case Study One Group Pretest-Posttest Static Group

True Experimental Pretest-Posttest Control Group Posttest: Only Control Group Solomon FourGroup

Quasi Experimental Time Series Multiple Time Series

Statistical Randomized Blocks Latin Square Factorial Design

Pre-experimental designs One-Shot Case Study


Measure effectiveness of a test commercial (X)for a deptt. store X 01

7-26

A single group of test units is exposed to a treatment X. A single measurement on the dependent variable is taken (01). There is no random assignment of test units. The one-shot case study is more appropriate for exploratory than for conclusive research. E.g. measure the effectiveness of a test commercial for a brand Test commercial is X, Dependent variable(Os) are unaided & aided recall of the ad

Pre-experimental designs One-Group Pretest-Posttest Design

7-27

01 X

A group of test units is measured twice. There is no control group. First a pre-treatment measure is taken(O1), then group is exposed to treatment (X) Finally a post-treatment measure is taken (O2), e.g. sales after the ad The treatment effect is computed as 02 01. The validity of this conclusion is questionable since extraneous variables are largely uncontrolled.

02

7-28

One-Group Pretest-Posttest Design: Example

Measure effectiveness of a test commercial for a deptt. store Respondents first administered a personal interview to measure attitudes towards the store (O1) Then they watch the test commercial (X) After that they are again administered a personal interview to measure attitudes towards the store (O2) Effectiveness of the commercial is measured as O2 - O1

Pre-experimental designs Static Group Design EG: CG:


7-29

01 02

A two-group design. The experimental group (EG) is exposed to the treatment, and the control group (CG) is not. Measurements on both groups are made only after the treatment. Test units are not assigned at random. The treatment effect would be measured as 01 - 02.

7-30

Static Group Design: Example

To measure the effectiveness of a test commercial for a deptt. Store Two groups of respondents would be recruited on the basis of convenience Only the experimental group would be exposed to the test commercial Then attitudes towards the store of both experimental & control group would be measured Effectiveness would be measured as O1 O2

True Experimental Designs: Pretest-Posttest Control Group Design

7-31

Distinguishing feature of this design compared to pre-experimental design is randomization Here the researcher randomly assigns test units to experimental groups The pretest-posttest control group design, the posttest-only control group design, and the Solomon four-group design.

True Experimental Designs: Pretest-Posttest Control Group Design

7-32

EG: R 01 X 02 CG: R 03 04 Test units are randomly assigned to either the experimental (exposed to test ad) or the control group (not exposed). A pretreatment measure (attitude towards a brand) is taken on each group. Only the experimental group is exposed to the treatment but post-test measures are taken on both. The treatment effect (TE) is measured as:(02 - 01) - (04 03). Selection bias is eliminated by randomization.

7-33

Posttest-Only Control Group Design


EG : CG :

R R

01 02

Does not involve any pre measurement Sample randomly split, half in EG (shown the advt.) & other half in CG (not shown), then questionnaire administered to both to obtain posttest measures on attitudes towards the brand, difference in attitudes of the EG & CG used as a measure of the effectiveness of the advt. The treatment effect is obtained by TE = 01 - 02 Except for pre-measurement, the implementation of this design is very similar to that of the pretest-posttest control group design. Simple to implement

7-34

Solomon Four Group Design

When we need to examine changes in attitudes of respondents then this design is used Expensive & time consuming hence not popular

Experimental Group 1: Control Group 1: Experimental Group 2: Control Group 2:

R O1 X R O3 R X R

O2 O4 O5 O6

7-35

Quasi-Experimental Designs
There is no randomization of test units to treatments: Researcher can control when measurements are taken & on whom Researcher lacks control over the scheduling of the treatments & also is unable to expose test units to treatments randomly Quicker & less expensive Time series & Multiple time series designs

7-36

Time series & Multiple Time Series Design


Time Series Design: series of periodic measurements on the dependent variable for a group of test units

01 02 03 04 05

X 06 07 08 09 010

Multiple Time Series Design: similar to time series design except that another group of test units is added to serve as a control group EG : 01 02 03 04 05 X 06 07 08 09 010 CG : 01 02 03 04 05 06 07 08 09 010

If the control group is carefully selected, this design can be an improvement over the simple time series experiment. Can test the treatment effect twice: against the pretreatment measurements in the experimental group and against the control group.

7-37

Multiple Time Series Design: Example


Examine the buildup effect of increased advertising One group of households was the EG & equivalent group was CG Groups were matched on demographic variables Data collected for 76 weeks Both recd same level of advt for first 52 weeks for the brand in qs Next 24 weeks, experimental group (EG) exposed to twice as much advt as the Control group (CG) Results indicated that buildup effect of advt was immediate with a duration of the order of the purchase cycle Useful for selecting advt timing patterns

7-38

Statistical Designs
Statistical designs consist of a series of basic experiments that allow for statistical control and analysis of external variables and offer the following advantages:

The effects of more than one independent variable can be measured. Specific extraneous variables can be statistically controlled. Economical designs can be formulated when each test unit is measured more than once.

The most common statistical designs are the randomized block design, the Latin square design, and the factorial design.

7-39

Randomized Block Design

Is useful when there is only one major external variable, such as store size or income, that might influence the dependent variable & needs to be blocked The test units are blocked, or grouped, on the basis of the external variable e.g. Impact of humor on the effectiveness of advertising for store Z- store patronage identified as the blocking variable The researcher must be able to identify & measure the blocking variable By blocking, the researcher ensures that the various experimental and control groups are matched closely on the external variable. Limitation is that researcher can control for only one external variable, when more than one variable must be controlled must use Latin square or factorial designs

7-40

Randomized Block Design


Treatment Groups Commercial Commercial Commercial A B C A A A A B B B B C C C C

Block Store Number Patronage 1 2 3 4 Heavy Medium Low None

7-41

Latin Square Design

Allows the researcher to statistically control two non interacting external variables as well as to manipulate the independent variable. E.g. wants to control two variables- store patronage & interest in store to measure impact of humor on the effectiveness of advertising for store Z Each external or blocking variable is divided into an equal number of blocks, or levels. The independent variable is also divided into the same number of levelsassignments of 3 test commercials can be made- A, B, C To implement this design store patronage would also have to be blocked at three rather than four levels (e.g. by combining the low & non patrons into a single block) A Latin square is conceptualized as a table with the rows and columns representing the blocks in the two external variables. The levels of the independent variable are assigned to the cells in the table. The assignment rule is that each level of the independent variable should appear only once in each row and each column

7-42

Latin Square Design


Interest in the Store Medium
A B C

Store Patronage Heavy Medium Low and none

High B C A

Low C A B

7-43

Factorial Design: Example


Used to measure the effects of two or more independent variables at various levels. Allows for interactions between variables- interaction takes place when the simultaneous effect of two or more variables is diff from the sum of their separate effects e.g. a person may prefer coffee & fav. temperature may be cold but does not mean he likes cold coffee In a two-factor design, each level of one variable represents a row and each level of another variable represents a column A cell for every possible combination of treatment variable Suppose in addition to examining effect of humor on advt, researcher was also interested in simultaneously examining the amount of store info in an advt & its impact- store info is also varied at 3 levels. This would require a 3x3 = 9 cells Thus 9 diff. commercials will be produced, each having a specific level of store info & amount of humor Respondents would be randomly selected & randomly assigned to the 9 cells & respondents in each cell would receive a specific treatment combination E.g. respondents in upper left hand corner will view an advt that had no humor & low store info

7-44

Factorial Design

Amount of Store Information


Low Medium High

Amount of Humor No Medium High Humor Humor Humor A D G B E H C F I

7-45

Laboratory versus Field Experiments


Factor
Environment Control Reactive Error Demand Artifacts Internal Validity External Validity Time Number of Units Ease of Implementation Cost

Laboratory
Artificial High High High High Low Short Small High Low

Field
Realistic Low Low Low Low High Long Large Low High

7-46

Limitations of Experimentation

Experiments can be time consuming, particularly if the researcher is interested in measuring the long-term effects. Experiments are often expensive. The requirements of experimental group, control group, and multiple measurements significantly add to the cost of research. Experiments can be difficult to administer. It may be impossible to control for the effects of the extraneous variables, particularly in a field environment. Competitors may deliberately contaminate the results of a field experiment.

7-47

Activities

You are a marketing research manager for the Coca-Cola Company. The company would like to determine whether it should increase, decrease, or maintain the current level of advertising dollars spent on Coke. Design a field experiment to address this issue. What potential difficulties do you see in conducting the experiment just described? What assistance would you require from the CocaCola management to overcome these difficulties?

7-48

Activities

A pro-life group wanted to test the effectiveness of an anti-abortion commercial. Two random samples, each of 250 respondents, were recruited in Atlanta. One group was shown the anti-abortion commercial. Then attitudes toward abortion were measured for respondents in both groups. A. Identify the independent & dependent variables in this experiment B. Design a field experiment to address this issue.

7-49

Fieldwork

Select two different advertisements of any brand Design and conduct an experiment to determine which ad is more effective. Use a student sample with 10 students being exposed to each ad (treatment condition). Develop your own measures of advertising effectiveness in this context

You might also like