You are on page 1of 26

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License.

Your use of this


material constitutes acceptance of that license and the conditions of use of materials on this site.

Copyright 2007, The Johns Hopkins University and Lynda Burton. All rights reserved. Use of these materials
permitted only in accordance with license rights granted. Materials provided AS IS; no representations or
warranties provided. User assumes all responsibility for use, and all liability related thereto, and must independently
review all materials for accuracy and efficacy. May contain materials owned by others. User is responsible for
obtaining permissions for use from third parties as needed.

Survey Research: Choice of Instrument, Sample


Lynda Burton, ScD
Johns Hopkins University

Section A
Sample

Survey Research Methods

What makes a survey a survey?


Scientific methodology
Data collection from an individual
Usually samples from a large population
Conducted for the purpose of . . .
X Description
X Exploration
X Explanation

Characteristics of Good Survey Research

Quantitative
Self-monitoring
Contemporary
Replicable

Systematic
Impartial
Representative
Theory-based

Types of Surveys

Cross sectional
Longitudinal
Trend
Time cohort
Panel

General Sampling Issues

Basic ruleall individuals must have equal chance of being


selected
May be more accurate data than a census
If all members of a population were identical, sampling would
not be necessary
Aim for a sample that is generalizable to total population of
interest

Some Sampling Definitions

Elementunit from which data are collected


Universeaggregation of all elements
Survey populationpopulation from which you are drawing
sample
Sampling frameactual list or hat from which you select
elements

Probability Sampling Approaches

Simple random sample


Random number generator
Draw number out of a hat

Probability Sampling Approaches

Systematic sample
Every element of list
Select every kth element
Make sure list isnt periodic

10

Probability Sampling Approaches

Stratified sample
Same as for simple random sample
However, select from within specific pre-determined
groupings
Insures heterogeneity

11

Probability Sampling Approaches

Multi-stage sample
Select cluster of elements first, for example, physicians
Then select elements, for example, patients

12

Non-Probability Sampling Approaches

Purposive or judgmental
Educated guess of representative unit

13

Non-Probability Sampling Approaches

Purposive or judgmental
Educated guess of representative unit
Quota sampling
Select any way you want following a pre-set quota
pattern
Available subject sampling
Use whomever is available

14

Section B: Sample Size Considerations


Based on chapter seven of Designing and Conducting Health
Surveys (2nd. ed.); Aday, LuAnn. San Francisco: Jossey-Bass
Publishers

Key Concepts in Determining Sample Size

From a sampling point of view, the key concern is having a


large enough number of cases to minimize the variable
sampling (standard) error in the estimates
Sampling error can be reduced by increasing the sample size
(denominator) or minimizing the random errors in the data
collection process

16

Normal Sampling Distribution

17

What Differences Do You Expect?

In experimental designs, there is a particular interest in


determining the effect (or difference) between experimental
and control groups
The effect size, which essentially reflects the hypothesized
difference between groups, provides a basis for calculating
the sample size for these types of designs

18

Type I and II Errors

A Type I error results from falsely rejecting the null hypothesis


when the hypothesis is actually true (alpha)
A Type II error refers to the reverse errorfailing to reject the
null hypothesis when it is actually false
The probability of Type I and Type II errors decreases as the
sample size increases, primarily because the estimates
obtained from larger samples are more reliable (have less
random sampling variation)

19

Criteria for Estimating Sample Size Based on the Study


Design: Aday

Objectiveto test a hypothesis


Frameworkpower analysis
Steps
1. Identify the major study hypotheses
2. Determine the statistical tests for the study hypotheses,
such as a t-test, F-test, or chi-square

20

Criteria for Estimating Sample Size Based on the Study


Design: Aday

Steps
3. Select the population or subgroups of interest (based on
study hypotheses and design)
4. Step 4
a. Indicate what you expect the hypothesized
difference to be
b. Estimate the standard deviation of the difference

21

Criteria for Estimating Sample Size Based on the Study


Design: Aday

Steps
4. Step 4
c. Compute the effect size
5. Decide on a tolerable level of error in rejecting the null
hypothesis when it is true (alpha) (this is usually set at
.05.)

22

Criteria for Estimating Sample Size Based on the Study


Design: Aday

Steps
6. Decide on a desired level of power for rejecting the null
hypothesis when it is false (power) (this is usually set at
.80.)
7. Compute sample size, based on study assumption

23

Example of a Major Study Hypothesis

Proportion of patients (improving in health [by a global


health index measure] and who received the treatment) will
not differ from the proportion who improve and did not
receive the treatment (null hypothesis)
Ho P1 - P2 = 0
Alternative hypothesis
Ha P1 - P2 ne 0

24

Determine Statistical Test

Calculation of sample size depends on the study design and


what statistical test you will be using to test the hypotheses
It is possible that you will have several outcomes, each of
which will be determined using a different statistical test
Different sample sizes will be required, depending on the
outcome

25

Weighting the Sample Data to Reflect the Population

You may want to sample some sub groups in the population


at different rates to ensure that there will be enough of these
individuals without having to increase the overall size of the
sample
The sample can be weighted so that it resembles the
population from which it was drawn
Weighting literally involves a process of statistically assigning
more, or less, weight to some groups than others so that their
distribution in the sample corresponds more closely to their
actual distributions in the population as a whole

26

You might also like