Sampling Methods readings

An edited reading from http://www.statpac.com/surveys/sampling.htm Retrieved 29/04/2009 It is incumbent on the researcher to clearly define the target population. There are no strict rules to follow, and the researcher must rely on logic and judgment. The population is defined in keeping with the objectives of the study. Sometimes, the entire population will be sufficiently small, and the researcher can include the entire population in the study. This type of research is called a census study because data is gathered on every member of the population. Usually, the population is too large for the researcher to attempt to survey all of its members. A small, but carefully chosen sample can be used to represent the population. The sample reflects the characteristics of the population from which it is drawn. Sampling methods are classified as either probability or nonprobability. In probability samples, each member of the population has a known nonzero probability of being selected. Probability methods include random sampling, systematic sampling, and stratified sampling. In nonprobability or purposive sampling, members are selected from the population in some nonrandom manner. These include convenience sampling, judgment sampling, quota sampling, and snowball sampling. The advantage of probability sampling is that sampling error can be calculated. Sampling error is the degree to which a sample might differ from the population. When inferring to the population, results are reported plus or minus the sampling error. In nonprobability or purposive sampling, the degree to which the sample differs from the population remains unknown. PROBABILITY SAMPLING The simplest form of random sampling is called simple random sampling. In this we select participants from a given population such that each person in the population has an equal chance of being selected. If number of participants is large (1000) true random selection will produce a participant group with very similar demographic features to the total population from which it is selected. A randomly selected sample of 1000 will have no more than a 5% margin of error when used to predict categorical characteristics (35% Labour, 45% National. 10% Greens, 10% dont know) of a population. Systematic sampling is often used instead of random sampling. It is also called an Nth name selection technique. After the required sample size has been calculated, every Nth record is selected from a list of population members. As long as the list does not contain any hidden order, this sampling method is as good as the random sampling method. Its only advantage over the random sampling technique is simplicity. Systematic sampling is frequently used to select a specified number of records from a computer file. Stratified Random Sampling involves dividing your population into homogeneous subgroups based on one factor and then taking a simple random sample in each subgroup. So, for example, you might want to compare the perceptions of Maori and Samoans using a questionnaire. You would then randomly select participants with each group. Or your subgroups might be based on culture (two), gender and age (two groups) which would give 8 (2x2x2) equal subgroups within which you randomly select participants. Quota Sample In this case also, the entire population is first divided into homogeneous subgroups with respect to the given characteristic such as culture. Then, you recruit a specified number of people from each strata as you come across them rather than selecting them through random procedure. The resulting samples are called quota samples.

PURPOSIVE SAMPLING From Patton, M. Q. (1990). Qualitative evaluation and research methods

(2nd ed.). Newbury Park, CA: Sage Publications

Purposive sampling starts with a purpose in mind and the sample is thus selected to include people of interest and exclude those who do not suit the purpose. Subjects are selected because of some characteristic. Patton (1990) has proposed the following cases of purposive sampling. Purposive sampling is popular in qualitative research. Type Random Sampling Definition A systematic process of selecting subjects or units for examination and analysis that does not take contextual or local features into account. A process of selecting subjects or units for examination and analysis that is based on accessibility, ease, speed, and low cost. Units are not purposefully or strategically selected. Purposefully picking a wide range of variation on dimensions of interest documents unique or diverse variations that have emerged in adapting to different conditions. Identifies important common patterns that cut across variations. The process of selecting a small homogeneous group of subjects or units for examination and analysis. The process of selecting a small number of important cases cases that are likely to "yield the most information and have the greatest impact on the development of knowledge" (Patton, 2001, p. 236). The iterative process of selecting "incidents, slices of life, time periods, or people on the basis of their potential manifestation or representation of important theoretical constructs" Definition When is it used? Random sampling is typically used in experimental and quasiexperimental designs. Random sampling typically involves the generation of large samples. Random sampling is used when researchers want their findings to be representative of some larger population to which findings can be generalized This is the least desirable sampling method, and researchers should typically avoid using it. There is no randomness and the likelihood of bias is high. You can't draw any meaningful conclusions from the results you obtain. Saves time, money, and effort. Poorest rational; lowest credibility. Yields informationpoor cases Often, researchers want to understand how a phenomenon is seen and understood among different people, in different settings and at different times. When using a maximum variation sampling method the researcher selects a small number of units or cases that maximize the diversity relevant to the research question

Convenience Sampling

Maximum Variation Sampling

Homogenous Sampling Critical Case Sampling

Homogeneous sampling is used when the goal of the research is to understand and describe a particular group in depth This is a good method to use when funds are limited. Although sampling for one or more critical cases may not yield findings that are broadly generalisible they may allow researchers to develop logical generalizations from the rich evidence produced when studying a few cases in depth. Permits logical generalization and maximum application of information to other similar cases because if it's true of this once case it's likely to be true of all other cases This method is best used when the research focuses on theory and concept development and the research team's goal is to develop theory and concepts that are connect to, grounded in or emergent from real life events and circumstances. Theoretical sampling is an important component in the development of grounded theories. This sampling approach has the goal of developing a rich understanding of the dimensions of a concept across a range of settings and conditions. When is it used?

Theorybased or Theoretical Sampling

Type

Type Random Sampling

Definition A systematic process of selecting subjects or units for examination and analysis that does not take contextual or local features into account. Identification of confirming and disconfirming case occurs after some portion of data collection and analysis has already been completed

Confirming and disconfirming cases

When is it used? Random sampling is typically used in experimental and quasiexperimental designs. Random sampling typically involves the generation of large samples. Random sampling is used when researchers want their findings to be representative of some larger population to which findings can be generalized Identifying confirming and disconfirming cases is a sampling strategy that occurs within the context of and in conjunction with other sampling strategies. Researchers seek out confirming and disconfirming cases (that do not fit emergent patterns and allow the research team to evaluate rival explanations) in order to develop a richer, more in depth understanding of a phenomenon and to lend credibility to one's research account. Identifying extreme or deviant cases is a sampling strategy that occurs within the context of and in conjunction with other sampling strategies. The process of identifying extreme or deviant cases occurs after some portion of data collection and analysis has been completed. Researchers seek out extreme or deviant cases in order to develop a richer, more indepth understanding of a phenomenon and to lend credibility to one's research account.

Extreme or deviant cases

Typical cases

Intensity sampling

Politically important cases Purposeful Random Sampling

The process of selecting or searching for highly unusual cases of the phenomenon of interest or cases that are considered outliers, or those cases that, on the surface, appear to be the 'exception to the rule' that is emerging from the analysis. E.g. such as outstanding success/notable failures, top of the class/dropouts, exotic events, crises. The process of selecting or searching for cases that are not in anyway atypical, extreme, deviant or unusual. The process of selecting or searching for rich or excellent examples of the phenomenon of interest. These are not, however, extreme or deviant cases e.g. such as good students/poor students, above average/below average. The process of selecting or searching for a politically sensitive site or unit of for analysis The process of identifying a population of interest and developing a systematic way of selecting cases that is not based on advanced knowledge of how the outcomes would appear. The purpose is to increase credibility not to foster representativeness

Identifying typical cases can help a researcher identify and understand the key aspects of a phenomenon as they are manifest under ordinary circumstances. Providing a case summary of a typical case can be helpful to those not famililar with a culture or social setting. Intensity sampling can allow the researcher to select a small number of rich cases that provide in depth information and knowledge of a phenomenon of interest. Intensity sampling requires prior information and exploratory work to be able to identify intense examples. One might use intensity sampling in conjunction with other sampling methods. For example, one may collect 50 cases and then select a subset of intense cases for more in depth analysis. Attracts attention to the study (or avoids attracting undesired attention by purposefully eliminating from the sample politically sensitive cases). The use of a randomized sampling strategy, even when identifying a small sample, can increase credibility. It adds credibility to sample when potential purposeful sample is larger than one can handle. Reduces judgment within a purposeful category. (Not for generalizations or representativeness.)

Type Stratified Purposeful or Quota Sampling

Definition Purposeful samples are stratified or nested by selecting particular units or cases that vary according to a key dimension. For example, researching primary care practices, we stratify this purposeful sample by practice size (small, medium and large) and practice setting (urban, suburban and rural).

When is it used? A stratified purposeful sampling approach is often used in market research. Interviewers are required to find cases with particular characteristics. They are given quota of particular types of people to interview and the quota are organised so that final sample should be representative of population. So, for example if we want our sample to represent the age of our population and 20% are between 20 and 30, and sample is to be 50 then 10 of sample (20%) must be in this age group. Complex quotas can be developed so that several characteristics (e.g. age, sex, marital status) are used simultaneously. When enough information is known to identify characteristics that may influence how the phenonmenon is manifest, then it may make sense to use a stratified purposeful sampling approach A disadvantage of quota sampling interviewers choose who they like (within above criteria) and may therefore select those who are easiest to interview, so bias can result. Also, it is impossible to estimate accuracy because this is not a random sample. Criterion sampling can be useful for identifying and understanding cases that are information rich. Criterion sampling can provide an important qualiative component to quantitative data. Criterion sampling can be useful for identifying cases from a standardized questionnaire that might be useful for followup.

Criterion Sampling

Opportunistic or emergent sampling

Snowball or chain sampling

Criterion sampling involves selecting cases that meet some predetermined criterion of importance. For example, we will only interview students whose Sequals are less than 3. This occurs when the researcher makes sampling decisions during the process of collecting data. This commonly occurs in field research. As the observer gains more knowledge of a setting, he or she can make sampling decisions that take advantage of unexpected events as they unfold. Snowball or chain sampling involves using well informed people to identify critical cases informants who have a great deal of information about a phenomenon. The researcher follows this chain of contacts in order to identify and accumulate critical cases.

A flexible research and sampling design is an important feature of qualitative research, particularly when the research being conducted is exploratory in nature. When little is known about a phenomenon or setting, a priori sampling decisions can be difficult. In such circumstances, creating a research design that is flexible enough to foster reflection, preliminary analysis, and opportunistic or emergent sampling may be a good idea. This method can be useful for identifying a small number of key cases that are identified by a number of key or expert informants as important cases or exemplars. Identifies cases of interest from people who know people who know people who know what cases are informationrich, that is, good examples for study, good interview subjects.

Probability and Purposive Sampling Techniques

From Kemper, E.A., Stringfield, S. & Teddlie, C. (2003) Mixed Methods Sampling Strategies in Social Science Research in Tashakkori, A. & Teddlie, C. (Eds. 2003) Handbook of Mixed Methods in Social and Behavioural Research, Sage, Thousand Oaks. Chapter Data Collection Strategies in Mixed Methods Research Chapter 10 p273278. Probability Sampling Techniques Simple random sampling Systematic random sampling Stratified random sampling Proportional Nonproportional Cluster random sampling Purposive Sampling Techniques Convenience sampling Extreme/deviant case sampling Confirming/disconfirming cases and typical case sampling Homogeneous case sampling Stratified purposive sampling and random purposive sampling Opportunistic and snowball sampling Additional purposive sampling techniques are described by Patton (1990) and include intensity sampling, maximum variation sampling, critical case sampling, criterion sampling, operational construct sampling, and I sampling politically important cases.

Probability Sampling
One of the goals of research that uses probability sampling techniques is generalizabilitly (external validity) or the ability to extrapolate findings from a subset of a population or particular setting to a larger defined population of people (population validity) or settings (ecological validity). This ability rests on three assumptions: (a) that the number of selected units is large enough for the random errors to cancel each other out (Tashakkori & Teddlie, 1998); (b) that the distribution of the population under study is normal, or at least knowable, so that sampling can proceed in accordance with that distribution; and (c) that a sample can and will be drawn that is large enough to plausibly produce a reasonable estimate of the population at large. The violation of any of these assumptions will severely limit the usefulness of inferences made about the population at large. The following subsections discuss the main approaches to probability sampling that, although typically associated with more quantitative research, could be applicable to qualitative studies and clearly have relevance to mixed methods studies (see Cells 1,2, and 3 in Table 10.2 later in the chapter). Simple Random Sampling. A simple random sample is one is which each person or unit in the clearly defined population has an equal chance of being included in the sample. To select a simple random sample, the technique can be as straightforward as drawing names out of a hat or as complex as using a computerized table of random numbers to draw a sample from the population at large. The advantage of this method is that the research data can be generalized from the sample to the entire population within a computable margin of error. A disadvantage of this method is that the selected units might be spread over a large geographic area, making them very difficult to reach. An example would be selecting 300 students randomly from the population of 10thgrade students in Atlanta, Georgia. Systematic Random Sampling. Systematic random sampling techniques involve selecting every nth unit of the target population from a randomly ordered list of the population. This method should be used only if the researcher is certain that the population list is not ordered in such a way that every nth unit results in a sample that is different from the population at large. For example, if every 10th person on a list is a group leader, then either the list should be reordered or a simple random sampling technique should be used. The advantage of employing a systematic sampling technique is that it is slightly easier to use than a simple random sampling technique; however, each member of the sample is not chosen independently beca use as soon as the first member is selected, the other members of the sample are determined as well.

Stratified Random Sampling (proportional and nonproportional). A stratified random sample is obtained by separating the population elements into groups, or strata, such that each element belongs to a single stratum. The research team then independently selects a random sample from each stratum. In proportional stratified random sampling, the proportion of the units randomly selected from each stratum is the same as the proportion in the target population. In nonproportional stratified sampling, the number of units selected from each stratum is typically the same, regardless of the relative proportion in the target population (see Box 10.1). BOX 10.1 Nonproportional Stratified Random Sampling Capraro, Capraro, and Wiggins (2000) conducted a study to investigate the effects of gender, socioeconomic status, race, and grades on standardized test scores. Using the National Educational Longitudinal Study of Eight Graders (NELS 88, which consists of 1,500 students), a random sample of 180 students consisting of 30 Black males, 30 Black females, 30 White males, 30 White females, 30 Hispanic males, and 30 Hispanic females was generated. This sampling logic provided the research team with equal statistical power in examining effects on three ethnic groups by two genders. Given the questions in the study, the researchers were able to sample in a way that ignored the fact that the larger population of American youths is not, in fact, so neatly divided. Cluster Sampling and Multistage Cluster Sampling. Cluster sampling is most appropriate when the sampling unit is not an individual but rather a group that is naturally occurring in the population such as neighborhoods, city blocks, schools, classrooms, or families. Instead of taking a random simple within each group (as in stratified random sampling), in cluster sampling a random sample of groups within the population is first performed. Multistage cluster sampling allows the sample to be further reduced by selecting a random sample from the selected cluster. Several sampling stages might be carried out to select the participants ultimately chosen to participate in the sample (see Box 10.2).

Purposive (Nonprobability) Sampling

Purposive or nonprobability samples are samples in which the researcher uses some criterion or purpose to replace the principle of canceled random errors. A list of these purposive sampling techniques was presented in Table 10.1 (and the note associated with that table). Several of these techniques are discussed in more detail in the rest of this section. BOX 10.2 Multistage Cluster Sampling Vega, Alderete, Kolody, and AguilarGaxioler (1998) used a multistage cluster sampling method in a study examining illicit drug use among Mexicans and Mexican Americans in California. The target population was Mexicans and Mexican Americans between 18 and 59 years of age in Fresno County, California. The county population is approximately 764,800, with 463,600 located in the FresnoClovis metropolitan area; of this, Hispanics constitute 38.3% of the total county population. From this population, 3,012 people were to be selected, stratified by urbanicity. During the first stage of the sampling process, the sampling units within each stratum (urban, rural, and town) was determined based on census blocks or block aggregates. A total of 200 sampling units within each stratum were selected with a probability proportionately based on the size of their Hispanic population. During the second stage, a quota of five households was randomly selected from each sampling unit. During the third and final stage, 1 person from each household was randomly selected to complete the survey. This resulted in a final sample of 3012 respondents (1,006 urban respondents, 1,006 town respondents, and 1,000 rural respondents). "The logic and power of purposive sampling lies in selecting informationrich cases for study in depth" (Patton, 1990, p. 169), with an underlying focus on intentionally selecting specific cases that will provide the most information for the questions under study. Researchers using random sampling techniques often seek to maximize the sample size so as to increase the probability of making accurate generalizations from the data. In contrast, researchers using purposive techniques seek to focus and, where practical, minimize the sample size, generally in nonrandom ways, so as to select only those cases that might best illuminate

and test the hypothesis of the research team. Although purposive sampling techniques are commonly associated with qualitative methods, purposive sampling can be used within studies with either a qualitative or a quantitative orientation and are quite common in mixed methods studies (see Cells 4, 5, and 6 in Table 10.2 later in the chapter). Several of the most commonly used purposive sampling techniques are discussed in the following subsections Convenience Sampling. Convenience sampling, which involves drawing elements from a group (usually most appropriately regarded as a subpopulation) that is easily accessible by the researcher is one of the most commonly used purposive sampling techniques. Due to the researcher focusing on an easily accessible or volunteer population, these elements are often not the most appropriate to answer the research question under study. Although convenience samples offer relatively low cost and ease of access, such sampling very often results in spurious conclusions and hence is rarely the sampling method of preference (see Box 10.3). Extreme/Deviant Case and Typical Case Sampling Techniques. Extreme/deviant case sampling and typical case sampling are similar in that both techniques are designed to find cases that best illuminate the research question at hand. Extreme/ deviant case sampling involves seeking out the most outstanding cases, or the most extreme successes and/or failures, so as to learn as much as possible about the outliers. Typical case sampling seeks those cases that are the most average or representative of the question under study. The Special Strategies studies (Stringfield et al., 1997) were mixed methods research projects conducted to provide detailed case studies of schools making exemplary use of Title I funds. The study design called for the identification of reform efforts that had been implemented in multiple sites and either had prior evidence of effectiveness or were being implemented very widely. (Note that by either definition, the sampling frame for the study would be restricted to atypical schools.) A total of 10 reform models were eventually chosen. Examples of selected reform models were Title I schoolwide projects, Success for All (Slavin & Madden, 2001), and the Coalition of Essential Schools (Sizer, 1992). BOX 10.3 Convenience Sampling The famous "Kinsey Report," actually titled Sexual Behavior in the Human Male (Kinsey, 1948), created a sensation when it was first released to the public. In this remarkable 804page study, Alfred Kinsey explicitly detailed results from his carefully detailed interviews of thousands of American males. The Kinsey Report appeared to demonstrate that, rather than being puritanical, the average American male of the 1940s was highly sexually active, often in quite colorful ways and with a wide range of partners. The study seemed to draw extraordinary strength from the fact that Kinsey himself had conducted hundreds of the interviews all across the country. However, as the science of sampling evolved over subsequent decades, scholars revisited Kinsey's methods and noted that he had obtained his sample through the device of taking a train into a new city and running an advertisement in the local newspapers, inviting males to be interviewed about their sexual activities. It does not require a great leap of imagination to conclude that the more than 99% of citizens in each city who did not volunteer to be interviewed were, on average, less sexually active (or, at least, with fewer partners) than the less than 1 % of citizens who, selfselected through answering a newspaper advertisement asking them to talk about their sex lives, volunteered for the study. In the end, while the Kinsey Report made a major contribution to getting a nation's people to talk about the sexual aspects of their existence, it made very little contribution to scholars' understanding of the sex lives of average Americans. In spite of the considerable fiscal investment, the sample of convenience seemed unlikely to be representative of the population about which Kinsey had attempted to generalize. For each model, a national search was undertaken to identify very strong implementations (the design sought "exemplary" sites, not "typical" ones). Design teams, state Title I directors, regional educational laboratories, and others were asked for nominations. A total of 25 final sites (purposefully stratified so that 12 were urban and 13 were suburban or rural) were selected only after they had received multiple nominations as being exemplary implementations of their respective reforms. The sampling scheme did not allow for statements about typical schools, or even typical implementations of the various reforms, but it did enhance the possibility that the research effort would identify and study strong implementations of the various reform efforts, as specified in the federal grant (see Box 10.4). ConfirminglDisconfirming Cases Sampling Techniques. The goal of using confirmingl disconfirming cases

sampling techniques is either to find specific sampling units that already fit into emerging patterns regarding the data (confirming cases) or to seek those cases that are exceptions or provide rival explanations to the emerging patterns (disconfirming cases) (see Box 10.5). Homogeneous Cases Sampling. Homogeneous cases sampling seeks to pick elements from a particular subgroup to study indepth. Studies that employ focus group interviews typically use this method because the goal is usually to gather opinions from people who are demographically, educationally, or professionally similar (see Box 10.6). BOX 10.4 Extreme Case Sampling Raywid (1999) conducted a case study using extreme case sampling on a high school to examine the components and qualities that combined to make the school successful. Over the past 10 years, the school selected, Central Park East Secondary in East Harlem, New York City, had received numerous awards, had been the subject of several research articles, and had a history of unusual educational success. This school was the ideal case to examine how a school can become successful when the odds are against success. Thus, the sampling unit in this case was the school, with the number of units in the study equal to one. Stratified Purposive (Quota) and Random Purposive Sampling Techniques. Stratified purposive sampling involves dividing the purposefully selected target population into strata (e.g., above average, average, below average) with the goal of discovering elements that are similar or different across the subgroups. Random purposive sampling involves taking a random sample of units in the purposefully selected target population. The logic behind using random sampling with purposive sampling is to add trustworthiness, and not generalizability, to the findings. Sample size in qualitative studies is typically too small to allow the results to be representative of an entire population, but randomization of the selection does offer an explanation as to why certain cases were excluded as well as a systematic method to make the sample manageable (see Box 10.7). BOX 10.5 Confirming Cases Shields and Foster (1989) used a confirming cases sampling methodology in their mixed method study exploring the emotional stereotyping of parents in childrearing manuals. Six separate and distinct time periods were identified with regard to sociocultural conditions, and the 54 books that fit the research criteria for the study were evaluated in terms of the era in which each manual was written and whether passages pertaining to parental emotion fit into the era in which the manual was written. BOX 10.6 Homogeneous Cases Sampling Silverstien, Auerback, Grieco, and Dunkel (1999) conducted a study of fathers who were members of the organization "Promise Keepers" to gather data about fathering experiences. A contact within the Promise Keepers group recruited members for three of the focus groups; another contact with the National Center for Fathering in Kansas City, Missouri, supplied the names of two members in the New York City area, who in turn recruited the members for the final focus group. The final sample consisted of 22 middleclass, primarily White suburban fathers, all of whom were members of the Promise Keepers organization. BOX 10.7 Stratified Purposive Sampling Ezzy, Bartos, de Visser, and Rosenthal (1998) used a stratified purposive sampling technique in their study of attitudinal, medical, and cultural correlates on the use of antiviral drugs by persons living with HIV or AIDS. A sample of 925 persons living with either AIDS or HIV was recruited through HIV/AIDS organizations, hospitals, doctors' offices, mailing lists of HIVrelated materials, and a targeted advertising campaign. The participants were stratified based on their HIV/AIDS status because one of the purposes of the study was to differentiate between the perceptions of those participants who had experienced an AIDSrelated illness and the perceptions of those who had not. Opportunistic and Snowball Sampling. Opportunistic and snowball sampling techniques both involve taking advantage of circumstances and events as

they arise while undergoing the data collection process. Opportunistic sampling involves taking opportunities as they come along and following up on leads as they arise within fieldwork. Snowball sampling involves using informants to identify cases that would be useful to include in the study. Both of these methods use insider knowledge to maximize the chance that the units included in the final sample are strong (highly appropriate) cases to include in the study. The Necessity of Mixed Methods Sampling Strategies The sampling techniques that have been discussed thus far have been presented in isolation. Many of the research topics under study in the social sciences are quite complex and require a combination of sampling techniques to adequately explore the phenomena of interest. As discussed elsewhere in this volume and in other texts, the use of mixed methods sampling strategies can greatly strengthen the research design of most studies in the social and behavioral sciences.