You are on page 1of 4

A Meta-Analysis Examining the Impact of Computer-Assisted

Instruction on Postsecondary Statistics Education: 40 Years of Research


Karen Larwin
Youngstown State University
David Larwin
Kent State University at Salem
JRTE | Vol. 43, No. 3, pp. 253278 | 2011 ISTE | iste.org

The research question(s):
In light of all of the conflicting research regarding CAI with these variables
as well as others, meta-analysis is the tool that can provide a general
measure of the impact of CAI on student achievement in statistics instruction
that might otherwise be obscured by these conflicting results. A metaanalytic
investigation is an appropriate and effective approach to synthesizing
and integrating the conflicting results from this quantitative research
(Cooper & Hedges, 2009; Johnson, 1989). Put more succinctly, the study was to show
whether or not students enrolled in postsecondary statistics courses benefit from CAI,
as evidenced by their achievement scores.

Data collection procedure:
Meta-analysis is like an aggregator, it compares and combines the conclusions of many studies.
The word meta is from the Greek, meaning about. In our current context, meta-analysis means
a study about other studies. In this case, many studies are brought together to form a
comprehensive assessment of the validity of a particular research question (Does Computer
Aided Instruction (CAI) improve comprehension of statistics students?). Many studies have
been performed exploring this question, this study brings all of those together to get an idea of
what the many studies conclude. The researchers gathered the studies in the following way:
1. First, we gathered and examined research studies. Studies included
in the meta-analysis must fit within the defined parameters selected for
analysis while representing as much of the population of data available
on the research area. Only studies addressing the primary research question
were considered.

Data analysis procedure :
Once the studies were gathered, the analysis portion begans. For this analysis, the researchers
used the following procedure:
1. Describe, classify, and code all the research studies to be included in the
meta-analysis. Once the pertinent research studies were gathered, they had to
be organized.
2. The analysis of the overall mean effect-size measures and the
mean effect-size measures for each research characteristic being examined.
Analyze the assembled, organized studies, for common research characteristics.
Also ,the studies were weighted based on sample size (assuming that a larger
sample size yields a more accurate result).

Validity and reliability (or trustworthiness) of the study:
In a quantitative study like this, validity and reliability are strictly a matter of numerical and
mathematical analysis. Based on research conclusions, a statistical construct is created to
compare and combine the results of the studies being analyzed.

How does the methodological approach serve the research purpose:
Meta-analysis is a valuable form of research analysis because the aggregation of many studies
has a tendency to correct errors or bias from any one particular study. Meta-analysis helps
avoid conclusions that may result from a single study. In this case, 71 different studies were
brought together to yield a comprehensive conclusion.

Citations:


The effectiveness of educational podcasts for teaching music and visual
arts in higher education-Focus Groups
Cheung On Tam*
Department of Cultural and Creative Arts, Hong Kong Institute of Education, Hong Kong
(Received 31 January 2011; final version received 11 May 2011)
The research question(s):
Research has been conducted to determine the effectiveness of podcasts in teaching
business, science, and languages. However, little or no research has been done to
investigate the effectiveness of podcasts in the instruction of music and visual arts. The
aims of this study were to investigate the effectiveness of using podcasts to learn music
and visual arts and to identify the characteristics of an effective educational podcast
from the perspective of students.

Data collection procedure:
An evaluation survey and focus-group interviews were used to collect students
feedback. The survey was administered to all students in the last session of each
course and three students from each course were randomly selected to attend the
focus-group interview. A total of 24 students attended the interview. The results of the
survey provided a general picture of how the students evaluated the effectiveness of
using podcasts to learn. The focus-group interview allowed the researcher to explore
the data collected from the survey in an interactive way. During the interview, semi-
structured questions were asked and students commented on the podcasts offered in
their courses and provided reasons for their ratings.

Data analysis procedure:
For the focus-group interviews, the students responses were audio recorded and a
complete typed transcript was made. To ensure anonymity, the name of each student
was replaced with a code. In order to identify and classify the target variables,
categories were developed and the related data were allocated to available typological
categories before the generalization of the data. The interviews were analyzed in the
following three areas:
(1) Students attitudes towards learning with podcasts;
(2) Comparing face-to-face learning with learning from podcasts;
(3) Students expectations of effective educational podcasting.
Validity and reliability (or trustworthiness) of the study:
In analyzing the validity and reliability of qualitative studies, such as focus groups, many
researchers will resort to trustworthiness. Studies such as this are hard to quantify
(numerically), therefore, researchers employ Gubas construct of trustworthiness that includes:
1. Credibility
2. Transferability
3. Dependability
4. Conformability

In this study, a total of 128 (70%) out of 184 students completed the questionnaire. Of
the 128 respondents, 24 (19%) students replied that had not watched any of the
podcasts. Only feedback from students who had watched the podcasts at least once
(n = 104) was used in the analysis of the results. Three students from each course,
totaling 24 students (n = 24), were interviewed. Cronbachs Alpha was 0.87, which was
higher than the level of acceptance.

How does the methodological approach serve the research purposes:
While I was directing my analysis towards focus groups, this study was actually a combination,
using both surveys and focus groups. These types of studies can be very useful, however,
critics point out a number of potential problems with focus group research, such as:
1. Observer Dependency - Results can be influenced by the researcher.
2. Participants are susceptible to group think. In a group setting, a lack of anonymity
means participants may hold back on their true opinions and provide responses they
think would be accepted by the group.
3. Testing arrangements (i.e. the Setting), the environment in which the focus group is
interviewed in, my have obvious recording equipment, for example, that some
participants may find intrusive and stifle a true opinion.

For a study such as this, you could create a quantitative study to determine if the podcast
actually improved comprehension, but that would not provide information about how the
students felt about using podcasts to learn music and visual arts (i.e. do they like using
podcasts, do they find this form of instruction more enjoyable). While the podcasts may not
improve the students learning, it may be a more experience of learning.

Citations:
Research in Learning Technology 2012; 20: 14919 - DOI: 0.3402/rlt.v20i0/14919
E.G. Guba, Criteria for assessing the trustworthiness of naturalistic inquiries,
Educational Communication and Technology Journal 29 (1981).

You might also like