A Meta-Analysis Examining the Impact of Computer-Assisted
Instruction on Postsecondary Statistics Education: 40 Years of Research
Karen Larwin Youngstown State University David Larwin Kent State University at Salem JRTE | Vol. 43, No. 3, pp. 253278 | 2011 ISTE | iste.org
The research question(s): In light of all of the conflicting research regarding CAI with these variables as well as others, meta-analysis is the tool that can provide a general measure of the impact of CAI on student achievement in statistics instruction that might otherwise be obscured by these conflicting results. A metaanalytic investigation is an appropriate and effective approach to synthesizing and integrating the conflicting results from this quantitative research (Cooper & Hedges, 2009; Johnson, 1989). Put more succinctly, the study was to show whether or not students enrolled in postsecondary statistics courses benefit from CAI, as evidenced by their achievement scores.
Data collection procedure: Meta-analysis is like an aggregator, it compares and combines the conclusions of many studies. The word meta is from the Greek, meaning about. In our current context, meta-analysis means a study about other studies. In this case, many studies are brought together to form a comprehensive assessment of the validity of a particular research question (Does Computer Aided Instruction (CAI) improve comprehension of statistics students?). Many studies have been performed exploring this question, this study brings all of those together to get an idea of what the many studies conclude. The researchers gathered the studies in the following way: 1. First, we gathered and examined research studies. Studies included in the meta-analysis must fit within the defined parameters selected for analysis while representing as much of the population of data available on the research area. Only studies addressing the primary research question were considered.
Data analysis procedure : Once the studies were gathered, the analysis portion begans. For this analysis, the researchers used the following procedure: 1. Describe, classify, and code all the research studies to be included in the meta-analysis. Once the pertinent research studies were gathered, they had to be organized. 2. The analysis of the overall mean effect-size measures and the mean effect-size measures for each research characteristic being examined. Analyze the assembled, organized studies, for common research characteristics. Also ,the studies were weighted based on sample size (assuming that a larger sample size yields a more accurate result).
Validity and reliability (or trustworthiness) of the study: In a quantitative study like this, validity and reliability are strictly a matter of numerical and mathematical analysis. Based on research conclusions, a statistical construct is created to compare and combine the results of the studies being analyzed.
How does the methodological approach serve the research purpose: Meta-analysis is a valuable form of research analysis because the aggregation of many studies has a tendency to correct errors or bias from any one particular study. Meta-analysis helps avoid conclusions that may result from a single study. In this case, 71 different studies were brought together to yield a comprehensive conclusion.
Citations:
The effectiveness of educational podcasts for teaching music and visual arts in higher education-Focus Groups Cheung On Tam* Department of Cultural and Creative Arts, Hong Kong Institute of Education, Hong Kong (Received 31 January 2011; final version received 11 May 2011) The research question(s): Research has been conducted to determine the effectiveness of podcasts in teaching business, science, and languages. However, little or no research has been done to investigate the effectiveness of podcasts in the instruction of music and visual arts. The aims of this study were to investigate the effectiveness of using podcasts to learn music and visual arts and to identify the characteristics of an effective educational podcast from the perspective of students.
Data collection procedure: An evaluation survey and focus-group interviews were used to collect students feedback. The survey was administered to all students in the last session of each course and three students from each course were randomly selected to attend the focus-group interview. A total of 24 students attended the interview. The results of the survey provided a general picture of how the students evaluated the effectiveness of using podcasts to learn. The focus-group interview allowed the researcher to explore the data collected from the survey in an interactive way. During the interview, semi- structured questions were asked and students commented on the podcasts offered in their courses and provided reasons for their ratings.
Data analysis procedure: For the focus-group interviews, the students responses were audio recorded and a complete typed transcript was made. To ensure anonymity, the name of each student was replaced with a code. In order to identify and classify the target variables, categories were developed and the related data were allocated to available typological categories before the generalization of the data. The interviews were analyzed in the following three areas: (1) Students attitudes towards learning with podcasts; (2) Comparing face-to-face learning with learning from podcasts; (3) Students expectations of effective educational podcasting. Validity and reliability (or trustworthiness) of the study: In analyzing the validity and reliability of qualitative studies, such as focus groups, many researchers will resort to trustworthiness. Studies such as this are hard to quantify (numerically), therefore, researchers employ Gubas construct of trustworthiness that includes: 1. Credibility 2. Transferability 3. Dependability 4. Conformability
In this study, a total of 128 (70%) out of 184 students completed the questionnaire. Of the 128 respondents, 24 (19%) students replied that had not watched any of the podcasts. Only feedback from students who had watched the podcasts at least once (n = 104) was used in the analysis of the results. Three students from each course, totaling 24 students (n = 24), were interviewed. Cronbachs Alpha was 0.87, which was higher than the level of acceptance.
How does the methodological approach serve the research purposes: While I was directing my analysis towards focus groups, this study was actually a combination, using both surveys and focus groups. These types of studies can be very useful, however, critics point out a number of potential problems with focus group research, such as: 1. Observer Dependency - Results can be influenced by the researcher. 2. Participants are susceptible to group think. In a group setting, a lack of anonymity means participants may hold back on their true opinions and provide responses they think would be accepted by the group. 3. Testing arrangements (i.e. the Setting), the environment in which the focus group is interviewed in, my have obvious recording equipment, for example, that some participants may find intrusive and stifle a true opinion.
For a study such as this, you could create a quantitative study to determine if the podcast actually improved comprehension, but that would not provide information about how the students felt about using podcasts to learn music and visual arts (i.e. do they like using podcasts, do they find this form of instruction more enjoyable). While the podcasts may not improve the students learning, it may be a more experience of learning.
Citations: Research in Learning Technology 2012; 20: 14919 - DOI: 0.3402/rlt.v20i0/14919 E.G. Guba, Criteria for assessing the trustworthiness of naturalistic inquiries, Educational Communication and Technology Journal 29 (1981).