Session M2E

Effects of Peer Tutoring, Attitude and Personality on Academic Performance of First Year Introductory Programming Students
Paul Golding, Lisa Facey-Shaw and Vanesa Tennant
School of Computing & Information Technology, University of Technology, Jamaica 237 Old Hope Road, Kingston 6, Jamaica cashmere@cwjamaica.com , lshaw@utech.edu.jm, vtennant@cwjamaica.com,

Abstract -Programming is the core of any computer science degree. Failure to grasp programming concepts such as logical reasoning and problem solving skills will have a negative impact on students academic performance. A preliminary research was carried out at the University of Technology, Jamaica amongst first year computing students to ascertain factors that affect poor performance in introductory programming course. The study sought to examine the effect of peer tutoring, and attitude on academic performance. A control and experimental group was established, and an instrument was administered to measure students attitude to programming along with the use of pre and post test scores. The study suggested that peer tutoring had a positive effect on students attitude, namely, personal confidence, and academic performance. Further investigations were carried with a similar methodology and personality was explored as an additional factor affecting academic performance. Both studies revealed that personal confidence in learning programming contributed more significantly than the usefulness of programming and the perception of teacher motivation. The statistical analysis revealed no significant influence of personality types on academic performance, which suggests that it does not play an important role. Index Terms Academic Performance, Attitude to Programming, Introductory Programming, Peer Tutoring, Personality. INTRODUCTION In computer science curriculums, introductory programming is probably the most important course, because it can shape the student's mind forever [1]. In light of this, Introductory Programming is a foundation first year course in many Computer Science degree programmes. The rate of drop out and failure in introductory programming classes in universities is high worldwide between 25% to 80% [2]. This can be detrimental to students future performance in Computer Science, as an introductory programming course enhances logical, reasoning and problem solving skills. Students learning to program must master the semantics in algorithm

development prior to developing the syntax of a program. This high failure rate has caused great concern within universities. The University of Technology, Jamaica (UTECH) offers a Bachelors Degree in Computing and Information Technology (BSCIT), and all students must pursue Introduction to Programming (ITP) course in the first semester of study. The School of Computing has experimented using peer tutoring to increase students performance in Introductory Programming. The failure rate for ITP over the past four years ranges from approximately 30% to 48%, which is the similar trend in universities worldwide. A research was carried out amongst first year students to examine the effects of peer tutoring and attitude to programming on students performance in ITP. The study of 33 students revealed that various aspects of a persons attitude contribute to academic performance. The categories of attitude studied were personal confidence in learning programming, usefulness of programming and the perception of teacher motivation. The findings revealed that personal confidence contributed more significantly to academic performance [3]. Thus, there is the need for further research to determine if the other categories play a role in academic performance. This study had a number of limitations, including the small sample size, and students were not paired based on compatibility of personality. This follow up research, with a larger sample examines the effects of peer tutoring, and attitude to first year programming taking the introductory programming course. However personality was added as a new factor affecting academic performance. It is the intent of the research to examine the impact of these factors as we seek to improve the success rate in ITP and add to the literature seeking to increase students academic performance. LITERATURE REVIEW Programming is the craft of implementing one or more interrelated abstract algorithms using a particular programming language to produce a concrete computer program. It has elements of art, science, mathematics, and engineering [4]. Prior to teaching students programming, it is

1-4244-0257-3/06/$20.00 2006 IEEE October 28 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference M2E-7

Session M2E
essential that the art of programming be conveyed effectively. Introductory programming courses provide students with algorithmic and computational capabilities for strong analytical and critical thinking skills [5]. However, the literature suggests that there are high failure rates in introductory programming worldwide. This has caused educators within the computing fraternity to seek various methods to improve the learning experience of students. It was noted that the conventional teaching approach (consisting of long lectures plus homework, quizzes, and exams) is not a very good approach for teaching introductory programming [6]. A requirement for teaching introductory programming to students whose branch of learning is engineering or science is bridging the gap between in-class lectures and real-world applications [7]. Failure to grasp the required concepts will affect the students attitude and performance. Therefore, there is a need to embark on different instructional methods for introductory programming courses. A methodology used to improve students performance in academics is peer tutoring. Peer tutoring is the process by which a pupil, with guidance from a teacher, helps one or more students at the same grade level learn a skill or concept [8]. This approach focuses on peers to solve a problem, and is most effective in fostering creativity, experimentation, problem-solving skills and the learning of deep concepts [9]. This learning method has been applied to programming and has produced positive results. Prior research indicated that pairs of programmers produce higher quality code in about half the time when compared with programmers that work on their own [10, 11]. Additionally, another survey reported programmers tend to collaborate as software developers generally spend 30% of their time working alone, 50% of their time working with one other person, and 20% of their time working with two or more persons [12]. A study related that poor performance in Introductory programming courses may be affected by students perception of the subject. It was highlighted that undergraduate students believe that programming is too hard, uninteresting, and unsocial [13]. Research has showed that peer tutoring and or pair programming increases students attitude towards programming. A study [14] reported students reaction to pair programming in their courses. Ninety-five percent (95%) of the students agreed with the statement I was more confident in our assignments because we pair programmed. Additionally, eighty-four percent (84%) of the students enjoyed the experience of pair programming. Therefore, tactics that increase students attitude to programming may yield increased performance and appreciation of the underlining concepts in programming. Some researchers have studied personality as a factor affecting the success rate in computer programming courses. Personality is a particular pattern of behavior and thinking prevailing across time and situations that differentiates one person from another [15]. It was highlighted that students may learn in different ways based on their personality profile. A study on personality profiles sought to identify people as introverts or extroverts, where introverts relate more to interest in a subject, and extroverts focus more specifically on objects in their environment[16]. Another study divided learners into four cognitive profiles, using two letter acronyms that include ST for Sensor Thinker, SF for Sensor Feeler, NT for Intuitive Thinker, and NF for Intuitive Feeler. It was related that sensor learners tend to gain knowledge through the use of their senses, while intuitive learners use visual learning methods to recall information. Thinkers like to have concrete evidence for decisions, while feelers tend to make decisions based on emotion or morality [17]. Computer programming requires the ability to collect and interpret information relating to a problem, followed by the application of this information to a corrective action [17]. This is supported by another research which highlighted that thinking students tend to enjoy and complete computer programming courses more than their feeling counterparts [18]. Additionally, it was revealed that thinking students are also more likely to experiment with software [19]. The study [19] concluded that thinking profiles are more inclined to process information in a manner conducive to successful writing of computer programs and understanding of concepts. A study [20] on personality and programming indicated that with the exception of the SF cognitive profile, all other profiles exhibit similarly positive outcomes in progamming principles. Therefore, it was recommended that the instructional methods should focus on improving the SF profiles up to the level of the other profile groups. It was noted that SF students tended to learn best through repetition and by breaking problems into steps or milestones. Additionally, it was highlighted that SF learners attempted to relate the information to real life objects. Hence, integrating real-world problems into the classroom should make the course more useful and relevant to a wide variety of students. [20]. It was noted that SF profiles tend to perform well when studying and working in pairs or groups, hence pair programming is a recommended instructional method for these students[20]. However, studies on pair programming suggested that students were sometimes teamed with incompatible personality types and complained about working with partners that had different personalities [21]. This may affect academic performance, hence factors such as peer tutoring, attitude and personality should not be examined independently. METHODOLOGY Design

A quantitative approach applying a pretest-posttest control group quasi-experimental design was adopted for the research. It must be noted that this was the same approach applied in the preliminary research. This design was chosen due to the existence of naturally assembled classes for which random assignments to the experimental and control groups could not be applied [22]. In order to avoid discrepancies in the results, the characteristics of the experimental group (E) and control group (C) were matched as closely as possible. The study was carried out for one semester (approximately 14 weeks) during 1-4244-0257-3/06/$20.00 2006 IEEE October 28 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference M2E-8

Session M2E
the period September to December. In order to eliminate the instructor as a factor affecting academic performance, both the control and experimental group had the same tutor. In addition, students repeating the course were excluded from the sample. The control group was taught using conventional teaching methods, whilst peer tutoring was applied to the experimental group. This study did not pair students based on personality but rather random selection. The diagram below represents the factors studied in the research. tutorial time for the ITP course a two-hour period, held once per week. The construction of pairs for peer tutoring within the experimental group was randomly selected. Group Composition The total number of participants was forty-two (42) with the gender composition being predominantly male approximately 60%. This reflects the gender makeup of SCIT. Seventy-two (72) percent of the population was nineteen (19) or younger. It must be noted that the sample group was not much greater than the previous study. Both the control and experimental groups had attained Olevel, 80% and 41% respectively. As it regards to A Level, the distribution was 20% and 59% for the control and experimental group respectively. Hence, the control group had much greater Olevel qualification whilst the experimental group had greater A level qualification. Both groups had the same percentage of students with A Level mathematics (50%). On the other hand, the difference between attainment of Olevel mathematics in the control and experimental group was 56% and 44% respectively. In terms of respondents taking a formal programming course, the results were 47% and 53% for the control group and experimental group respectively. Tasks Similar learning materials and resources were adopted for both groups. Group work included problem-oriented questions based on the lecture previously held on that topic. In the experimental group, students were encouraged to work in peers and practice discussion and problem solving techniques. The tutor was available to explain any questions and provide guidance where necessary. Within the control group, students generally worked alone on problems. On completing questions, the results were discussed amongst the class with tutor. In most instances, both the control and experimental groups were assigned the same problems, but problems would occasionally differ. All tests were taken individually. FINDINGS Peer Tutoring and Academic Performance Table I contains the pre-test and post-test score means and standard deviations for the experimental and control groups. Both the pre-test and post-test scores were higher for the experimental group in comparison to the control group. This was not the case in the first study as the control group had a decrease in mean score from the pre-test to the post-test, while the experimental group had an increase in scores

Peer Tutoring Attitude Academic Performance

C
Personality

FIGURE 1 DIAGRAM SHOWING FACTORS STUDIED IN THE RESEARCH

Survey Students were given a survey constituting a brief programming pre-test which measured their knowledge of programming concepts. It required that students analyze algorithms and answer questions on their execution as well as to evaluate problem-based mathematical expressions. Another survey was developed to measure general attitudes towards ITP. This instrument was derived from the Fennema-Sherman mathematics attitudes scales, which was modified to reflect programming [23]. The survey uses three of the seven subscale categories used in the Fennema-Sherman instrument, namely 1) Confidence in learning programming, 2) Usefulness of programming, and 3) Effective motivation in programming. The attitude survey consisted of a total of thirty-six (36) positive and negative statements. Each scale consisted of twelve statements, six positively worded and six negatively worded. The statements on the questionnaire were rated on a five-point Likert type scale, ranging from strongly agree (1) to strongly disagree (5). Additionally questions on the instrument gathered demographical data namely age, gender, highest educational level attained, highest mathematical level attained and prior programming experience. An abbreviated Myers-Briggs Personality Type Indicator model was used to ascertain students personality. The instrument used four basic scales with opposing characteristics to measure an individual preference. The four scales are: (1) extraversion/introversion (E or I), (2) sensing/intuitive (S or N), (3) thinking/feeling (T or F), and (4) judging/perceiving (J or P). The various combinations of these preferences result in 16 personality types. The sample consisted of two BSCIT first year classes of students, each consisting of approximately twenty to twenty five students. The study was conducted during the normal

1-4244-0257-3/06/$20.00 2006 IEEE October 28 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference M2E-9

Session M2E
TABLE I MEAN PRE-TEST AND POST-TEST SCORES BY GROUP Type of Group Control Group Mean Std. Dev Experimental Group Mean Std. Dev Total Mean Std. Dev Pre-Test Score 39.13 17.30 42.35 19.54 40.50 18.11 Post-Test Score 53.18 17.33 56.11 17.78 54.50 17.37 Control Group Experiment Group TABLE IV DISTRIBUTION OF GRADES FOR POST-TEST SCORES A 0% 11% B 18% 17% C 50% 33% D 32% 38%

Attitude and Academic Performance The attitude of students was examined under three areas: namely personal confidence in programming, usefulness of programming and perception of teacher motivation. The calculated mean values for each category is represented in Table V.
TABLE V MEAN ATTITUDE SCORES BY GROUP Personal Usefulness confidence in learning of programming programming Mean Std. Deviation Experimental Group F .304 Sig. 0.585 Total Mean Std. Deviation Mean Std. Deviation 3.64 0.81 3.74 0.56 3.68 0.71 3.96 0.71 4.26 0.48 4.09 0.63

A one-way ANOVA (Table II) and a t-test done for the pre-test scores indicates that there were small differences between the groups at the beginning of the instruction .The p value of 0.585 indicates that there is no significant difference in the pre-test results for the control and experiment group. Similar results are shown in Table III with f value of 0.276; the p value of .602 indicates that there is also no significant difference in the post-test scores for both groups. The prior study also reported no significant difference between the pretest and post-test for both groups.
TABLE II ANOVA FOR PRE-TEST SCORES Sum of Mean Squares df Square Between Groups Within Groups Total 101.509 12688.491 12790.000 1 38 39 101.509 333.908

Type of Group Control Group

Perception of teacher motivation 3.69 0.46 3.63 0.53 3.66 0.49

TABLE III ANOVA FOR POST-TEST SCORES Sum of Mean Squares df Square Between Groups Within Groups Total 84.949 11679.051 11764.00 1 38 39 84.949 307.343

F 0.276

Sig. 0.602

Due to the fact that Table III indicated that there was not much difference in the post test scores for the control and the experiment group, a further examination was carried out in terms of the quality of grades. Actual grades were categorized as follows: 80-100 (A), 70-79 (B), 50-69 (C) and 0-49 (D), with grade D representing failure. The percentage of passes in the control and experimental group was 68% and 61% respectively. However, only persons in the experimental group (11%) received grade A. It is important to note that a greater percentage of students from the control group received grades B and C. This is also the situation for students failing with D. Therefore it cannot be concluded that the existence of A grade in experimental speaks to greater performance.

A mean value of 3 suggests that there is indifference in attitude to programming. On examining the sub-categories, it was noted that students in the experimental group were more confident in their ability to learn programming than students in the control group. This may have contributed to better academic performance in the pre-test scores. A linear regression was performed which revealed a strong correlation (0.653) between confidence in programming and academic performance (see table VI). This result is significant with a p value of 0.000 (see table VII)
TABLE VI MODEL 1 SUMMARY Adjusted R Std. Error of R R Square Square the Estimate .653(a) .426 .411 13.32825 a Predictors: (Constant), Personal confidence in learning programming Model 1

1-4244-0257-3/06/$20.00 2006 IEEE October 28 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference M2E-10

Session M2E
TABLE VII ANOVA FOR MODEL 1 Model 1 Regression Residual Total Sum of Squares 5013.60 6750.41 Mean Square 5013.60 177.64

df 1 38

F 28.23

Sig. .000(a)

11764.0 39 a Predictors: (Constant), Personal confidence in learning programming b Dependent Variable: Post-Test Score

Regarding other categories, the experimental group also had a greater belief in the usefulness of programming. This may have also contributed to a greater performance in pretest scores than the control group. The analysis revealed a weak correlation of 0.319 (see table VIII). This was statistically supported with a p value of 0.045 (see table IX). The previous study also reported that the experimental group had greater belief in usefulness of programming
TABLE VIII MODEL 2 SUMMARY Adjusted R Std. Error of the R R Square Square Estimate .319(a) .102 .078 16.67384 a Predictors: (Constant), Total usefulness of programming Model 2

The portion of sample (11%) that received grade A had personality traits of INFP and INTP. However it cannot be deduced if the common traits (I, N and P) played a role in the performance. For students failing the course, there was no particular personality combination that was common amongst those students. The analysis revealed a weak significance with p value of 0.662. It is important to note that much research has been carried out on programming and personality, however results are mixed. Prior research has suggested that intuitive thinkers (NT) tended to perform better in programming principles than sensor feelers (SF)[20]. This was not evident in this research. One of the major research performed in this area [18], that studied students enrolled in an introductory programming classes concluded that personality has little to do with overall achievement in computer programming classes. It was suggested that cognitive profile may influence problem representation and planning but has little effect on the coding and debugging stages of computer programming Other Factors The role of gender was examined as it relates to post and pretest scores and attitude towards programming. It is important to note that this is not the focus of the study; however it was observed to note any relevance. The results, in Table X, revealed that both genders received similar mean scores for the pre-test score. The increase in scores from the pre-test to the post-test for the male and female was 39% and 14% respectively. Therefore, not only did the males receive a higher mean score for the posttest, but also they reflected a better improvement in score from the pre-test. Statistical analysis with gender and post test scores revealed a weak correlation of 0.319 and a p value of 0.054.
TABLE X MEAN PRE-TEST AND POST-TEST SCORES BY GENDER Gender Male Mean Std. Deviation Mean Std. Deviation Mean Std. Deviation Pre-Test Score 41.81 17.63 41.33 19.22 41.62 18.03 Post-Test Score 58.05 16.73 47.06 16.49 53.30 17.30

TABLE IX ANOVA FOR MODEL 2 Mo -del 2 Sum of Squares Regression Residual Total 1199.36 10564.64 Mean Square 1199.36 278.02

df 1 38

F 4.314

Sig. .045 (a)

11764.00 39 a Predictors: (Constant), Total usefulness of programming b Dependent Variable: Post-Test Score

On the other hand, the control group had a greater belief that perception of teachers motivation affects academic performance. Analysis on this subcategory highlighted a weak correlation of 0.309 and a p value of 0.052. Personality The various combinations of preferences in the personality test result in 16 personality types. Only four personality types were outstanding in this sample population. The majority of the students in this sample had personality traits combinations of ESTP (extroversion, sensing, thinking, perceiving [17%]), INFP (introversion, intuition, feeling, perceiving [19%]), and ISTP (introversion, sensing, thinking, perceiving [19%]). The control group had a stronger combination of the abovementioned traits than the experimental, with 14 students and 9 students respectively.

Female

Total

Statistical analysis revealed weak correlation on examining the three categories in attitude with gender. Both total usefulness of programming and perception of teacher motivation reported no significant difference. However personal confidence in learning programming reported a p value of 0.029 and correlation of 0.35. In this study, males displayed a higher level of personal confidence in learning programming. There has been ongoing research examining the

1-4244-0257-3/06/$20.00 2006 IEEE October 28 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference M2E-11

Session M2E
relationship between gender and personal confidence in programming [27, 28]. CONCLUSIONS AND FUTURE WORK The primary purpose of this research was to investigate peer tutoring, attitude, and personality to understand factors affecting academic performance in Introductory Programming courses. The findings were rather interesting, as it revealed that attitude to programming does affect academic performance. The findings suggested that personal confidence in learning programming was the most significant factor affecting academic performance. This was further supported as the experimental group had more personal confidence and had a higher mean score than the control group. This may suggest that peer tutoring has a positive effect on students academic performance. This study reported that personality traits were not a significant factor in academic performance. There is not one particular solution in the quest to decrease the attrition rate in Introductory Programming courses within universities. This is so as many studies have reported related mixed results on the effect of peer tutoring, attitude and personality on academic performance in programming courses. It is recommended that further work be carried out with a larger and diverse sample such as day and evening students to measure the effects of attitude on academic performance. In addition, to effectively study personality with peer tutoring, students should be paired based on compatible for peer tutoring exercises. The above-mentioned may aid in obtaining more conclusive results. This will aid in the modification of instructional methods at UTECH and other institutions. REFERENCES
[1] Kessler, M., Meyer, B., Exercise Design for Introductory Programming Learn-by- doing basic O-O-concepts using Inverted Curriculum, 2004 [2] http://www.ifi.uio.no/infdid/Slides/LearningProgramming.pdf. Retrieved March 2, 2006. [3] Facey-Shaw, L., Golding, P., Effect of Peer Tutoring, Attitude on Academic Performance of First Year Introductory Programming Students, 35th ASEE/IEEE Frontiers in Education Conference, 2005 [4] http://en.wikipedia.org/wiki/Programming. Retrieved March 4, 2006 [5] Rufinus, J., Kortsarts, Y., Teaching an Introductory Programming Course For Non- Majors Using Python, Proc ISECON, v22 (Columbus OH), 2005 [6] Cohen, D. K., Jackson, P., Contributing to Educational Change, Berkeley, CA, 1989 [7] Wirth, M., Mechanisms for teaching introductory programming using active learning, 2004 [8] http://ali.apple.com/ali_sites/ali/exhibits/1000328/Peer_Tutoring.html Retrieved March 03, 2006 [9] Williams, L. W., Wiebe, E., Yang, K., Ferzli, M., Miller, C., " In Support of Pair Programming in the Introductory Computer Science Course ", Computer Science Education, Vol. 12, No. 3, pp. 197-212, 2002. [10] Williams, L., Kessler, R., Cunningham, W., & Jeffries, R., Strengthening the case for pair programming, IEEE Software, 17(3), 2004. [11] Cockburn, A. & Williams, L., The costs and benefits of pair programming" (Unpublished). [12] DeMarco, T., and Lister, T. Peopleware, New York: Dorset House Publishers, 1987. [13] Farkas, D., Murthy, N., Attitudes toward Computers, the Introductory Course and Recruiting New Majors: Preliminary Results, 17th Workshop of the Psychology of Programming Interest Group, Sussex University, 2005, pp. 268-277. [14] Thomas, L., Ratcliffe, M., Robertson, A., " Code Warriors and Code-APhobes: A Study in Attitude and Pair Programming ", 34th SIGCSE Technical Symposium on Computer Science Education (Reno, Nevada, 2003), pp. 363-367 [15] http://www.sasked.gov.sk.ca/docs/social/psych30/Glossary.htm. Retrieved March 6,2006 [16] Jung, C. G. Psychological Types (Collected Works of C. G. Jung, Vol. 6). Princeton University Press, 1971 [17] Krause, L., How We Learn and Why We Don't, Thomson Learning, Cincinnati, OH, 2000 [18] Bishop-Clark, C. "Cognitive Style and its Effect on the Stages of Programming." Journal of Research on Computing in Education. Vol. 27, No. 4, Summer 1995, pp. 373-386 [19] Jones, W. P., "Computer Use and Cognitive Style." Journal of Research on Computing in Education. Vol. 26, No. 4, Summer 1994, pp. 514-522. [20] Shade, S., Guthrie, T., Woszczynski, A., Personality and Programming, Journal of Information Systems Education. Fall 2005 [21] Nagappan, Nachiappan, Williams, Laurie, Ferzli, Miriam, Wiebe, Eric, Yang, Kai, Miller, Carol, and Balik, Suzanne "Improving the CSl Experience with Pair Programming." SIGCSE '03, (Reno, Nevada,2003), pp. 359-362. [22] Best, J.W., Kahn, J.V., Research in Education 9th Ed., 2003, pp. 178. [23] Doepken, D., Lawsky, E., Padwa, L., Modified Fennema-Sherman Attitude Scale. Retrieved August 16, 2004 from http://www.woodrow.org/teachers/math/gender/08scale.html [24] http://www.crlt.umich.edu/CRLTNorth/F05CRLT-N_News.pdf. Retrieved March 12, 2006 [25 ] Pintrich, P., & Zusho, A., Student motivation and self-regulated learning in the college classroom., Higher education: Handbook of theory and research. Boston: Kluwer Academic Publishers, 2002 [26] Wigfield, A., & Eccles, J. S., . Expectancy-value theory of motivation Contemporary Educational Psychology, Vol 25, 2000, pp.68-81. [27] Milto, E., Rogers, C., Portsmore, M., Gender Differences in Confidence level, group interactions, and feelings about competition in an Introductory Robotics Course, 32nd ASEE/IEEE Frontiers in Education Conference, 2002. [28] Beckwith, L., Burnett, M., Gender: An Important Factor in End-User Programming Environments?, VL/HCC'04: IEEE Symposium on Visual Languages and Human-Centric Computing, Rome, Italy, 2004 pp. 107114.

1-4244-0257-3/06/$20.00 2006 IEEE October 28 31, 2006, San Diego, CA 36th ASEE/IEEE Frontiers in Education Conference M2E-12