Vol 16 No 1 - Janaury 2017

IJLTER.
ORG
International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN: 1694-2493
e-ISSN: 1694-2116
Vol.16 No.1
PUBLISHER International Journal of Learning, Teaching and
London Consulting Ltd Educational Research
District of Flacq
Republic of Mauritius
www.ijlter.org The International Journal of Learning, Teaching
and Educational Research is an open-access
Chief Editor journal which has been established for the dis-
Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the
Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER
welcomes research articles from academics, ed-
Editorial Board
Prof. Cecilia Junio Sabio ucators, teachers, trainers and other practition-
Prof. Judith Serah K. Achoka ers on all aspects of education to publish high
Prof. Mojeed Kolawole Akinsola quality peer-reviewed papers. Papers for publi-
Dr Jonathan Glazzard cation in the International Journal of Learning,
Dr Marius Costel Esi Teaching and Educational Research are selected
Dr Katarzyna Peoples through precise peer-review to ensure quality,
Dr Christopher David Thompson
originality, appropriateness, significance and
Dr Arif Sikander
Dr Jelena Zascerinska readability. Authors are solicited to contribute
Dr Gabor Kiss to this journal by submitting articles that illus-
Dr Trish Julie Rooney trate research results, projects, original surveys
Dr Esteban Vzquez-Cano and case studies that describe significant ad-
Dr Barry Chametzky vances in the fields of education, training, e-
Dr Giorgio Poletti learning, etc. Authors are invited to submit pa-
Dr Chi Man Tsui
Dr Alexander Franco
pers to this journal through the ONLINE submis-
Dr Habil Beata Stachowiak sion system. Submissions must be original and
Dr Afsaneh Sharif should not have been published previously or
Dr Ronel Callaghan be under consideration for publication while
Dr Haim Shaked being evaluated by IJLTER.
Dr Edith Uzoma Umeh
Dr Amel Thafer Alshehry
Dr Gail Dianna Caruth
Dr Menelaos Emmanouel Sarris
Dr Anabelie Villa Valdez
Dr zcan zyurt
Assistant Professor Dr Selma Kara
Associate Professor Dr Habila Elisha Zuya
VOLUME 16 NUMBER 1 January 2017
Table of Contents
Item Consistency Index: An Item-Fit Index for Cognitive Diagnostic Assessment ....................................................... 1
Hollis Lai, Mark J. Gierl, Ying Cui and Oksana Babenko
Factors That Determine Accounting Anxiety Among Users of English as a Second Language Within an
International MBA Program ................................................................................................................................................ 22
Alexander Franco and Scott S. Roach
(Mis)Reading the Classroom: A Two-Act Play on the Conflicting Roles in Student Teaching .............................. 38
Christi Edge
Coping Strategies of Greek 6th Grade Students: Their Relationship with Anxiety and Trait Emotional Intelligence
................................................................................................................................................................................................. 57
Alexander- Stamatios Antoniou and Nikos Drosos
Active Learning Across Three Dimensions: Integrating Classic Learning Theory with Modern Instructional
Technology ............................................................................................................................................................................ 72
Thaddeus R. Crews, Jr.
The Effects of Cram Schooling on the Ethnic Learning Achievement Gap: Evidence from Elementary School
Students in Taiwan .............................................................................................................................................................. 84
Yu-Chia Liu, Chunn-Ying Lin, Hui-Hua Chen and He Huang
Teachers Self-Efficacy atMaintaining Order and Discipline in Technology-Rich Classrooms with Relation to
Strain Factors ....................................................................................................................................................................... 103
Eyvind Elstad and Knut-Andreas Christophersen
Using Reflective Journaling to Promote Achievement in Graduate Statistics Coursework...................................... 120

J. E. Thropp
Competence and/or Performance - Assessment and Entrepreneurial Teaching and Learning in Two Swedish
Lower Secondary Schools .................................................................................................................................................. 135
Monika Diehl and Tord Gran Olovsson
Review in Form of a Game: Practical Remarks for a Language Course ...................................................................... 161
Snejina Sonina
1
International Journal of Learning, Teaching and Educational Research

Vol. 16, No. 1, pp. 1-21, January 2017
Item Consistency Index: An Item-Fit Index for

Cognitive Diagnostic Assessment
Hollis Lai,1 Mark J. Gierl,2 Ying Cui,2 Oksana Babenko3
School of Dentistry, Faculty of Medicine & Dentistry

1
2Centrefor Research in Applied Measurement and Evaluation
3Department of Family Medicine, Faculty of Medicine & Dentistry
University of Alberta, Canada
Abstract. An item-fit index is a measure of how accurately a set of

item responses can be predicted using the test design model. In a
diagnostic assessment where items are used to evaluate student
mastery on a set of cognitive skills, this index helps determine the
alignment between the item responses and skills that each item is
designed to measure. In this study, we introduce the Item
Consistency Index (ICI), a modification of an existing person-
model fit index, for diagnostic assessments. The ICI can be used to
evaluate item-model fit on assessments designed with a Q-matrix.
Results from both a simulation and real data study are presented.
In the simulation study, the ICI identified poor-fitting items under
three manipulated conditions: sample size, test length, and
proportion of poor-fitting items. In the real-data study, the ICI
detected three poor-fitting items for an operational diagnostic
assessment in Grade 3 mathematics. Practical implications and
future research directions for the ICI are also discussed.
Keywords: Item Consistency Index; cognitive diagnostic assessment; test
development
Introduction
In educational testing, items are developed to elicit a correct response
when examinees demonstrate adequate knowledge or understanding on
the required tasks and skills within a specified domain. The methods of
specifying knowledge, the conceptualization of content domains, and the
design of how an item elicits responses are currently undergoing
significant change with the evolution of our test designs. But one outcome
that remains the same is that an item must assess the tasks and skills as
intended, and the quality of each item must be judged to be high if it is to
be included on the test. In most test designs, item discrimination power is
a statistical criterion that is synonymous with describing item quality.
2017 The authors and IJLTER.ORG. All rights reserved.

2
Item discrimination helps describe how well an item can differentiate

examinees at different performance levels. Depending on the test design
and how the scale of examinee performance is realized, different
measures of item discrimination may be used. Additional information
about item discrimination can also be garnered from measures of item-
model fit. An item-model fit index describes the overall difference
between real responses on a given item with a corresponding set of
expected responses predicted by the test design. Item-model fit indices
can be summarized, in general, as a ratio between the expected and actual
correct responses on each item to compare the proportion of correct
responses across examinees of different abilities with an expected correct
proportion from the test design model. Different criterions that represent
the examinee overall performance such as total score, estimated ability, or
pseudo-scores have been used to group the responses of examinees' with
similar ability to produce variations of item-model fit (Bock, 1972; Yen,
1981; Rost & von Davier, 1994; Orlando & Thissen, 2003). Application of
item-model fit indices include the identification of poor performing items,
cheating, or test administration anomalies, along with addressing issues
related to dimensionality, item construction, calibration, and model
selection (Reise, 1990).
Cognitive Diagnostic Assessment and Model Fit

Demand for more assessment feedback to better guide instruction and
learning has led to the development of more complex test designs.
Cognitive diagnostic assessment (CDA) is an example of a test design that
yields enhanced assessment feedback by providing test takers with
specific information about their problem-solving mastery on a given
domain (Gierl, Leighton, & Hunka, 2007). The cornerstone of a CDA is the
use of a cognitive model to guide test development. The use of a
cognitive model allows CDA to provide enhanced feedback because
cognitive information can be extracted from the examinees item
responses which, in turn, provide more detailed and instructionally
relevant results to test takers. Compared to traditional tests where an
item response is linked to a single outcome scale, the cognitive inferences
made in CDA allow each item to measure multiple skills related to
student learning. Due to the complexity of interpreting and modeling
different aspects of cognitive skills, many approaches to modeling and
scoring examinee responses are available. Sinharay, Puhan, and
Haberman (2009) summarized three common features among different
methods of CDA:
(1) tests assess student mastery based on a cognitive model of skills; (2)
items probe student mastery on a pattern of skills expressed in a Q-
matrix; and (3) items probing the same pattern of skill mastery should
elicit a similar pattern of student responses.

3
An essential part of CDA development relies on the definition of a Q-

matrix. The Q-matrix is an item-by-attribute matrix used to describe the
skills probed by each item. For example, if a CDA is designed to
determine examinee mastery on four skills, and an item was designed to
elicit a correct response if the examinee has mastered the first and the
fourth skill, then the row corresponded to that item in the Q-matrix
would be expressed as {1,0,0,1}. The Q-matrix and the student response
patterns are used to calibrate the model parameters and provide students
with diagnostic results related to their cognitive problem-solving
strengths and weaknesses.
To ensure that CDA results provide the most accurate information to

examinees about their cognitive skills, the quality of CDA items must be
scrutinized. The evaluations of the claim that items are to probe a
specified set of skills have varied by the scope of how item-skill relations
are represented. Model-data fit has traditionally been used to evaluate
how items are aligned with construct of the skills based upon item
responses. Few studies have investigated the relations of item-skill
alignment. Wang, Shu, Shagn, and Xu (2015) have developed a measure
which allows the evaluation of skill-to-item fit based on the DINA model
that assumes a probabilistically scaled skill representation. To evaluate
item-model fit in CDA, items need to be evaluated beyond the
relationship of the correct responses on a particular item and single
outcome scores. Because each item is designed to provide student
mastery information on multiple skills, an item-model fit index is needed
to ensure item responses are aligned with the intended cognitive skills.
Evaluating Model-Fit for CDA

The rationale evaluating model fit in CDA can be considered in two
approaches, evaluating the fit with the expected psychometric properties
of the test items or evaluating the fit of responses with the blueprint of
skills. Existing developments tend to focus on the former approach. For
example, Jang (2005) compared total raw score distributions between
observed and predicted responses using the mean absolute difference
(MAD). Jangs approach to evaluating model-fit is akin to IRT model fit
approaches, where the level of fit is determined by total score differences
between the expected and examinee results. But with each correct
response of a CDA item linked to mastery on a vector of skills, evaluating
item-model fit for CDA need to consider the fit of an item with the pre-
requisite skills rather than a single test-level outcome.
Sinharay and Almond (2007) also developed an approach for evaluating

item fit for CDA by assuming that examinees categorized with the same
skill pattern should also have the same diagnostic outcome. With their

4
approach, the proportion correct response for examinees with the same
skill pattern is compared with the expected proportion predicted by the
cognitive model. Differences between the expected and observed correct
proportions are then summed across all skill patterns and weighted
proportionally by sample size. That is, model-fit for item j was defined
as:
2
. ( )
2 = ( ) ,

where is the number of examinees with skill pattern k, is the

number of examinees with skill pattern k that responded correctly to item
j, and is the product of the expected proportion of correct response for
pattern k multiplied by . Although this approach can be applied to
account for fit among multiple sets of skills, results rely on an expected
correct response rate of a given item for each skill pattern. As the
expected correct response for a given set of skill pattern is not readily
available, application of this method for determining model fit may be
problematic. Moreover, a poor sample representation of a skill pattern or
psychometrically indistinguishable skill patterns will also misestimate
item-model fit. One way to avoid the influence of misclassification on an
item-model fit measure for CDA is to comparatively evaluate items that
measure the same skills. That is, items measuring the same skills are
expected to elicit similar response patterns with one other.
Hierarchy Consistency Index (HCI)
One statistic developed specifically for CDA to evaluate person-model fit
is the Hierarchy Consistency Index (HCI; Cui & Leighton, 2009; Cui & Li,
2014; Cui & Mousavi, 2015). The HCI is a statistic for evaluating the fit of
the observed responses from an examinee with the expected responses
from a CDA model based on a comparison between the observed and
expected response vectors. The main assumption for the HCI is that if an
examinee gives a correct response to an item requiring a set of skills, then
the examinee is assumed to have mastered that set of skills and therefore
should also respond correctly to items that designed to measure those
skills. For example, if an examinee gives a correct response to an item
that requires the first and third skill in a CDA that assess four skills (or an
item with a skill pattern of [1,0,1,0] in the Q matrix), then the examinee is
also expected to respond correctly to items that probe the same set of
skills [1,0,1,0], or a subordinate or prerequisite of those skills (e.g., [1,0,0,0]
, [0,0,1,0]), which require skills should have been acquired. In this
manner, the number of misfitting responses across all items with their
corresponding subsets of skills is calculated for each examinee to
determine an index of person-fit.

5
Given I examinees were administered with J items, the HCI for examinee i
is calculated as:
.
2 =1 (1 )
= 1 , (1)

where Xj is the examinees scored response for item j, sj is an index set

that includes items requiring the subset of attributes measured by item j,
and Xg is the examinees scored response for item g. For example, if item j
is answered correctly, then all items that measure the attributes or a
subset of attributes probed by item j is represented by index set sj , where
g is an item index within sj. N is the number of comparisons made across
all sj. The HCI has a maximum of 1 and minimum of -1, where a high
positive HCI value represents good person-fit with the expected response
model.
The HCI is a useful index for analyzing person-fit across different types of
CDAs, as it requires only the use of the Q-matrix and examinee responses.
In this study, we modify the HCI to create an index for analyzing item-
model fit. Thus, the purpose of this study is twofold. First, we introduce
and define an item-model fit index called the item consistency index (ICI).
The ICI is used to evaluate the fit of an item related to the underlying
cognitive model used to make diagnostic inferences with that item.
Second, we present results from two studies to demonstrate both the
simulated and practical performance of the ICI across of host of testing
conditions typically found in diagnostic assessments.
Item Consistency Index (ICI)

As elaborated earlier, the HCI measures the proportion of misfitting
observed examinee responses relative to the expected examinee responses
on a diagnostic assessment. This principle can also be extended to
evaluate item-fit. With the HCI, the misfitting responses related to each
item is summed across all items for each examinee. As described in (1),
misfit for examinee i (mi) can be written as:
.
= (1 ) . (2)
Alternatively, to evaluate the misfit for item j, the number of misfitting

responses from the subset of item j can be summed across all examinees.
This modification can be written as:
= .
1 . , (3)
where is student s score (1 or 0) to item , and is student s score

(1 or 0) to item . Item g belongs to , a subset of items that require the

6
subset of skills measured by item j. In this manner, for a correct response

to item j for examinee i ( = 1), one can consider any incorrect responses
in to be a misfit for examinee i. The number of misfits is then summed
across all examinees.
It should be noted that the HCI only considers students correct responses
for analyzing misfit of a given item ( = 1). That is, misfit is calculated
against the required skills only when students have provided the correct
response. While this was adequate for analyzing misfit for person-fit,
analyzing item-fit against a cognitive model also requires comparisons to
be made when students respond to an item incorrectly ( = 0). As such,
an evaluation of item-fit needs to account for this alternative comparison.
For example, suppose an incorrect response was given on our exemplar
item that required the skill pattern of [1,0,1,0]. From this item response,
we could infer that the examinee does not possess all the necessary skills
required to solve this item and, therefore, should respond incorrectly to
all items that require the same skill pattern of [1,0,1,0]. Furthermore, the
examinee should also respond incorrectly to items that require more skills
than the current item (i.e., [1,1,1,0], [1,0,1,1], [1,1,1,1]). These items that
require the same skill or a more complex skill pattern can be
conceptualized as an alternative subset of item j ( ), and a correct
response in any of the items belonging to can be conceptualized as a
misfit. This outcome can be expressed as:
= .
(1 ). (4)
The set of alternative comparisons combined with comparisons from

correct responses form the numerator of the ICI. To maintain the same
scale of comparison with HCI, the numerator is then divided by the total
number of comparisons, which effectively transforms the outcome to a
proportion of misfit responses for item j. The proportion is then rescaled
to a maximum of 1 and a minimum of -1. The ICI for item is then given
as:
2 (1 )+ (1 )

= 1 , (5)

where is student s score (1 or 0) to item , is an index set that

includes items requiring the subset of attributes measured by item , is
student s score (1 or 0) to item where item belongs to , is an
index set that includes items requiring all, but not limited to, the
attributes measured by item , is student s score (1 or 0) to item
where item belongs to , and is the total number of comparisons for

7
item across all students.
To illustrate the calculation of the ICI, consider a hypothetical

administration of a CDA with 15 items and a Q-matrix presented in (6).
1 0 0 0
0 1 0 0
1 1 0 0
0 0 1 0
1 0 1 0
0 1 1 0
1 1 1 0
0 0 0 1 . (6)
1 0 0 1
0 1 0 1
1 1 0 1
0 0 1 1
1 0 1 1
0 1 1 1
1 1 1 1
Suppose this CDA of four skills was administered to an examinee who

produced the item response vector (0,0,0,0,0,1,1,0,0,0,0,0,0,0,0). That is, the
examinee responded correctly to items 6 and 7 only. To calculate the ICI
for item 6, we first consider that the examinee has responded to the item
correctly, therefore comparisons should be made with items that require
skills that are prerequisites to or same with the original item. In this case,
items 2 and 4 belong in 6 . Since both item responses were incorrect, two
comparisons were made (6 = 2) and two unexpected responses were
found (6 = 2) for this examinee. In addition, suppose we wanted to
calculate the ICI of item 2 for this examinee. The alternative subset ( )
will be needed since the examinee responded to the item incorrectly. For
this instance, seven items form the alternative subset for item 2 (2 =
{3,6,7,10,11,14,15}). Since the examinee responded correctly to item 6 and
7, there were two unexpected responses (2 = 2) from a total of seven
comparisons (2 = 7). In this manner, the number of unexpected
responses and comparisons are summed across all examinees and
rescaled to form the ICI.
To demonstrate the performance of this item-model fit index across a

variety of different testing situations, a simulation study was conducted
to determine the performance of ICI for detecting poor-fitting items.
Then, a real data study was conducted to demonstrate how the ICI can be
applied in operational testing situations a CDA in Mathematics.

8
Methods and Results
Study 1: Simulation Study

To evaluate how well the ICI can identify items that fit poorly relative to
their underlying cognitive model, a Monte-Carlo study was conducted by
simulating responses from a diagnostic test designed to measure seven
skills. To determine the performance of the ICI using simulated CDA
data, examinee responses were generated under the Bernoulli
distribution. In addition to generating examinee responses, different
testing conditions were manipulated to probe conditions that may occur
in a real CDA administration. Finally, to classify poor-fitting items using
ICI, a common evaluation criterion was used to determine which items
were fit poorly with the given cognitive model.
The simulation process is similar to the actual steps used in developing

CDAs (Gierl, Leighton, & Hunka, 2007), where cognitive model, items,
and responses were developed in a sequential manner. First, an existing
cognitive model from Cui and Leighton (2009) was used to guide the
simulation process. The cognitive model consists of seven skills, with 15
patterns of skill mastery identified as permissible. The patterns of
required skills for each item are expressed in the Q-matrix presented in
Table A1 in the Appendix. To generate examinee responses, examinees
were first assigned to an expected pattern of skill mastery from one of the
15 skill patterns. In addition to the 15 skill patterns, a null pattern
[0,0,0,0,0,0,0] was also used to represent examinees who did not master
any skills. In total, sixteen expected skill patterns are distributed equally
among the sample examinees. To simulate response for an examinee on a
given item, the examinees assigned skill pattern is compared with the
skills required by that item as indicated by the Q matrix. A probability of
correct response is assigned based on whether the examinee has all the
prerequisite skills of the item. Based on this assigned probability, the
examinees response to each item was generated using a Bernoulli
function.
To simulate the effectiveness of ICI under different testing conditions,

three factors were manipulated. First, the number of items representing
each skill pattern in the CDA was varied by three levels. If a CDA is
lengthened by including multiple items probing the same set of skills,
then the reliability of each corresponding skill measured is expected to
increase (Gierl, Cui, & Zhou, 2009). In our study, the number of items in
the CDA varied by one, two, or three items representing each possible
skill pattern. These three levels of variation on a total of 15 skill patterns
resulted in test lengths of 15, 30, and 45 items, respectively.

9
Second, unlike the related person-fit HCI which is independent of sample

size, the ICI is based on the proportion of misfit responses from all
examinees. Therefore, different sample sizes may affect the outcome of
the ICI. Three levels of sample sizes were manipulated: 800, 1600, and
2400. Since the 15 skill patterns and a null pattern are distributed equally
among the examinees, the numbers of examinees representing each skill
pattern are 50, 100, and 150, respectively.
Third, an important feature for an item-model fit index is to detect items

that fit poorly with the expected response determined by cognitive model.
This concept is contaminated when the ICI is influenced by misfitting
items related to the skills of the original item. To investigate whether the
proportion of poor-fitting items have an effect on the ICI, the proportion
of poor-fitting items were manipulated at three levels proportional to the
test length: 5%, 10% and 25%. In Cui and Leighton (2009), a well-fitting
item was deemed to have a 10% chance for slips, where an examinee
without mastery of the necessary skills will have a 10% chance of
responding correctly while an examinee who has mastered the necessary
skills will have a 90% chance of responding correctly. While there can be
many reasons for an item to fit poorly with the underlying cognitive
model (e.g., model misspecification, item quality, option availability),
generally a poor-fitting item yields a response that is aberrant from the
cognitive model. To simulate a poor-fitting item, items responses were
generated close to random. Table 1 contains the probabilities of correct
response given the level of item fit (good or poor fit) and whether the
examinee possesses the required set of skills. Taken together, three
manipulated factors with three levels each yielded a total of 27 conditions
as shown in Table A2 of the Appendix.
Table 1. Correct response probability given the level of item fit and whether
the examinee possesses the required set of skills
Item Fit
Required skills Good Poor
Present 0.9 0.6
Not present 0.1 0.4
To evaluate the effectiveness of the ICI for detecting poor-fitting items, a

criterion is needed for the ICI to differentiate between poor- and well-
fitting items. A classification approach was used to measure the precision
of the ICI in this study. A cut-score criterion, set at an ICI value of 0.5, was
used to illustrate the classification characteristics for poor-fitting items.
For example, if an item was calculated to have an ICI value of less than
0.5, then that item was deemed to fit poorly with the expected responses
from the cognitive model. This preliminary criterion for dichotomizing

10
item fit was needed because no point of comparison currently exists in

determining an appropriate level of fit with an existing cognitive model.
Further, an ICI value of 0.5 for any item translates to roughly 75% of the
responses on a given item fitting with the expected skill pattern as
defined by the cognitive model. Using this initial cut-score, we could then
classify items as poor- or well-fitting.
To ensure the classification results were consistently produced, each of

the 27 testing conditions was replicated 100 times. The dependent
variables for the simulation study included the average proportion of
correctly identified poor-fitting items and misclassification of well-fitting
items across all conditions. The simulation environment, the
implementation of the ICI, and the replication of results were
programmed in R (R Core Development Team, 2011), and are available
from the first author.
Table 2 contains a summary of the mean ICIs for each condition. The
mean ICIs were calculated separately for the poor- and well-fitting items.
The overall mean for poor-fitting items was 0.30 whereas the mean ICI for
well-fitting items was 0.53. Three observations must be noted from the
results in Table 2. First, test length tended to have a positive impact on the
values of ICI. For example, CDAs with only one item measuring each
skill pattern (i.e., test length=15) had consistently lower ICIs compared to
CDAs with two or three items measuring each skill (i.e., test length=30 or
45). Second, as expected, the magnitude of the mean ICI differences
between poor and well-fitting items tended to decrease when an increase
in poor-fitting items included in the ICI. Third, the means of ICI were
relatively stable across different sample sizes for each condition.
Table 2. Summary of the mean ICIs across the three variables manipulated in
the simulation study
Proportion of Mean ICI

Sample Test
Poor Fitting Poor Fitting Well-Fitting
Size Length
Items Items Items
800 5% 15 0.24 0.49
5% 30 0.22 0.57
5% 45 0.30 0.59
10% 15 0.31 0.48
10% 30 0.29 0.56
10% 45 0.38 0.58
25% 15 0.37 0.43
25% 30 0.29 0.56
25% 45 0.32 0.51

11
1600 5% 15 0.21 0.41

5% 30 0.22 0.56
5% 45 0.29 0.59
10% 15 0.27 0.44
10% 30 0.29 0.57
10% 45 0.38 0.58
25% 15 0.36 0.41
25% 30 0.29 0.56
25% 45 0.32 0.51
2400 5% 15 0.24 0.55
5% 30 0.23 0.58
5% 45 0.30 0.59
10% 15 0.32 0.53
10% 30 0.30 0.57
10% 45 0.38 0.58
25% 15 0.32 0.53
25% 30 0.29 0.56
25% 45 0.32 0.51
Items were also classified based on the cut-score criterion. This simulation
process was repeated 100 times, with the correct classification rate, or
power, being the likelihood of correctly identifying a poor-fitting item
using the ICI across the conditions in the simulation study. The power
values for the 27 conditions are shown in Table 3. The conditions with the
highest power were found in CDAs with the longest test-length (45),
specifically with conditions that had the largest proportion of poor-fitting
items (25%). Under those conditions, the highest power was 0.99,
meaning that for the ICI criterion of 0.50, 99% of all poor-fitting items
were correctly classified across 100 replications. The lowest power values
were found in conditions with the smallest sample size (800), where a
power of 0.67 was found for a 30-item CDA with 5% of poor-fitting items
and 1600 examinees.

12
Table 3. Power of ICI for identifying poor-fitting items
Test Sample Proportion of Poor-Fitting Items

Length Size 5% 10% 25%
15 800 0.68 0.76 0.92
1600 0.93 0.89 0.95
2400 0.79 0.79 0.92
30 800 0.67 0.73 0.79
1600 0.77 0.74 0.81
2400 0.73 0.72 0.79
45 800 0.76 0.80 0.99
1600 0.77 0.83 0.99
2400 0.76 0.81 0.99
Table 4 summarizes the likelihood of a well-fitting item being

misclassified by the ICI as a poor-fitting item in each condition. The
lowest misclassification rates were associated with CDAs that have the
longest test-length (45) and the smallest proportion of poor-fitting items
(5%). Under those conditions, the lowest misclassification rate was 15%.
The highest error rates were observed with the shortest test length (15),
where misclassification was 78%.
Taken together, the simulation study results highlight important trends

and outcomes that can be used to interpret how accurately the ICI
identifies poor-fitting items. The power values of ICI were erratic when
the number of items probing each skill pattern was small, but stabilized
as the number of items representing each skill pattern increased. For
example, each increase in test length resulted in a decrease in the
variation of power values among the same proportion of poor-fitting
items and between different sample sizes. This finding suggests that the
reliability of using the ICI to classify poor-fitting items is related to the
reliability of the CDA as a whole. Moreover, the proportions of
misclassification were approximately 2.5 times higher in CDAs with a
single item representing each test skills as compared to the other two
levels. This outcome further supports the conclusion that as skills are
measured more accurately, the ICI better distinguishes poor- from well-
fitting items.

13
Table 4. Misclassification rate of ICI in identifying well-fitting items
Test Sample Proportion of Well-Fitting Items

Length Size 5% 10% 25%
15 2400 0.28 0.35 0.66
1600 0.78 0.65 0.72
800 0.50 0.50 0.66
30 2400 0.16 0.20 0.22
1600 0.28 0.20 0.27
800 0.27 0.22 0.24
45 2400 0.15 0.18 0.33
1600 0.17 0.19 0.34
800 0.15 0.19 0.33
There were no obvious trends that the sample size manipulated across the
three levels yielded important differences among the power or
misclassification of well-fitting items. This finding suggests that the
sample sizes used in this study do not yield important ICI differences
across our study conditions. This outcome could also suggest that the
representation of approximately 50 examinees per skill pattern may be
sufficient for evaluation of the ICI.
When the proportion of poor-fitting items was manipulated, the power

increased with the proportion of poor-fitting items in the CDA, where the
overall power rose as the proportion of poor fitting item increased. An
increase of poor-fitting items also yielded more misclassification of well-
fitting items. This finding suggests that poor-fitting item responses
contribute to an overall decrease in the magnitude of ICI, where the
resulting errors are reflected using the classification criterion of 0.50.
Study 2: Use Case Application

The purpose of the second study is to demonstrate how the ICI can be
used to identify poor-fitting items in an operational CDA. The ICI was
used to evaluate item-model fit for a CDA program designed to assess
students knowledge and skills in Grade 3 mathematics. From this CDA
program, 324 students responded to an 18-item CDA (see Gierl, Alves, &
Taylor-Majeau, 2010).
The CDA we used was designed to evaluate student mastery for

subtraction skills. Each item was designed to yield specific diagnostic
information in a hierarchy of cognitive skills were the first skill was the
easiest (Subtraction of two consecutive 2 digit numbers) and the last skill
was most difficult (Subtraction of two 2 digit numbers using the digits 1

14
to 9 with regrouping). The CDA was developed as follows. First, a

cognitive model of task performance was created by specifying the
cognitive skills necessary to master subtraction in Grade 3. The domain of
subtraction was further specified into a set of six attributes related in a
linearly hierarchical manner by a group of subject matter experts. The
attributes produced a total of seven unique patterns of skill mastery (six
plus null). Three items were created by content experts to probe student
mastery on each attribute to ensure adequate representativeness of each
skill pattern resulting in eighteen items for this CDA. The test was
administered to students in 17 Grade 3 classrooms. A list of the attributes
and the Q-matrix for the 18-item CDA are shown in Table A3 and Table
A4 of the Appendix, respectively.
Three hundred and twenty four student responses were collected, which
would yield approximately 45 students per skill pattern if the patterns
were distributed equally across the skills. Participating teachers would
first instruct on the topics relevant to subtraction within their classrooms,
and then administer the CDA to students at a convenient time within
two-week of instruction. The CDA was delivered using an online
computer-based testing system. Students were presented with CDA
items that contain both an item stem to prompt for a typed-response and
an interactive multimedia component that provided additional
information for students to understand the item. From this
administration process, responses were collected, formatted and scored
dichotomously. As the participation of this CDA was voluntary, students
with greater than two missing responses were removed from the analysis
to minimize unmotivated responses (as the completion of the CDA was
not mandatory). For the purposes of demonstrating the ICI, only the
scored student responses were used.
The results are summarized first at the test level and then at the item
level. Overall, the results were ideal at the test level. The median HCI,
which is used to quantify the fit of the responses to the expected model of
response on a CDA, was 0.81. With a cut-off of 0.70 as the quality criterion
for CDA designs (Gierl, Alves, & Taylor-Marjeau, 2010), this result
suggests that the student responses fit with the expect model of response
for subtraction. As the purpose of this CDA is to identify non-mastery
students in order to refine and enhance instruction, the majority of
students were expected to master the CDA.
At the item level, Table 5 provides a summary of the results from the
subtraction CDA. The p-values of each item and the discrimination value
(i.e., point-biserial correlation) are presented along with the ICI values.
Three findings should be noted from these results. First, the ICI was not

15
correlated with either the difficulty or discrimination values. This result

supports the idea that item-model fit is summarizing a different outcome
from the classically defined notion of difficulty and discrimination.
Second, with items created in a principled manner, with three items
representing each skill pattern, the real data results support the results of
the simulation study. Further, as p-values decrease, ICI values increase
because the items change from measuring simple to more complex skills.
Third, using the cut-score criterion of 0.50 from the simulation study, only
three items were deemed to have poor item fit (Items 1, 2, 3). The poor ICI
values for these items may suggest a problem at the attribute level (see
Table A3 in the Appendix for the description of the skills assessed). It is
important to note that without the ICI conventional scoring and
psychometric approaches would not have identified issues of misfit at the
attribute level, where items one through three are performing nominally
at the item level. Although subject matter experts did not evaluate the
cognitive model in the light of the student results, a follow-up study may
find that a reorganization of the attributes may yield better fitting
responses.
Table 5. Summary of the results from the subtraction CDA
Attribute Item Number P-Value Discrimination ICI

1 1 0.76 0.58 0.22
2 0.78 0.87 0.39
3 0.80 0.96 0.46
2 4 0.84 0.89 0.64
5 0.87 1.11 0.72
6 0.85 0.94 0.65
3 7 0.86 1.06 0.76
8 0.80 0.68 0.65
9 0.84 1.01 0.75
4 10 0.77 0.79 0.73
11 0.72 0.78 0.72
12 0.75 0.82 0.73
5 13 0.74 0.82 0.78
14 0.77 0.92 0.79
15 0.79 0.98 0.80
6 16 0.35 0.56 0.81
17 0.34 0.57 0.81
18 0.33 0.53 0.80

16
Discussion
The purpose of this study is to introduce a statistic for determining item-
model fit with CDA. The item consistency index (ICI), an extension of a
person-fit index for CDA called the Hierarchy Consistency Index (HCI), is
a standardized outcome that measures the ratio of misfitting responses
relative to the total number of response across all examinees on a given
item. Similar to the HCI, the requirements for evaluating item-model fit
using the ICI is an item-by-attribute definition of skill mastery called the
Q-matrix in addition to the student response vectors. The ICI has a
maximum value of 1, which suggests all students responded identically to
an expected skill pattern, and a minimum value of -1, which suggests
item responses were the exact opposite to what the expected skill patterns
suggest. We present two use cases to demonstrate the properties of the
ICI under simulation. In addition, we demonstrate the applicability of the
ICI through the use of real data to highlight how the ICI can be applied to
identify poor-fitting items on a CDA. These two proof-of-concept
applications demonstrate how the ICI can be applied in the real world
and call for future studies to establish better evaluation criterion for the
ICI.
Results from the simulation study provided some general insights on how
the ICI performs as a method for detecting item misfit in CDA across a
range of testing conditions. Using a cut-score classification method to
determine poor-fitting items, the ICI was able to identify the majority of
the poor fitting items across different simulated conditions. Although the
item-model fit is described in a range by the ICI, the use of a cut-score to
classify poor fitting items provided a simple outcome to interpret for
evaluating how the ICI will perform in a given testing scenario. In
addition, results from the simulation study demonstrated a few
assumptions that must be met for the ICI to detect item misfit accurately.
The number of items used for each skill pattern and the total number of
poor fitting items were two features that affected ICI performance. The
implication from these findings demonstrate that although CDA demands
a different paradigm of scoring and statistical approaches, traditional
issues such as consistency of the responses for a given set of skill can still
be problematic in estimating item-model fit. From our simulation results,
we suggest the use of three items per attribute or more per skill pattern to
ensure adequate ICI detection. This finding is consistent with the
research in establishing an adequate reliability in measuring attributes of
skills (Gierl, Cui, & Zhou, 2009), where the authors stated that the idea of
a short yet diagnostic test will not likely yield results with sufficient
reliability.

17
Sinharay and Almond (2007) noted that tests with many poor-tting items
indicate a problem with the overall model, whereas tests with few poor-
fitting items indicate problems lie in the items themselves. In our
simulation, we demonstrated that the ICI will produce similar results,
where an increase of poor-fitting items in a CDA will lower the precision
of the ICI. This finding may be linked to the fact that as more poor-fitting
items are introduced, these items affect the fit of items requiring the same
set of skills leading to an overall decrease in magnitude of ICIs. Table A5
in the Appendix illustrates this effect, where the mean ICI for well- and
poor-fitting items under the 45-item simulation decreases as the
proportion of ICI increases. In sum, a rigorous and principled test
development process is needed for CDA to ensure all test items are
created with minimal deviation from the expected set of skills they were
designed to probe. Otherwise, poor model-fit results will lead to poor
diagnostic outcomes.
The second study provided a snapshot on the utility of the ICI when
applied to an operational CDA. Using a set of carefully designed CDA
items, the ICI detected three consecutive poor-fitting items at the
beginning of the assessment. This finding suggests that the ICI can not
only be used for evaluating item-model fit, but can also be used for
evaluating the consequences of test design at the item, attribute, or the
cognitive model level. In our example, the three items flagged as poor
fitting measure the same attribute revealing that the attribute may be mis-
specified in the cognitive model. In addition, the independence of ICI
from the difficulty and discrimination values suggest that item model-fit
for CDA provides a unique measure of how an item is able to accurately
predict performance. Hence, the definition of a good item for CDA may
not only be how well an item is able to distinguish poor-performers from
good-performers, but also how consistently an item can elicit responses
that match the expected response patterns specified in the cognitive
model (i.e., Q-matrix).
Item-model fit is challenging to measure, especially when cognitive

inferences are involved in the test design. Items have to be aligned with
the cognitive skills in the Q-matrix, skills have to be defined and
organized in a systematic manner, and examinee responses have to match
the expected skill patterns. The ICI can provide a source of evidence for
identifying poor-fitting items or poor models for Q-matrix based CDA.
Implications for Future Research

By introducing and demonstrating an item-model fit index for CDA, our
study provides two practical implications for the development of
diagnostic assessments in addition to a new measure of item-fit. The ICI

18
has the benefit of applicability, meaning that it can be used with a Q-

matrix based CDA for determining the relationship between items and
skills. Using the Q-matrix, item and examinee responses can be
compared to provide a measure of item model-fit. While research on
CDA has prompted a plethora of diagnostic scoring methods, one
common starting point is the use of the Q-matrix in defining the skills and
item requirements. Because item development, validation, and
administration all depend on the veracity of the Q-matrix, evidence for
validating the cognitive model is paramount. The ICI offers some initial
evidence that can be used for validating the definition of skills through
item response patterns to determine the relative fit between an item and
its set of required skills defined in the Q-matrix.
While the ICI provides a new statistical method for scrutinizing CDA
development, the second study highlighted the fact that the most crucial
part of a well-designed CDA remains with item development. The
importance of item development is, sometimes, neglected in CDA.
Although CDA scoring methods can account for different levels of skill
contributions, the link between how a skill is measured with how the skill
is presented in the form of an item remains largely a subjective
interpretation of the test developer and content specialist who create the
CDA. To reliably measure a set of skills, multiple items are needed. Yet
creating parallel items is often time consuming and expensive. Ensuring
that each item is uniformly developed with the same set of skills is one
critical activity in test development for CDA that ensures examinees
receive useful diagnostic feedback.The ICI is co-dependent with all items
requiring a related set of skills. Therefore, to ensure adequate item model-
fit, every item in the CDA must adhere to a high level of quality and
alignment relative to the expected skill the item is designed to measure.
Through introducing an item model-fit index for CDA, we have

demonstrated how such measure can be applied to identify problematic
items that are aberrant from the expected response model. This initial
study provides directions of future research as further investigation is
needed to apply and validate the use of this index. We also suggest three
directions of future research. First, more research is needed to ensure
different structures of knowledge represented by the Q-matrix can be
evaluated with the ICI to identify misfitting items. The number of
possible skill pattern representation increases exponentially as the
number of evaluated skills increases, therefore more research is needed to
ensure ICI provides an appropriate measure for different organization of
skills. Second, guidelines to interpret ICIs are needed so we can
accurately identify and distinguish adequate and problematic items. As
the ICI provides a scaled measure of item model-fit, interpretations of the

19
index has not yet been established and is required to determine the
adequacy threshold of item model-fit. Third, as the reliability of CDA
measures is highly dependent on the defined skills, more research is
needed to determine which model structure is ideal in the application of
the ICI. Our analysis relies on non-compensatory attributes, meaning
skills are independently defined, acquired and cannot be moderated by
existence of other skills. This will likely limit the ICI in measuring item fit
for testing complex skills but not for general skills such as elementary
mathematics. More research is needed to evaluate appropriate use cases
of the ICI.
References
Bock, R. (1972). Estimating item parameters and latent ability when responses
are scored in two or more nominal categories. Psychometrika, 37, 29-51.
Cui, Y., & Leighton, J. (2009). The hierarchy consistency index: Evaluating
person fit for cognitive diagnostic assessment. Journal of Educational
Measurement, 46(4), 429-449.
Cui, Y, & Li, J. C.-H. (2014). Evaluating person fit for cognitive diagnostic
assessment. Applied Psychological Measurement, 39, 223-238.
Cui, Y, & Mousavi, A. (2015). Explore the usefulness of person-fit analysis on
large scale assessment. International Journal of Testing, 15, 23-49.
Gierl, M., Leighton, J., & Hunka, S. (2007). Using the attribute hierarchy method
to make diagnostic inferences about examinees cognitive skills. In J.
Leighton & M. Gierl (Eds.), Cognitive diagnostic assessment for education:
Theory and applications (pp. 242-274). Cambridge, MA: Cambridge
University Press.
Gierl, M., Cui, Y., & Zhou, J. (2009). Reliability and attribute-based scoring in
cognitive diagnostic assessment. Journal of Educational Measurement, 46(3),
293-313.
Gierl, M., Alves, C., & Taylor-Majeau, R. (2010). Using the Attribute Hierarchy
Method to Make Diagnostic Inferences about Examinees Knowledge and
Skills in Mathematics: An Operational Implementation of Cognitive
Diagnostic Assessment. International Journal of Testing, 10(4), 318-341.
Jang, E. (2005). A validity narrative: Effects of reading skills diagnosis on
teaching and learning in the context of NG TOEFL (Doctoral
dissertation). University of Illinois at Urbana-Champaign, IL, USA.
Orlando, M., & Thissen, D. (2003). Further investigation of the performance of
S-X2: An item fit index for use with dichotomous item response theory
models. Applied Psychological Measurement, 27(4), 289-298.
R Development Core Team (2011). R: A language and environment for statistical
computing. R Foundation for Statistical Computing, Vienna, Austria. .
Reise, S. (1990). A Comparison of item- and person-fit methods of assessing
model-data fit in IRT. Applied Psychological Measurement, 14(2), 127-137.
Rost, J., & von Davier, M. (1994). A conditional item-fit index for Rasch models.
Applied Psychological Measurement, 18(2), 171-182.

20
Sinharay, S., Puhan, G., & Haberman, S. (2009, April). Reporting diagnostic
scores: Temptations, pitfalls, and some solutions. Paper presented at the
National Council on Measurement in Education, San Diego, CA, USA.
Sinharay, S., & Almond, R. (2007). Assessing fit of cognitive diagnostic models a
case study. Educational and Psychological Measurement. 67(2), 239-257.
Wang, C., Shu, Z., Shagn, Z., & Xu, G. (2015). Assessing Item-Level Fit for the
DINA Model. Applied Psychological Measurement, 1-14.
Yen, W. (1981). Using simulation results to choose a latent trait model. Applied
Psychological Measurement, 5, 245-262.
APPENDIX A
Table A1. The Q-matrix and skill patterns used for the simulation of CDA
responses
Skill
Pattern 1 2 3 4 5 6 7
1 1 0 0 0 0 0 0
2 1 1 0 0 0 0 0
3 1 1 1 0 0 0 0
4 1 1 0 1 0 0 0
5 1 1 1 1 0 0 0
6 1 1 0 1 1 0 0
7 1 1 1 1 1 0 0
8 1 1 0 1 0 1 0
9 1 1 1 1 0 1 0
10 1 1 0 1 1 1 0
11 1 1 1 1 1 1 0
12 1 1 0 1 0 1 1
13 1 1 1 1 0 1 1
14 1 1 0 1 1 1 1
15 1 1 1 1 1 1 1
Table A2. Variables manipulated in the simulation
Level
Conditions 1 2 3
Test length 15 30 45
Sample size 800 1600 2400
Proportion of poor-fitting items 5% 10% 25%

21
Table A3. Description of the skills assessed in the CDA for subtraction in Grade 3
Cognitive
Skill Descriptor: Apply a mental mathematics strategy to subtract
Attribute #
6 Two 2 digit numbers using the digits 1 to 9 with regrouping
5 Two 2 digit doubles (e.g., 24, 36, 48, 12)
4 Two 2 digit numbers where only the subtrahend is a multiple of 10
3 Ten from a 2 digit number
2 Two 2 digit numbers where the minuend and subtrahend are multiples of 10
1 Two consecutive 2 digit numbers (e.g., 11, 22, 33)
Table A4. Q-matrix of the CDA for subtraction in Grade 3
Skill
Pattern 1 2 3 4 5 6
1 1 0 0 0 0 0
2 1 0 0 0 0 0
3 1 0 0 0 0 0
4 1 1 0 0 0 0
5 1 1 0 0 0 0
6 1 1 0 0 0 0
7 1 1 1 0 0 0
8 1 1 1 0 0 0
9 1 1 1 0 0 0
10 1 1 1 1 0 0
11 1 1 1 1 0 0
12 1 1 1 1 0 0
13 1 1 1 1 1 0
14 1 1 1 1 1 0
15 1 1 1 1 1 0
16 1 1 1 1 1 1
17 1 1 1 1 1 1
18 1 1 1 1 1 1
Table A5. Summary of the mean ICI in extreme situations when n=2400
Proportion of Poor-Fitting Items

Item Quality
0% 25% 50% 100%
Well-Fitting Items 0.61 0.49 0.39 n/a
Poor-Fitting Items n/a 0.33 0.28 0.15

22

Vol. 16, No. 1, pp. 22-37, January 2017
Factors That Determine Accounting Anxiety

Among Users of English as a Second Language
Within an International MBA Program
Alexander Franco and Scott S. Roach

Stamford International University, Graduate School of Business
Bangkok, Thailand
Abstract. The primary goal of this study was to determine the factors
related to accounting anxiety among MBA students who utilize English
as a second language (ESL). The analysis included components within
the learning environment and also differentiations as to demographic
variables such as gender, age, ethnicity, and any prior undergraduate
exposure to the study of accounting. A secondary goal of the study was
to determine perception of anxiety among ESL students in an MBA
program regarding quantitative courses as opposed to qualitative
courses. Finally, the study examined different strategies used by ESL
students to deal with accounting anxiety. The study found that there
were significant differences in accounting anxiety based on gender,
ethnicity, and exposure to undergraduate accounting. However, age was
not a factor. In addition, the study supported the hypothesis that there is
a negative relationship between levels of English proficiency and
accounting anxiety. It also supported the hypothesis that there is a
positive relationship between the levels of anxiety with classes involving
quantitative subject matter. Finally, the study rejected significant
differences in coping strategies by levels of accounting anxiety.
Keywords: accounting; accounting anxiety; English as second language

(ESL); language anxiety, strategies regarding accounting anxiety
Introduction
Within the context of globalization, English has become the lingua franca of the
business world, a transnational instrument vital in both a local and a global
context (Buripakdi, 2014; Easthope, 1999). The study of language anxiety among
students using English as foreign language has been steadily growing for the
past three decades (Horwitz, 1991; Kao & Craigie, 2013; Kondo & Yang, 2004;
Mahmoodzadeh 2012, Marwan, 2007; Ozturk & Gurbuz, 2014; Semmar, 2010;
Wang, 2010). During this period, a body of work has also been developed that
focused on anxiety suffered by students while studying accounting, although

23
none of the studies specifically examined a student body primarily consisting of

ESL students (Ameen, Guffey, & Jackson, 2002; Borja, 2003; Buckhaults & Fisher,
2011; Chen, Hsu, & Chen, 2013; Clark & Schwartz, 1989; Dull, Schleifer, &
McMillan, 2015; Duman, Apak, Yucenursen, & Peker, 2014; Ghaderi & Salehi,
2011; Malgwi, 2004; Uyar & Gungormus, 2011).
This study sought to investigate those factors that are related to varying anxiety
levels among students of accounting who are challenged with learning this
quantitative subject and its nomenclature while utilizing English as a second
language. The first section of this paper presents a review of related material on
accounting anxiety and proposes the hypotheses to be tested. The second part of
this paper provides a discussion of the research methodology and analysis of the
data collected. The final part presents utilitarian suggestions for minimizing
anxiety by ESL students as they learn accounting, as well as recommendations
for future research.
1. Literature Review
Academic anxiety, within a pedagogical context, can best be seen as emotional
state that is not inherent, but which is situational and can be treated by
creating an effective association between teaching and receiving apprehension
(Chu & Spires, 1991; Malgwi, 2004). Anxiety as to the learning of accounting at a
level of higher education has been based on students perceptions that the
nomenclature of the subject is akin to learning a new language (Borjas, 2003).
Further, the knowledge base for this subject is perceived as being extensive and
usually there is a corresponding apprehension that the period of time necessary
to properly comprehend the principles and application of accounting is
inadequate (Malgwi, 2004).
Previous studies suggest that differences in anxiety levels regarding the study of
technical material may related to variables such as gender (Todman, 2000), age,
background experience or exposure to the subject being studied (Chu & Spires,
1991; McIlroy, Bunting, Tierney, & Gordon, 2001; Towell & Lauer, 2001) or
nationality/ethnicity (Burkett, Compton, & Burkett, 2001; Rosen & Weil, 1995).
Based on this, the following hypotheses were examined:
H1: There will be differences in accounting anxiety levels of ESL students

in an international MBA program across different demographic
groups.
H1a: There will be differences in accounting anxiety levels of ESL

students in an international MBA program across age groups.
H1b: There will be differences in accounting anxiety levels of ESL
students in an international MBA program across genders.
H1c: There will be differences in accounting anxiety levels of ESL
students in an international MBA program across different
ethnic groups.

24
H2: There will be differences in accounting anxiety levels of ESL students

in an international MBA program for those students who took an
undergraduate accounting course as opposed to those who did not.
Among ESL students, the level of anxiety in learning technical subjects and in
communication apprehension has been tied to the degree of their proficiency in
the use of the English language (Casado & Dereshiswsky, 2004; Horwitz,
Horwitz, & Cope, 1986; Marwan, 2007; Onwuegbuzie, Bailey, & Daley, 1999;
Pappamihiel, 2002). Therefore, H3 was proposed:
H3: There will be a negative relationship between level of English

proficiency and accounting anxiety for ESL students enrolled in an
international MBA program.
The degree of quantification in a course of study impacts on the level of anxiety

experienced by students (Kao & Craigie, 2013; Kondo & Yang, 2004; Rosen &
Weil, 1995; Todman, 2000). Kondo & Yang (2004) devised a typology of
strategies (5 strategy categories from 70 basic tactics) that ESL students use to
cope with language anxiety. The strategies include peer seeking, positive
thinking, preparation, and resignation. From this, the following hypotheses were
proposed for testing:
H4: There will be a positive relationship between level of anxiety with

classes involving quantitative subject matter and accounting anxiety
for ESL students enrolled in an international MBA program.
H5: There will be differences in the accounting anxiety associated with the
coping strategy selected by ESL students enrolled in an international
MBA program.
2. Research Methodology and Findings

2.1 Sample
The population studied was an international university in Thailand with an
MBA student body consisting of 380 ESL students which were 57% female, 43%
male; 64% were Thai and 36% were non-Thai. As per Krejcie and Morgans
(1970) table of sample size determination, a sample population of 190 was
calculated for this study. The sample consisted of 107 females (56% of the sample
population), and 83 males (44%). Within the sample, 105 (55.3%) were Thais, 16
(8.4%) were Thai of Chinese lineage (1st and 2nd generations) and 69 (36.3%) were
non-Thai.
2.2 Instrument
A self-administered questionnaire was used with 15 accounting-focused, Likert
scale questions, many which were modifications from the Horowitz et al. (1986)
Foreign Language Classroom Anxiety Scale (FLCAS), a survey that has been
used in several studies (Argaman & Abu-Rabia, 2002; Casado & Dershiwsky,
2004; Marwan, 2007; Matsuda & Gobel, 2004; Semmar, 2010; Yashima, 2002). All
scales had a Cronbach alpha internal reliability score of over .80, indicating
consistency (Hair, Black, Babin, & Anderson, 2010; Sekaran, 2000; Tavakol &

25
Dennick, 2011). The questionnaire also tested coping strategies by incorporating

the Foreign Language Anxiety Coping Scale, which was designed by Kondo and
Wang (2004). This scale was assessed to have an alpha coefficient of .91
(Marwan, 2007), demonstrating high internal reliability.
The questionnaire consisted of a forced, 4-point Likert scale from strongly

agree to strongly disagree. A neutral option (e.g., not sure) was
deliberately avoided because of cultural traits within Thai society that inhibit the
motivation to express personal opinion: a strong hierarchical system with high
power-distance and kreng jai the culturally operationalized practice of avoiding
the display of emotion or asserting ones opinion (Holmes, Tangtongtavy, &
Tomizawa, 2003; Johnson & Morgan, 2016; Suntaree, 1990). The questionnaire
was translated into Thai for Thai students (and translated back into English to
assure accuracy) in order to maximize effective feedback (Behling & Law, 2000;
Harkness, Van de Vijer, & Mohler, 2002; Domyei & Taguchi, 2009). An English
language version was distributed to non-Thai ESL students. The questionnaire
was administered during a six-month period by the same lecturer who taught
the only accounting course (a core course) required by the universitys MBA
program. The actual day in which the questionnaire was administered was the
first day of each starting class during that period.
2.3 Findings
The first hypothesis proposed that there would be differences in accounting
anxiety levels across groups defined by the demographic variables of age,
gender and ethnicity. Descriptives for the first of these three demographic
factors are presented below in Table 1. As shown in the table, the mean
accounting anxiety rating declines consistently across the four age groups.
Table 1: Descriptive Analysis of Accounting Anxiety Ratings by Age Group*
Age Group N Min Max M SD
18-22 58 1 4 3.17 .920

23-25 48 1 4 2.94 .836
26-30 46 1 4 2.91 .784
30 + 38 1 4 2.74 .724
Total 190 2.96 .838
*Where 1 = Strongly Disagree and 4 = Strongly Agree with the statement: Taking
an accounting class gives me high anxiety (i.e., feeling of stress, fear).
In order to test whether this decline was statistically significant, a one-way

ANOVA was performed to analyze differences in accounting anxiety ratings
across the age groups. The results are displayed in Table 2 below. Results
indicate no significant difference across the four age groups for accounting
anxiety, F (3, 186) = 2.242, p = .085. Therefore, Hypothesis 1a is rejected.

26
Table 2: One-Way Analysis of Variance of Accounting Anxiety Scores by Age

Group
Source df SS MS F p
Between Groups 3 4.633 1.544 2.242 .085
Within Groups 186 128.109 .689
Total 189 132.742
The second part of this hypothesis proposed differences in accounting anxiety

across gender groups. Descriptive statistics by gender are presented below in
Table 3. As shown in the Table, the mean female accounting anxiety rating is
slightly higher than the mean rating for males.
Table 3: Descriptive Analysis of Accounting Anxiety Ratings by Gender*
Gender N Min Max M SD
Male 83 1 4 2.77 .860
Female 107 1 4 3.11 .793
Total 190 2.96 .838
an accounting class gives me high anxiety (feeling of stress, fear).
In order to test whether this difference was significant, a t-test was conducted.
Results of that test are provided in Table 4, below. The results indicate a
significant difference in scores with women reporting significantly higher levels
of accounting anxiety (M=3.11, SD= .793) as compared to males (M=2.77, SD=
.860), t (188) = -2.834, p = .005. Therefore, Hypothesis 1b is supported.
Table 4: Comparison of Anxiety Ratings by Gender*
Gender N Mean SD t df p 95% Confidence

Interval
Male 83 2.77 .860

Female 107 3.11 .793
Total 190 2.96 .838 -2.834 188 .005 -.578 -.101

27
The third part of Hypothesis 1 proposed that there would be differences in

accounting anxiety ratings across different ethnic groups. Table 5 provides the
descriptive statistics associated with the three ethnic groups that were analyzed.
Table 5: Descriptive Analysis of Accounting Anxiety Ratings by Ethnic Group*
Ethnic Group N Min Max M SD
Thai of 18 2 4 3.13 .619

Chinese
Thai 106 1 4 3.09 .810
Not Thai 69 1 4 2.74 .885
Total 190 2.96 .838
Testing for significant differences in accounting anxiety ratings across the three
ethnic groups was conducted with a one-way ANOVA. Findings of this analysis
are presented in Table 6 below. As depicted in the table, there was a statistically
significant difference between the ethnic groups as determined by the one-way
ANOVA F (2, 187) = 4.010, p = .020. Therefore, Hypothesis 1c is supported. A
Tukey post hoc test was then performed revealing that the Thai group had
statistically significant higher ratings of accounting anxiety as compared with
the Other Than Thai group (3.09 + .810, p = .020).
In sum, Hypothesis 1 proposed that there would be differences across the

demographic groups of age, gender and ethnicity. Upon testing, the age portion
of Hypothesis 1 was rejected, the gender differences hypothesis was supported
and differences in accounting anxiety were found to exist between Thai and
the Other Than Thai groups.
Table 6: One-Way Analysis of Variance of Accounting Anxiety Scores by Ethnic

Group
Source df SS MS F p
Between Groups 2 5.459 2.730 4.010 .020

Total 189 132.742
Hypothesis 2 proposed that there would be differences in accounting anxiety

levels for those ESL students that had taken an undergraduate accounting course

28
and those who had not. Descriptive statistics for these two groups are presented
in Table 7.
Table 7: Descriptive Analysis of Accounting Anxiety Ratings by Whether or Not

Student Had an Undergraduate Accounting Class*
Undergrad N Min Max M SD

Class
Yes 96 1 4 2.79 .882

No 94 1 4 3.14 .756
Total 190 2.96 .838
*Where 1 = Strongly Disagree and 4 = Strongly Agree with the statement:

Taking an accounting class gives me high anxiety (i.e., feeling of stress, fear).
As shown in the table, those students who reported having had an

undergraduate class in accounting had lower mean accounting anxiety ratings.
To test to see if this difference was significant, a t-test was run on the accounting
anxiety ratings between the two groups. The results of this test are below
reported in Table 8. The results indicate a significant difference in scores with
ESL students in the group that did have an undergraduate accounting course
reporting significantly lower levels of accounting anxiety (M=2.79, SD= .882) as
compared to those students who had not had an undergraduate accounting
course (M=3.14, SD= .7.56), t (188) = -2.271, p = .004. Therefore, Hypothesis 2 is
supported.
Table 8: Comparison of Anxiety Ratings by Whether or Not Student Had Taken

an Undergraduate Accounting Class*
Undergrad N Mean SD t df p 95% Confidence

Class Interval
Yes 96 2.79 .882

No 94 3.14 .756
Total 190 2.96 .838 -2.834 188 .004 -.582 -.111

The third hypothesis proposed that there is a significant negative relationship

between English proficiency and accounting anxiety for ESL students. Self-
reported English proficiency levels ranged from 1, Bad to 5, Excellent (N =

29
190; M = 3.54; SD = .801). Ratings of accounting anxiety ranged from 1, Strongly

Disagree to 4, Strongly Agree with the statement Taking an accounting class
gives me high anxiety (i.e., feeling of stress, fear) (N = 190; M = 2.96; SD = .838).
A simple regression analysis showed that the level of English proficiency
significantly affected ratings of accounting anxiety. Results of the analysis are
presented in Table 9, below. The higher the English proficiency ratings, the
lower the accounting anxiety ratings (t = -2.899; p < .001). Therefore, Hypothesis
3 is supported. However, the R2 =.043, so the predictive power of the model is
quite low.
Table 9: Summary of the Simple Regression Analysis for English Proficiency and
Accounting Anxiety
Variable B SE(B) t p
English -.216 .075 -.207 -2.899 .004

Proficiency
R2 =.043
Hypothesis 4 suggests a positive relationship between classes involving

quantitative subject matter and accounting anxiety ratings. This was based on
self-reported anxiety with classes that are quantitatively based and which was
ranged from 1, Strongly Disagree to 4, Strongly Agree with the statement, I
get anxiety from an accounting class because of the numbers involved (N = 190;
M = 2.75; SD = .913). A simple regression analysis was used to test this
relationship. The results of this analysis are presented below in Table 10. These
results indicate that as a persons anxiety with quantitatively based classes
increases so does their ratings of accounting anxiety ratings (t = 10.386; p < .001).
Therefore, Hypothesis 4 is supported. The R2 =.365 so the independent variable
(anxiety with quantitative based classes) explains 36.5% of the variance in the
dependent variable, accounting anxiety.
Table 10: Summary of the Simple Regression Analysis for Quantitative Class
Anxiety and Accounting Anxiety
Variable B SE(B) t p
English .555 .053 .604 10.386 < .001

Proficiency
R2 =.365
The final hypothesis suggests that differences in accounting anxiety will be

associated with the coping strategy employed by ESL students. As displayed in
Table 11, the means do differ across the various strategies employed by the
students. This is particularly true for Positive Thinking and for Peer Seeking
which fall at the lowest and highest levels of accounting anxiety, respectively. In
order to determine whether these differences were significant, a one-way

30
ANOVA was performed to examine group differences in accounting anxiety

scores. The results of this analysis are reported in Table 12.
Table 11: Descriptive Analysis of Accounting Anxiety Ratings by Coping

Strategy*
Coping Strategy N Min Max M SD
Preparation 100 1 4 2.97 .758

Relaxation 22 2 4 2.91 .921
Positive 47 1 4 2.79 .977
Thinking
Peer Seeking 21 2 4 3.38 .669
Total 190 2.96 .838

Table 12: One-Way Analysis of Variance of Accounting Anxiety Scores by

Coping Strategy
Source df SS MS F p
Between Groups 3 5.189 1.730 2.522 .059

Total 189 132.742
As shown in Table 12, the results indicate no significant difference across the
four coping strategy groups for accounting anxiety, F (3, 186) = 2.522, p = .059.
Therefore, Hypothesis 5 is rejected.
A summary of the findings of this study is provided below in Table 13. There
was support for two of the demographic factors and varied levels of accounting
anxiety (gender and ethnicity) but differences by age was rejected. Having taken
an undergraduate course in accounting significantly reduced accounting
anxiety. In addition, English proficiency was shown to be negatively related to
higher levels of accounting anxiety. Anxiety toward courses with quantitative
content was positively related to accounting anxiety. Coping strategies
employed by students did not vary significantly by level of accounting anxiety.

31
Table 13: Summary of Study Findings
Hypothesis SS
H1a Differences in Accounting Anxiety by Age Rejected

H1b Differences in Accounting Anxiety by Gender Supported
H1c Differences in Accounting Anxiety by Ethnicity Supported
H2 Differences in Accounting Anxiety by Undergraduate Supported
Accounting
H3 Negative Relationship between English Proficiency Supported
and Accounting Anxiety
H4 Positive Relationship between Anxiety for Supported
Quantitative Courses and Accounting Anxiety
H5 Differences in Coping Strategy by Level of Rejected
Accounting Anxiety
As a part of this study, the ESL students were requested to rate the various core
subjects and work on their thesis in terms of difficulty of learning the subject in
English. Table 14 presents results of these questions. As shown in the table, those
subjects that are based on a primarily quantitative content (accounting M = 2.09
SD = .733; and finance M = 2.22 SD = .751) were rated as more difficult than
those subjects that are more theoretical in nature (marketing M = 2.97 SD = .6.29
and management M = 2.94 SD = .6.72). The two subject areas that employ both
quantitative analysis and theory (research methods M = 2.55 SD = .780 and
thesis M = 2.25 SD = .8.77) were rated in the middle in terms of difficulty with
thesis being closer to the quantitative subjects.
Table 14: Difficulty of Studying Subjects in English Ratings by Percentage*
Subject Very Somewhat Somewhat Very Mean Standard

Difficult Difficult Easy Easy Deviation
Accounting 16.8 63.2 14.2 5.8 2.09 .733

Finance 12.6 59.5 21.1 6.8 2.22 .751
Marketing 1.1 17.9 63.7 17.4 2.97 .629
Research 6.8 42.1 40.0 11.0 2.55 .780
Methods
Management 1.6 21.1 59.5 17.9 2.94 .672
Thesis 19.5 45.8 25.3 9.5 2.25 .877
*Where 1 = Very Difficult and 4 = Very Easy
3. Conclusion and Recommendations

Though the findings did not support a significant statistical difference in
accounting anxiety by age, it did reveal significant differences for the factors of
gender, ethnicity, and exposure to undergraduate accounting. The findings also

32
supported a negative relationship between levels of English proficiency and

accounting anxiety as well as a positive relationship between the levels of
accounting anxiety and the quantitative nature of business courses. Finally, the
study did not find significance difference between levels of accounting anxiety
and the selecting of coping strategies for such anxiety. These mixed results
conform within the disparity within the studies discussed in the literature
review. However, it is important to emphasize that this study differs from most
of the literature review studies in that it examines anxiety within the context of
learning the subject of accounting by using English as a second language. Within
that context, Franco (2016) suggested the following eight tactical components for
lowering anxiety in general, and accounting anxiety in particular, within an ESL
environment:
1. Initial assessment of students. This can be done in two ways: On the

first day of class, he student fills out a simple one-page form that
requests information on the students knowledge of the subject matter
but also asks the student to evaluate himself/herself as to English
proficiency by way of a Likert scale. The form should also include
questions like, Who do you admire most? Each student is then asked to
introduce himself/herself to the class and verbally answer some of the
questions on the form. This allows the teacher make initial assessments
of each student (written and oral presentations) as well as obtain a
general assessment of the level of English proficiency of the group in
order to adapt the course accordingly. Secondarily, the assessment form
would allow the instructor to determine any previous knowledge of
accounting by the students as a result of undergraduate courses and or
work-related experience. This allows making a better initial
determination as to the speed the accounting course should take.
2. Vocabulary Buildup and Word Dissection. Absorption of the

nomenclature of accounting is difficult enough for those tackling the
subject in their native language. In an ESL environment, it is vital that
students be introduced to key words and phrases even if this requires a
discussion of such vocabulary before beginning the lecture. The lecturer
should reinforce the meaning of key terms/phrases and provide a
context within which they have meaning. Without a focus on building up
the vocabulary for a particular lecture, there is a stronger likelihood that
some students will not be able to follow the narrative. Frustration will set
in as key terms, not properly absorbed by the student, will become
obstacles in comprehending the narrative and context of the discussion.
The lecturer should write key words and phrases on the board, along
with their definitions, and require the students to them write down. This
creates a mental imprimatur since students are more likely to remember
a word if they physically see it and work with it. Grammatical analysis of
a word can be performed by dissecting it and presenting its
grammatical variations. For example, a word like accountability
defined as being held responsible for something can be broken up from

33
its noun form to its adjective accountable and the verb phrase to
account for. This dissection, along with the lecturers use of the word
within a context and the solicited use of the word from students in a
sentence or two, allows the students to chew on the word or phrase
and obtain an adequate comfort level of understanding.
3. Concept Checking. Concept checking involves asking questions to

students to test the depth of their knowledge of newly accumulated
information. These questions are sometime difficult to construct and
some see their creation as more of an art form than a skill. The checking
of concepts is developed in part, by anticipating, beforehand, concept
checking questions you might use. However, it is primarily developed
through practice and experience thinking on your feet. Concept
checking should be used throughout the lecture. In some situations, you
can repeat a concept checking question that was successfully used in the
same lecture in the past. However, the teacher will have to be conscious
of coming up with new and pertinent concept checking questions within
the serendipitous dynamics of the classroom discussion. This is art form
more than anything else and the interaction of concept checking allows
for a good balance between teacher talking time and student talking
time.
Concept checking is not open questioning. Avoid questions such as, Do

you understand? that can merely be answered with yes or no. If your
narrative flow causes you to create a question that can be answered in
that way, follow up with why? Marry students in the class to come
up with financial solutions to a marriage or business partnership
problem. This personalizes the class analysis and gets students to interact
with each other. The teacher should avoid adding unfamiliar vocabulary
when working through concept checking. This is part of a self-imposed
discipline that is always conscious of the ESL experience and the
appropriate implementation of knowledge within that setting.
4. Eliciting. Eliciting can be simply defined as asking for answers

(information) instead of just giving out the information. In a learner-
centered classroom this provides for constant interaction. Eliciting
should be performed by choosing students not by depending on
volunteers (i.e., the alpha few that will dominate classroom discussions
if the teachers allows for this). Choosing students also keeps all students
alert (on their toes) and avoids the awkward situation where a
question asked to the entire class is met with silence. Even if the student
chosen by the lecturer does not have an answer, he/she will usually
provide some response that the teacher can build on. Letting everyone
know that they can and will be called on helps to identify students who
are falling behind (stragglers).
Pace yourself in your elicitations. Avoid repetition, condescension, and

the need to turn everything into a question. Avoid asking questions

34
about material that has already been covered unless you are conducting a
review for an examination.
5. Pacing. Even while abiding by the institutions guidelines, rules, and

expectations, the lecturer remains the master of his domain within
his/her classroom. Lectures, homework, assignments, projects, and
examinations are all the creations of the teacher. Especially in the ESL
environment, the teacher must recognize the need to alter the pace of a
lecture and even the pace of the entire course. Slow down when red flags
and bells are going off. This is particularly true regarding subject matter
that is built in layers (like accounting) where the next layer requires that
you fundamentally understand the prior layer(s) of knowledge. If the
lecturer keeps moving just to follow a schedule of his own design (e.g., a
stated calendar on the syllabus), the results will be poor performances on
the midterm exam. At that point the lecturer will have to go back to
basics or risk moving forward and witnessing poor performances again,
this time on the final exam. Almost nothing is more nonsensical for a
lecturer than shackling himself/herself to rigid or impractical time
restraints that were self-created and self-imposed.
6. Monitoring. In an ESL, learner-center environment the interaction

should not only be verbal but also physical. The lecturer should not hide
behind a podium or desk. Instead, the lecturer should be moving around
to keep the students alert, away from their phones, or Facebook on their
laptops. Moving amongst the students also allows for better eliciting,
marrying students, and concept checking. When students are
performing an in-class exercise (e.g., accounting), the teacher should
move from one student to the next to see if the student is stuck on a word
or a concept. Sometimes they are stuck on a verb or some other word
within an explanatory or instructional text. An explanation or
clarification at that moment is crucial. Otherwise, the student gets stuck
and needlessly frustrated at the very start and gives up on solving the
problem or resorts to looking to the student next to him/her for the
answer. Sometimes a student who is stuck asks another student for an
explanation. When a teacher sees this, he/she should step in, do the
explanation, and provide further guidance.
7. Use of Paper. ESL students need to see physical words, not just hear
them. They need a physical imprimatur. Power points have limited
impact, unless the students have the physical text of the power point
slides in front of them. If the lecturer gives handouts of core material
(material that will be tested) the student has the pertinent text and can
make notes including the meaning of the word in their native language.
For test preparation, ESL students tend to rely on paper since they are
not only looking at concepts but also the specific words that constitute
the definition or explanation of that concept.

35
8. Feedback. It is nonsensical to wait until the student evaluations to obtain

feedback on how well ESL students were coping with their English
comprehension in a business course. Feedback is best solicited from the
first day of the course, on an individual basis when the student feels
he/she can be more candid or less embarrassed (i.e., no disclosure in
public). Feedback can be attained before and after class, during breaks,
by email, and at office hours. The teacher can also specifically approach
students that he/she feels are having trouble. Individual feedback, in the
aggregate, can help the teacher determine the overall situation in the
class and who the stragglers are.
The continuation of globalization guarantees the internalization of higher

education business studies using English as the commercial lingua franca. This
study focused specifically on accounting anxiety experienced by ESL students. A
body of literature needs to be created to specifically address accounting anxiety
within the context of ESL education.
References
Ameen, E.C., Guffey, D. M., & Jackson, C. (2002). Evidence of teaching anxiety among
educators. Journal of Education for Business, September/October, 16-22.
Argaman, O., & Abu-Rabia, S. (2002). The influence of language anxiety on English
teading and writing tasks among Hebrew speakers. Language, Culture, and
Curriculum, 15(2), 143-160.
Behling, O., & Law. K. S. (2000). Translating questionnaires and other research instruments:
Problems and solutions. Thousand Oaks, CA: SAGE Publications, Inc.
Borja, P. M. (2003). So youve been asked to teach principles of accounting. Business
Education Forum, 58(2), 30-32.
Buckhaults, J., & Fisher, D. (2011). Trends in accounting education: Decreasing
accounting anxiety and promoting new methods. Journal of Education for
Business, 86, 31-35.
Buripakdi, A. (2014). Hegemonic English, standard Thai, and narratives of the subaltern
in Thailand. In P. Liamputtong (Ed), Contemporary Socio-cultural and Political
Perspectives In Thailand (pp. 95-109). Dordrecht, Netherlands: Springer.
Burkett, W. H., Compton, D.M., & Burkett, G.G. (2001). An examination of computer
attitudes, anxieties, and aversions among diverse college populations: Issues
central to understanding information sciences in the new millennium. Informing
Science 4(3), 77- 85.
Casado, M. A., & Dereshiswsky, M. I. (2004). Effect of educational strategies on anxiety
in the second language. College Student Journal, 38(1), 23-35.
Chen, B. H., Hsu, M., & Chen, M. (2013). The relationship between learning attitude and
anxiety in accounting classes: The case of hospitality management university
students in Taiwan. Qual Quant 47, 2815-2827.
Chu, P. C., & Spires, E. E. (1991). Validating the computer anxiety rating scale: Effects of
cognitive style and computer courses on computer anxiety. Computers in Human
Behavior 7(1/2), 7-21.
Clark, C. E., & Schwartz, B. N. (1989). Accounting anxiety: An experiment to determine
the effects of an intervention on anxiety levels and achievement of introductory
accounting students. Journal of Accounting Education 7, 149-169.
Domyei, Z., & Taguchi, T. (2009). Questionnaires in second language research: Construction,
administration, and processing (2nd ed.). London: Routledge.

36
Dull, R. B., Schleifer, L. F., & McMillan, J. J. (2015). Achievement goal theory: The
relationship of accounting students goal orientations with self-efficacy, anxiety,
and achievement. Accounting Education: An International Journal 24(2), 152-174.
Duman, H., Apak, I., Yucenursen, M, & Peker, A. A. (2015). Determining the anxieties of
accounting education students: A sample of Aksaray University. Procedia Social
and Behavioral Sciences 174, 1834-1840.
Eastrope, A. (1999). Englishness and national culture. London: Routledge.
Franco, A. (2016). MBA instructors guide for teaching business to ESL students.
Unpublished manuscript, Bangkok, Thailand.
Ghaderi, A. R., & Salehi, M. (2011). A study of the level of self-efficacy, depression and
anxiety between accounting and management students: Iranian evidence. World
Applied Sciences Journal 12(8), 1299-1306.
Hair, J. F. Jr., Black, W. C., Babin, B.J., & Anderson, R. E. (2010). Multivariate data analysis:
a global perspective (7th ed.). Saddle River, NJ: Prentice-Hall International.
Harkness, J. A., van de Vijver, F. J. R., & Mohler, P. P. (2002). Cross-cultural survey
methods. Hoboken, NJ: Wiley-Interscience.
Holmes, H., Tangtongtawy, S., & Tomizawa, R. (2003). Working with the Thais: A guide to
managing in Thailand (2nd ed.). Bangkok: White Lotus Press.
Horwitz, E. (1991). Preliminary evidence for the reliability and validity of a foreign
language anxiety scale. In E. K. Horwitz & D. J. Young (Eds.) Language anxiety:
From theory and research to classroom implications. Englewood Cliffs, NJ: Prentice
Hall.
Horwitz, M. B., Horwitz, E. K., & Cope, J. A. (1986). Foreign language classroom anxiety.
The Modern Language Journal, 70(2), 125-132.
Johnson, R. L., & Morgan, G. B. (2016). Survey scales: A guide to development, analysis, and
reporting. New York: The Guilford Press.
Kao, P., & Craigie, P. (2013). Coping strategies of Taiwanese university students as
predictors of English language learning anxiety. Social Behavior and Personality
41(3), 411-420.
Kondo, D. S., & Yang, Y-L. (2004). Strategies for coping with language anxiety: The case
of students of English in Japan. ELT Journal 58(3), 258-265.
Krejcie, R. V., & Morgan, D. (1970). Determination of sample size for research activities.
Educational and Psychological Measurement 30, 607-610.
Mahmoodzadeh, M. (2012). Investigating foreign language speaking anxiety within the
EFL Learners inter-language system: The case of Iranian learners. Journal of
Language Teaching and Research 3(3), 466-476.
Malgwi, C. A. (2004). Determinants of accounting anxiety in business students. Journal of
College Teaching and Learning 1(2), 81-94.
Marwan, A. (2007). Investigating students foreign language anxiety. Malaysian Journal of
ELT Research, 3, 37-55.
Matsuda, S., & Gobel, P. (2004). Anxiety and predictors of performance in the foreign
language classroom. System 32, 21-36.
McIlroy, D., Bunting, B., Tierney, K., & Gordon, M. (2001). The relation of gender and
background experience to self-reported computing anxieties and cognitions.
Computers in Human Behavior 17, 21-33.
Onwuegbuzie, A., Bailey, P., & Daley, C. E. (1999). Factors associated with foreign
language anxiety. Applied Socio Linguistics 20(2), 218-239.
Ozturk, G. & Gurbuz, N. (2014). Speaking anxiety among Turkish EFL learners: The case
at a state university. Journal of Language and Linguistic Studies 10(1), 1-17.
Pappamihiel, N. E. (2002). English as a second language student and English language
anxiety issues in the mainstream classroom. Proquest Education Journal 36(3), 327-
355.

37
Rosen, L. D., & Weil, M. M. (1995). Computer anxiety: A cross-cultural comparison of

university students in ten countries. Computers in Human Behavior 11(1), 45-64.
Sekaran, U. (2000). Research methods for business: A skill building approach (4th ed.). NY:
John Wiley & Sons, Inc.
Semmar, Y. (2010). First year university students and language anxiety: Insights into the
English version of the foreign language classroom anxiety scale. The International
Journal of Learning, 17(1), 81-93.
Suntaree, K. (1990). Psychology of the Thai people: Values and behavioral patterns. Bangkok:
Research institute of Development Administration.
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbachs alpha. International
Journal of Medical Education 2, 53-55.
Todman, J. (2000). Gender differences in computer anxiety among university entrants
since 1992. Computers & Education 34, 27-35.
Towell, E. R., & Lauer, J. (2001). Personality differences and computer related stress in
business students. Mid-American Journal of Business 16(1), 69-75.
Uyar, A., & Gungormus, A. H. (2011). Factors associated with student performance in
financial accounting course. European Journal of Economic and Political Studies 2,
139-154.
Wang, S. (2010). An experimental study of Chinese English major students listening
anxiety of classroom learning activity at the university level. Journal of Language
Teaching and Research, 1(5), 562-568.
Yashima, T. (2002). Willingness to communicate in a second language. The Japanese EFL
context. Modern Language Journal 86(1), 54-66.

38

Vol. 16, No. 1, pp. 38-56, January 2017
(Mis)Reading the Classroom: A Two-Act Play on

the Conflicting Roles in Student Teaching
Christi Edge, Ph.D.
Northern Michigan University
Marquette, Michigan, United States of America
Abstract. This case study examined concentric and reciprocal notions of

readingthat of high school students, a pre-service teacher, and a
teacher educator. An intern charged with teaching students to read,
interact with, and compose texts in an English/language arts classroom
constructed her role in the classroom based on her reading the text of
her internship experiences, relationships, and responsibilities. Using
interviews and observations, a teacher educator read and interpreted the
classroom text the pre-service teacher composed during her
internship and then constructed a two-act play which details the
conflict in the interns enacting the dual role of student-teacher and her
subsequent reading of the classroom text from her stance as student-
teacher. Concepts of classroom literacy for teachers and teacher
educators are considered.
Keywords: teacher education; reading classroom text; classroom

literacy; student teaching internship; stance
Introduction
In light of growing pedagogical, professional, and public awareness that twenty-
first century literacy involves more than just printed words on a page and that
specific literacies are acquired throughout the duration of an individuals
education (Barton, 2000; Biancarosa & Snow, 2006; Buehl, 2014; Clark & Flores,
2007; Draper, 2011; Gee, 2012; International Reading Association, 2012; Langer,
1987; Lankshear & Knobel, 2007; Maclellan, 2008; National Council Teachers of
English, 2007, 2008; National Center for Education Statistics [NCES], 2006, 2007;
Rogers, 2000), it is time to consider the professional literacy needs of the very
individuals to whom we look to educate our children and our adolescents
(International Literacy Association, 2015).
Review of the Literature

Lad Tobin (2004) implies a connection between the disciplinary focus of
studying texts and the pedagogical importance of studying classrooms as text by
asserting that teaching is a way of reading and writing. Students learn to teach
through, first, learning to read the classroom and, second, learning to write
themselves within that classroom (p. 129). A teacher is simultaneously a reader
and a writer of her classroom. Like readers whose meaning making is framed by
2017 The author and IJLTER.ORG. All rights reserved.

39
the stances they take along an aesthetic-efferent continuum (Rosenblatt, 1978,

2005), the stance or horizon of expectations (Popper, 1962, p. 47) with which a
teacher approaches a classroom conceivably guides his or her meaning-making
process of that classroom text (Edge, 2008; Edge, 2011). How a teacher constructs
and enacts her perception of her role as teacher is an important focus for
investigation; this perception guides the teachers stance toward the classroom
and potentially shapes how she makes sense of what happens in the classroom.
A teachers definition of her role in a classroom shapes her stance as a reader
and a writer of the classroom text; it influences how she plans her instruction,
creates learning opportunities, gauges her students learning, and reflects on or
assess her own teaching.
While some scholars have alluded to the similarities between reading
and writing pieces of text to the reading and composing of a classroom text (e.g.
Burke, 2008; Edge, 2008; Tobin, 1991; Tobin, 2004; Perl, 1994), none has yet to
explore how stance shapes that reading, what kinds of cues teachers use to read
the classroom text, or what implications the reading of the classroom text might
hold for prospective teachers or for teacher educators. Given the relative
importance of literacy instruction, the gap in our knowledge base between the
development of teachers professional classroom literacy and the literacy
instruction they provide for their students, one must wonder, as Tobin (2004)
does, what we are to make of colleagues who read everything as a textexcept
their own classrooms and pedagogical methods? It seems a comically studding
act of denial, like Freud asserting that everything is a phallic symbolexcept his
own cigar (p. 129, original emphasis).
In teacher education curriculum and reform, clinical field experiences
and the practice of teaching are topics of perennial interest in the United States
(e.g., Edge, 2015; Zeichner, 2012; Zeichner & Beir, 2012). For students in a
College of Education, completing their full-time student teaching internship is a
culmination of their undergraduate education (Cherian, 2003; Cruickshank &
Armaline, 1986; McIntyre, Byrd, & Foxx, 1996) and an influential time (Veal &
Rikard, 1998; Wilson, 2006) of transition between being a student and becoming
a teacher (Burke, 2008; Graham, 2006). It was the goal of this study to investigate
how stance, as revealed in three interns expressed perception of their roles in
the classroom, guided their teaching experiences. Understandings gained from
studying the relationship between the role a teacher enacts and her stance
toward reading and interpreting the classroom, could potentially contribute
insight into prospective teachers classroom literacy (Edge, 2011)development as
these teachers move along the teaching-learning continuum, or what Jim Burke
(2008) refers to as the continuum of performance (p. 22), in transition from
being expert students to becoming novice teachers. Specifically, understanding
how interns construct their roles during internship offers insight into: a) how
interns construct their interpretation of their role in the classroom ; b) how their
perception of their role is shaped by their internship experiences and
relationships ; c) how the perceived roles of the other members of the internship
triadthe university supervisor and the cooperating teacherinfluence the
stance an intern takes toward her internship; and d) how a teacher educator
might learn from reading the text of her university students classrooms during
their internship.

40
Theoretical Framework
Understanding how pre-service teachers read and interpret the classroom text is
a complex issue. In a transactional paradigm (Dewey & Bentley 1949; Rosenblatt
1978, 2005), reading and teaching share a common theory of meaning making.
Like Louise Rosenblatts transactional theory of reading, teaching too is an
experience, an event in the moment of a classroom a dynamic interaction
between a particular persons repertoire of experience and the text of the
classroom itself. To be classroom literate, a teacher must read and make meaning
of specific classroom situations, including reading classroom discourse, making
meaning of teachable moments, connecting theory and practice, constructing
scaffolds to aid students understandings, questioning and evaluating student
progress, reading between the lines of students verbal and nonverbal
language, and thinking critically and metacognitively about the process of
teaching (Edge, 2011). Couched in the concept of the classroom as a type of
living, dynamic text (Edge, 2008; Edge, 2011; Tobin, 2004; Witte, 1992) with
which the teacher must transact in order to effectively teach, this study is guided
by a framework which acknowledges the transactional and communicative
nature of teaching and learning (Allen, 1995; Cooper & Simonds, 2007; Dewey &
Bently, 1949; Greene, 1983; Hurt, Scott, & McCroskey, 1987; Langer, 1987;
Rosenblatt, 1978, 2005; Smagorinsky & Whiting, 1995) and relies on
interpretivisim to focus on meanings and the interactive communication of
meanings through story (Bochner, 2005).
Method
Interpretation and the gathering of interpretations are central to case study
research (Erickson, 1986; Stake, 1995). Since this study was guided by a
framework which acknowledges the transactional and communicative nature of
teaching and learning (Allen, 1995; Cooper & Simonds, 2007; Dewey & Bently,
1949; Greene, 1983; Hurt, Scott, & McCroskey, 1987; Langer, 1987; Rosenblatt,
1978, 2005; Smagorinsky & Whiting, 1995), case study methodology is
appropriate for interpreting how a teachers stance shapes meaning-making
within the context of classroom teaching and learning.
Sample. This qualitative case study was originally part of a multiple-case

study of three pre-service teachers who served as interns to high schools in a
large metropolitan school district in the southeast United States. Each participant
was a female, senior, English education major in a College of Education housed
in a large research university in the same district that the internships took place.
Participants. The participants were selected using theoretical sampling

according to participants potential ability to contribute to a developing theory
(Creswell, 2018; Onwuegbuzie & Collins, 2007; Onwuegbuzie & Leech, 2007) for
purposes of refining ideas and determining relevant contexts and conditions of a
developing theory (Charmaz, 2000) of classroom literacy (Edge, 2011). Prior to
this study, each pre-service teacher demonstrated diverse abilities, personality,
and internship placement that marked her as a potential participant. As former
students in two university courses I taught, Methods of Teaching English and

41
Methods of Teaching Reading in the English Classroom, I had a context for

knowing if these individuals could contribute to an emerging understanding of
stance in relation to reading the classroom as a text, in general, and stance as a
lens through which to understand pre-service teachers roles during internship,
specifically. Therefore, I invited these three former students to participate, and
each agreed. This manuscript focuses on one of the three participants in order to
offer thick description (Stake, 1995, p. 43) of a central theme that emerged
from all three case studies.
Data Collection. Stake (1995) writes that the interview is the path toward
discovering multiple realities. To investigate the interns reading of the
classroom and to identify the stance(s) that she use to read it, interviews were
the primary source of data collection. Data also included classroom observations
and an interview with the interns cooperating teacher to triangulate data
obtained through interviews.
The intern participated in an interview at the beginning of her internship,
approximately three weeks into her semester-long assignment and in a second
interview during the final two weeks of her internship. The second interview
took into consideration the themes emerging from data collected during the first
round and then formulated questions to further explore these themes and to
gather additional details in light of extended experiences in the field. The first
interview took place on the internship site and was accompanied by
observations of each preparation the intern taught: freshman English and junior
honors English. To better accommodate the interns schedule, the second-round
interview was conducted off campus after school hours and included an email
follow-up component. Interviews and observations were digitally recorded.
Observations were also videotaped. Copious field notes were taken during the
observations and interpretive anecdotal notes were written immediately
following observations and interviews.
Data Analysis. The primary data analysis used to investigate expected

and unexpected patterns was interpretation of observation and interpretation of
interviews (Stake, 1995). Interviews, field notes, and anecdotal notes and
interpretations were read and analyzed within the context of each case.
Secondary data analysis included coding data into emerging themes and then
rereading and analyzing data to refine themes and chart examples of themes (see
tables 1 and 2 in Appendix A for examples). Analysis through a search for
patterns, called correspondence by Stake (1995, p. 78) allowed for construction
of meaningful understanding. Although finding similarities across cases was not
the goal of this study, a cross-case chart of themes was made in order to
aggregate themes across cases and to consider contexts and conditions under
which a developing theory (Charmaz, 2000) of stance in teaching and reading a
classroom text might exist.
Credibility. For consensual validation (Eisner, 1991 as cited in Creswell,

2018), two other English education teacher educators, who had no prior
knowledge of this study or participants, reviewed both ongoing interpretations
of classroom observations and the findings. Coded data and interpretations from

42
interviews were also member checked with participants (Creswell, 2018).

Findings from multiple data sources were composed with attention to structural
corroboration, (Creswell, 2018; Eisner, 2005), thick description (Stake, 1995), and
researcher reflexivity.
Methodological Concerns. A few limitations of this study need be

acknowledged. First, as previously mentioned, the participant was my former
student in two university methods courses. While it appears that familiarity
allowed her freedom to speak without feeling as if I were assessing her, it is
possible that traditional student/teacher power dynamics could still have
limited both the scope of what she revealed as well as her ability to be candid
about ways in which her university experience did or did not prepare them to be
successful in her classroom. Second, although I did observe the intern teach in
conjunction with her first interview, it is plausible that this limited exposure did
not provide context enough to fully understand the subtleties of the feelings,
ideas, and experiences she shared with me in interviews. Hillocks (1999) has
suggested that classroom observations are necessary to understand the more
subtle nuances of teachers thinking. While I did get to know the intern over the
span of an academic year prior to her internship, that knowledge was limited to
her being a student and a preservice teacher in simulated teaching experiences to
their peers. Prolonged observations might offer additional insight and serve to
further crystallize (Richardson, 2000) my understanding of interview data.
Finally, interpretation of data is very much shaped by my own conceptual
framework. Although findings were analyzed in light of literature on field
experiences and triad relationships, I openly acknowledge that in any given
reading, there exists a text, a reader, a context, and an intertext that greatly
influence the meaning made. To account for this, I have tried to make my own
reading transparent when possible.
Findings
To offer thick description and experiential understanding (Stake, 1995, p.
43), the findings of this study are organized around a central theme that
emerged in the case of Teri Thompson (the participants name and school site
have been changed to protect and respect her anonymity) who approached her
internship from her stance or mindset as a student. I later refer to this as a
student-teacher stance to underscore the divided nature of reading the
classroom from the perspective of a student who is attempting to be the teacher.
The blurred roles of student and teacher appeared to make the intern feel like
she was doing neither as well as she wished. Furthermore, this student-teacher
stance shaped the manner in which she evaluated her success in the internship
placement, her ability to take on the role of teacher, and her thoughts about
her future as a high school English teacher.
Although case studies are not stories in the traditional sense (Stake,
1995), detailed portions of this case are heuristically presented as a two-act play
to allow others to develop vicarious experiences (Stake, 1995, p. 63) through
the detailed snapshots into the text of this case. The telling of Teris story is
organized around classroom moments when her student-teacher stance revealed

43
tensions in her role as teacher, and it is set within Teris physically, emotionally,
and educationally bifurcated classroom, where I observed her teaching.
Findings are reported and framed by my own reading of the participants
classroom. This reading took place during my observations of her teaching and
interviews, and is read in light of both the specific context I observed and, at
times, in light of my long-term reading of the participant as a university student.
Teri: A Two-Act Play on the Tensions of Teaching
Act One: The Conflict of Playing Two Roles

As I slip into a student desk in Teris classroom and into a lesson that is now
well underway, I quickly gather that students are reading a play. The majority of
the students sit in desks facing forward, listening and watching or following
along in their books, but a few are seated at the apex of the room as if on stage,
reading parts from their textbooks. Teri stands front and center, reading the
narrators part with enthusiasm and inflection, pausing from time to time to
comment or to ask a question from the students; she seems quite pleased and
comfortable to be the focal point on this classroom stage. Over the last year, I
have come to know Teri to be confident, passionate, and outspoken, a noticeable
presence in any room, so what I see doesnt surprise me. What did emerge as a
bit of a surprise is that this one lesson captured the crux of the actual plot and
conflicts in the story of Teris internship. The text of Teris classroom is written
around a series of related tensions: the tension between simultaneously enacting
student and teacherly roles in the context of her internship and the tension
created in (mis)reading students from her own vantage point and history of
experiences as a student.
In trying to simultaneously enact a dual rolethat of participant
(student) and director (teacher) in the living dramatic text of her internship
Teri blurred the bounds of classroom student and teacher in a way that created
not only tension, frustration, and uncertainty in her perception of herself as
teacher, but also left her feeling as if her role was written, and her part must be
carried out until the end of the show. As she later told me,
In my [honors] classes, and I know this isnt right, but unfortunately, this
is the foot I started out on, and now its going to be difficult to correct it, I
am cool Mrs. Thompson who tries to do fun activities. They kind of
take me seriously sometimes, and they will work with me. My age, I
think is a bit of an aid, and um, at the same time, it prohibits the full
range of my teaching abilities. They look at me like Im their peer. Who
is this girl telling me what to do? Well, this girl has an almost bachelors
degree, a house, a marriage, another jobthis girl has a lot more life
experience than you do, so just shut up and listen to this girl!
Even though Teri laughs off the seriousness of her perceived inability to be seen
as the teacher, this tension is quite apparent in her relationship with her
students. In one of her eleventh grade honors classes, a student asks,

44
Wheres Dr. B [the cooperating teacher]?
Im Dr. B. now, replies Teri.
No, youre Tommy, concludes a second student.
The students tone is playful but serious, as if to remind Mrs. Thompson of her
place. Teri responds by playfully poking her tongue out a little. Her light-
hearted attempt to save face and brush off the students remarks only position
her further into the role of a student-teacher simultaneously both student and
teacher and yet fully neither.
In her ninth-grade class, similar patterns of interaction emerge. After a
student answers a question correctly during the introduction to her lesson, Teri
excitedly slaps the student a high-five.
Ouch! he yelps. Your ring hurt me!
Well, dont give me a high-five on my ring hand! she retorts through a

grimaced face and then playfully sticks her tongue out at him in an
exaggeratedly childish manner.
As I watch, I surmise that she seems embarrassed and struggles to regain
the professional role of the teacher and still be liked by her students. The
students dont seem to know how to respond either.
Later into her lessona lesson designed to help the ninth-grade students
prepare for the standardized writing exam that they would be taking the
following dayTeri writes the words And, But, and Because in big letters
on the board to emphasize the ways students should not begin their sentences
when they write. One student frowns at his graded practice essay, glances back
up at the words written on the white board, and then responds,
Im retarded. Damn, Im so retarded!
His comments indicate that he hears her as an authority figure who evaluates
and finds writing such as his to be poor. Nevertheless, his choice of language is a
subtle statement, heard by the entire class, that her authority does not extend to
what is acceptable language in the composition of the classroom. Another
student openly challenges Teri in a way that implies she is not his authority
figure,
You began a sentence with Because, he snidely accuses.
That wasnt a sentence, it was a fragment, she quickly rationalizes in

her rebuttal. Despite her quick wit, a brief but awkward silence hangs like a
thick curtain of tension in the classroom. Teri appears unable to rewrite or
redirect the flow of the lesson. The act of classroom learning ends without
dnouement. Nevertheless, these scenes illumine Teris tension in not being fully
teacher. Although she is a full-time university student, she is not among peer

45
students here either. Teri seems to have no clearly defined space in which to
write her role into the text of this classroom.
During a second interview conducted through email in the final week of

her internship, Teri reveals with what she describes as brutal honesty that this
tension remains unresolved:
I have found establishing a relationship with my students to be

challenging. Part of me enjoys the same music and movies that my kids
do, and find common grounds in that aspect. Part of me looks down on
my kids for having such superficial worldviews, and I have made that
clear to them (oops) in the past. Im torn between two worlds myself
Im 23, and find some juvenile things interesting and humorous (for
instance, I quote Family Guy with my kids sometimes) but Im also
married and a homeowner, and these experiences have broadened my
perception and made me wonder why my kids, who arent that much
younger than I am, still only see value in superficial nonsense. I think I
have a lot of self-discovery to do, and I dont know if Im going to be at a
healthy mentality by fall. Regardless, I have to teach next year I cant
wait tables forever. But Im wondering if I need to consider middle
school or (YELP) elementary school, where such a dramatic age
difference exists between me and my kids that some of my neuroses
arent exposed. Im wondering if all new teachers feel that way, or if I
have some baggage I need to unload before I can be a successful mentor.
The tension in trying to take on the role of a teacher while still trying to relate to
students from her stance as a student leads her to question her emotional
maturity and thus, her ability and future plans to teach high school students.
In her interviews, Teris perception of others roles and her definition of a
successful internship underscore how her liminal role as student-teacher in the
high school classroom is related to her overall place in the internship triad. For
instance, as much of Teris language connotes, and nearly all of her comments
combine to reveal, her primary stance toward her internship experience is that of
an advanced student or a teaching apprenticesomeone who is charged with
mastering discrete components of teaching. The following excerpts from her
interviews underscore this primary stance:
Researcher: How would you define a successful and a failed

internship?
Teri: Successful? Fulfilling the requirements set by [the

university], establishing a good rapport with students and
colleagues, learning about the system from within
Failed? Not meeting any of the aforementioned
benchmarks.
Researcher: How would you define your role within your

internship?

46
Teri: I feel like a young Jedi knightThese other teachers

[gesturing to the teachers she usually shares lunch
with] took me under their wingIm not just Dr. Bs
intern, Im some woman whos going to be teaching,
maybe teaching their kids if they have little ones.
They want to make sure I do it right and represent
the field well Im a little Anakin Skywalkerbut I
wont go to the dark side...
The first criterion Teri used to define success is meeting the universitys
objectives. To her, she is foremost a student, working to complete her
coursework and earn a good grade. Her second criterion, establishing a good
rapport, also resonates with the kind of studentwell-liked by students and
teachersshe was prior to her internship. Furthermore, her description of
herself as a young Jedi knight is a clear statement of her position or stance as a
student apprentice. In the movie Star Wars, a Jedi Knight is an apprentice who is
taught the ways of the force by masters. Jedi Knights are also socially well-
received, even popular in the Star Wars galaxy.
Teri also described her specific relationships with her cooperating teacher
(CT) and university supervisor (US) in light of her being a student whose goal it
is to learn from masters and to demonstrate her learning to her supervisors. For
instance, when I asked her to describe her relationship with her CT, Teri referred
to her as Dr. Grandma. Someone who has taken a very maternal role with
her, yet someone who expects certain things and shows her what to do with a
voice of experience. Someone who sees where [she] is weak, and pulls [her] up
a little. Teri also said that Dr. B. was someone who might possibly evolve into
the role of a mentor by the time internship concluded. However, when I asked if
her US could be a mentor, she corrected, No, no, hes an evaluator who grades
her, and hes a supervisor and a source of information. Teris relationships
with the other two members of her triad communicate that she sees herself as a
student who is evaluated and an apprentice who is to be taught and nurtured by
experienced teachers in the profession.
Her professed focus and goals for the semester also reveal her student
stance in that she focuses on what she has not yet mastered. Teri eschews her
own assessment of herself as an excellent planner and chooses to center her
comments on talk of classroom management and the need to be able to
recognize how kids learn. Her focus is not necessarily on what students are or
are not learning, although she does communicate that she is concerned with
their learning; rather, her focus is on herself as someone who is not yet able to
master a skill of teaching. This student-apprentice focus fuels a second, related
conflict in her classroom: her struggle to comprehend students who are
unlike her.
Act Two: Conflict Created from Misreading the Text of the Classroom
Acts One and Two in Teris teaching schedule are separated by an
intermission of lunch. When the bell rings, Teri takes me back stage to the
teachers cafeteria where I conduct her first interview. Initially, Teri had only

47
invited me to observe her eleventh-grade honors English class, but before lunch
ends, she extends her invitation to ask me to watch her ninth-grade regular
English classthe class from Hell, as she put it, that Teri initially did not want
me to observe. Unlike the eleventh-grade honors classes where she feels she
must be Cool Mrs. Thompson, in her ninth grade classes Teri describes her
role as someone who has to be the locus of control and the gatekeeper of
knowledge for students.
Im a drill sergeant, a jail warden, prison guard, wicked witch, and, uh,
behavior corrector, manners enforcer, No, we dont pull on girls hair,
Sit back down in your seat, Dont use aint, Dont use the F word in
my classroom, and so on and so forth.
Within minutes, maybe even seconds of the beginning of the next class
period, I see the shift in roles that Teri articulated during her interview. She
seems to step into character according to how she perceives her role as the
classroom teacher. The juxtaposition of the two types of classes she teaches
further depicts tensions created by Teris teaching from her stance as a student-
teacher.
Teris fifth period freshman English class has 17 students present, 11
males, six females; five students are students with special needs. Just before
class, Teri warned me that she would be laying down the law with this class
today, and true to her word, class begins with a speech that seems designed to
remind student who is boss, who is not, and that referrals await any who dont
want to play the right role. As I watch and listen, I come to contextualize the
interview responses she shared with me, and I begin to see her room as a
metaphorical stage and a means of understanding her bifurcated teaching roles,
as she explained them to me.
Separated by a moveable wall best described as a heavy curtain, Teris
room is really two rooms, one a tiny classroom and the other a book closet. In
the center of the room, the curtain is pulled back, but it doesnt recede all the
way to the walls, causing it to jut out a little and create an odd configuration and
a one-room-two-room sensation. Some students sit on the bookroom end of the
partition, but the majority are in the classroom section where the whiteboard is,
and where Teri writes as she teaches.
The room itself seems to echo the tensions Teri communicates in her
interview and in her stance as a student-teacher. On the one side of her
internship experience, Teri teaches students who are like herselfbright, self-
motivated, mostly middle-class, traditional honors students. On the other side,
she teaches students who are very different from herself, and she has trouble
reading them from her stance as a traditional, successful, well-liked student.
Their knowledge is sophisticated but worldly rather than academic, and
they struggle to follow Teris vision for the lesson as they fidget through a
traditional listen-and-take-notes kind of class. It appears to me that because Teri
is interpreting the happenings of the classroom from her own student stance, she
does not know how to read her students and connect with them. In fact, she
seems to repeatedly misread their comments as deliberate attempts to thwart her
lesson and rebel against her as the teacher. For instance, Teri attempts to think-

48
aloud some ideas for answering an essay prompt, After 2:52 [the end of the
school day], youre not learning
Yes we are, a student interrupts. Im learning football plays.
Im learnin tricks on my bike, shouts out another student.
Suddenly the room is bustling with students talking over one another about
everything they learn after school is over. Teri is frustrated at the direction the
conversation is taking and appears to interpret their comments as attempts to
annoy her. Hands, hands! Simmer down, guys, simmer down. The majority of
the students appear really willing to participate by offering their opinions and
personal experiences at a point in the lesson that they can connect to. This is
frustrating to Teri who doesnt quite know how to read their responses or
channel their energy, and so she collectively chides them for being silly like
middle schoolers. She doesnt hear them making connections between their
own personal, less academic, perceptions of learning; she heard talk that seemed
irrelevant to her conception of learning and interpreted their actions as
disruptive behavior that interrupted the lesson she had prepared.
The lesson on this day is geared to prepare students for the standardized
writing test that the students will take the following day. Teris plan is to first
use students practice essays as a vehicle for talking about what to do and what
not to do when they write, and then to think through an essay prompt with
students. These students have a hard time following her oral construction of an
essay intermitted with asides on the proper use of conventions. As a result, Teri
ends up doing the majority of the mental work by telling students what to think
and do from her perspective as a successful writer.
When I got a [highest score possible] on my [state writing assessment] ,
I
The learning objectives appear distant from students background

knowledge and interests, but several students try to please her by responding
when they can. When they do respond, Teri tends to misinterpret what they say
and then responds with the lion-tamer mentality that she feels defines her role in
these classes. For instance, Teri begins the lesson by holding up a students
practice essay.
This person is using vivid details from her life experiences. This is a
very good example of what students should do.
And then a moment later,
Do not use you unless you are addressing a specific audience. I want
to see everyone writing this down in their notes
How do you spell audience? calls out one student.
What? Teri seems shocked out of her train of thought.

49
Repeat that, Miss, requests a second student who has been fumbling
with his paper.
Whats vivid? a boy in the front of the room asks.
Dont interrupt me and call yourself stupid, Teri chides and then sighs,
exasperated with what she interprets as misbehavior.
A few minutes later, a girl who has been silent the entire lesson offers an
idea for the essay they are constructing out loud as a class, Teachers hate us.
Thats what we should write about. When Teri responds with disbelief, the
class quickly becomes loud as students shout out (vivid) examples of how
teachers demonstrate their negative feelings toward them.
Talk to them, Teri ironically replies.
They dont listen!
An exasperated Teri stops to hush the class and proceed with her lesson. She
misses the fact that students are speaking from their personal experiences (the
writing trait she praised in the beginning of the lesson), and that by listening to
them she could use their experiences as a bridge to help them conceptualize
ways to write a persuasive essay. Teri talks in terms of things to do and not to do
from her personal knowledge of her own successful, academic writing
experiences while the students respond from their knowledge base of
experiences outside of school and to their negative experiences with teachers in
school. As a result, students and teacher talk at and over one another, never
really connecting. Much like the layout of the classroom itself, a wall seems to
separate their ability to accurately read and understand one another.
Discussion: An Epilogue for Educators

Positioning theory has been used to explain how triad members take on roles or
parts to play within the unfolding classroom story. For instance, Bullough and
Draper (2004) write, that [w]ithin storylines, actors take different parts, and the
parts they play and how they play them reveal the meaning of events as well as
present definitions of self and other (p. 408). They also assert that
contextpersonal characteristics and individual biographiesprofoundly
affect the kind and quality or relationships that form between beginning
teachers and those who would assist them in learning to teach (Bullough &
Draper, 2004, p.409). In light of Teri Thompsons conflicts, I might add that such
factors potentially profoundly influence the quality of teaching and the
relationships that form between beginning teachers and the students they assist.
Considering that student teachers come to the classroom after spending a
lifetime in an apprenticeship of observation (Lortie, 1975) from the vantage point
of the student desk, transitioning from playing the part of a student studying to
teach, to actually being the classroom teacher to other students requires a
challenging and complex shift in thinking. This transition is complicated by the

50
regular role shifting (Bullough and Draper 2004; Veal & Rikard, 1998;
Smagorinsky, Cook, Moore, Jackson, & Fry, 2004) student teachers must do in
order take on competing roles as student and as teacher during the semester of
their internship. The relationships of the roles played by each member of
internship triad is one which often results in power shifts, depending on who is
present (Bullough & Draper, 2004; Veal & Rikard, 1998). When a student teacher
is alone in the classroom, she is expected to be the teacher; when the university
supervisor comes to observe and brings an assessor stance, she must shift to
student; and when the cooperating teacher is present, the intern could be
expected to assume a stance anywhere along the student-teacher continuum.
When we consider what it is that we ask interns to dowalk out of the world
they know and step into one theyve only studied and then straddle both worlds
while the line distinguishing the two continues to move, it is a wonder pre-
service teachers dont feel more academically and professionally schizophrenic
during their internship. Its not a surprise then, that each participant studied
during this investigation expressed degrees of failure in terms role ambiguity
and developing a sense of self-as-teacher (Sudzina & Knowles, 1993, p. 255)
through questioning her emotional maturity and whether she would actually
pursue a teaching position after internship. The fact of the matter is that
prospective teachers are students, and stepping into a classroom does not a
teacher make (as Yoda might phrase it). Simply trading the college campus for a
secondary school campus does not instantly transform a student into a teacher
any more than stepping into an operating room makes a medical student a
surgeon or stepping into a courtroom makes a student of the law an attorney.
Can we, as teacher educators and mentor teachers expect interns to bring to their
internship a stance other than student or student-teacher? After all, arent
teachers like Teri supposed to be learning from their internship experiences? Well,
yes. And yet, potential exists for the scenes from Teris classroom to be re-
envisioned, revised, and re-composed.
Re-Envisioning Alternate Tales from the Cut Scenes of Teris Two-Act

Play. The Holmes Group (1986) reported in Tomorrows Teachers,
[p]aradoxically, teachers are the butt of most criticism, yet singled out as the
one best hope for reform. Before I proceed further, I think it should be said that
despite Teris misreading her classroom text and her struggle to construct a
teacherly role within the already composed lines of the parts she perceived she
needed to play during internship, Teri was a pre-service teacher who
communicated both a desire to be and the knowledge to do the kind of teaching
which resonates with best-practice research and literature. That she repeatedly
acknowledged the dissonance between what she knew she should do and what
she realized, with dismay, she was actually doing demonstrates a level of
awareness that holds hope for her ability to see and perhaps to re-compose her.
This alternate version of Teris tale can be pieced together from fragments of
data that didnt make it into the two-act play production of her classroom
teaching.
In Teris first interview, she shared that her understanding of and
reasons for teaching had changed significantly in the last year. When asked to
explain, she suddenly spoke in a high pitched and excessively jovial voice that

51
sounded like a cartoon characters, You know, I, Ilike everybodyI wanted

to guide others to learn. I interpreted this tin voice as her way of expressing the
naivet of her initial perception of teaching. Speaking once again in her adult
voice, Teri listed two factors which helped her to gain a more three-dimensional
conceptualization of teaching: a methods course in teaching reading and her
present, real-life experiences.
What Teris two turning points have in common is that each required her
to work closely with students. In the methods of teaching reading course, Teri
wrote a case study after working with a struggling reader. From this experience
Teri realized, I never knew there were people who didnt have the capacity to
do certain things and had to be shown and given skills in order to help them
learn. Before, I just thought I would give them information. Prior to her
experiences in the methods course, Teri looked at readers through her own
stance as a successful reader who never even gave much thought to the complex
moves required to make sense of written texts. However, in working with a
struggling reader, she took a stance that differed slightly from that of student
stance to what Id like to call a learnership stance. In both stances, Teri was
learning; however, when she had guided opportunities to reflect on and make
meaning of her working closely with a student who was very different from her,
she was able to take on a transactional learnership stance that enabled her to
read her interactions with this student in a way that taught her from his
experiences. Thus, the experience was transactionaleach perspective informing
the other. Terri read and made meaning from her experiences working with
students
Reading Classrooms as Texts. Students potentially teach the teacher

when a learnership stance guides the teachers reading of the classroom. In the
same way that the transactional nature of a written text can lead a reader toward
exploration (Rosenblatt, 1938), meaningful understanding, and new knowledge,
so too can the transactional nature of communication (Cooper & Simonds, 2007)
lead teachers to make meaning of and to generate new knowledge about their
profession. Said another way, because of the transactionally communicative
nature reading and writing share with teaching (Cooper & Simonds, 2007;
Dewey & Bently, 1949; Greene, 1983; Hurt, Scott, & McCroskey, 1987; Langer,
1987; Rosenblatt, 1978, 2005; Smagorinsky & Whiting, 1995), what is known
about readers and writers processes can also offer frameworks for
understanding how teachers develop their ability to read and to communicate
with their classrooms. In a transactional paradigm (Dewey & Bentley 1949;
Rosenblatt 1978, 2005), reading and teaching share a common theory of meaning
making. Like Rosenblatts transactional theory of reading, teaching too is an
experience, an event in the moment of a classroom a dynamic interaction
between a particular persons repertoire of experience and the text of the
classroom itself (Edge, 2011). To become classroom literate, a teacher must read
and make meaning of specific classroom situations, including reading classroom
discourse, making meaning of teachable moments, connecting theory and
practice, constructing scaffolds to aid students understandings, questioning and
evaluating student progress, reading between the lines of students verbal and
nonverbal language, and thinking critically and metacognitively about the

52
process of teaching. Couched in the concept of the classroom as a type of living,

dynamic text (Edge, 2008; Edge, 2011; Tobin, 2004; Witte, 1992) with which the
teacher must transact in order to effectively teach, pre-service teachers can learn
to construct their perception of teaching using their prior knowledge of their
pedagogical training and learning to attune to the cues of a classroom as text.
Through a learnership stance, Teri could conceivably learn to read the classroom
as text and to compose her role in it by transacting with her students even as
they transact with the texts they learn to read and write.
Learning to read, and learning from ones misreading of the text of the
classroom is a first step toward developing teacher literacy. For interns like Teri,
shifting from a student-teacher stance where meanings are fixed based on
previous readings of the classroom from her perspective as a student, to a
learnership stance where the transactional nature of teaching and learning is
valued and cultivated through a classroom community, places emphasis on the
process of making meaning in the classroom context. It invites students to
participate in the construction of their learning. It shifts power dynamics. It
empowers teachers to create and revise teaching roles that are not scripts set in
type.
References
Allen, J. (1995). Its never too late: Leading adolescents to lifelong literacy. Portsmouth, NH
Heinemann.
Barton, P. E. (2000). What jobs require: Literacy, education, and training, 1940-2006.
Washington, DC: Educational Testing Service. Retrieved December 10, 2007
from http://www.ets.org/Media/Research/pdf/PICJOBS.pdf.
Biancarosa, C., & Snow, C. E. (2006). Reading nextA vision for action and research in
middle and high school literacy: A report to Carnegie Corporation of New York
(2nd ed.). Washington, DC: Alliance for Excellent Education.
Burke, J. (2008). Thinking about teaching and learning. The English teachers companion: A
complete guide to classroom, curriculum, and the profession (3rd ed., pp. 3-27).
Portsmouth, NH: Heinemann
Bullough, R. V., Jr., & Draper, R. J. (2004). Making sense of a failed triad: Mentors,
university supervisors, and positioning theory. Journal of Teacher Education, 55(5),
407-420.
Charmaz, K. (2000). Grounded theory: Objectivist and Constructivist methods. In N. K.
Denzin and Y. S. Lincolm (Eds.), Handbook of qualitative research (2nd ed., pp. 509-
535). Thousand Oaks, CA: Sage.
Cherian, F. V. (2003). Student teachers and their journey: A reflexive ethnography. Ph.D.
dissertation, University of Toronto (Canada), Canada. Retrieved November 14,
2008, from Dissertations & Theses: Full Text database. (Publication No. AAT
NQ78608). 439 pages.
Clark, E. R. & Flores, B. B. (2007). Cultural literacy: Negotiating language, culture, and
thought. Voices in the Middle (15)2, 8-14.
Cruickshank, D. R., & Armaline, W. D. (1986). Field experiences in teacher education:
Considerations and recommendations. Journal of Teacher Education, 37(3), 34-40.
Cooper, P. J., & Simonds, C. J. (2007). Communication for the classroom teacher (8th ed.).
Boston: Pearson, Allyn and Bacon.
Creswell, J. W. (2018). Qualitative inquiry & research design: Choosing among five approaches
(4th ed.). Thousand Oaks: CA, Sage.

53
Dewey, J., & Bently, A. F. (1949). Knowing and the known. Boston: Beacon.
Edge, C. (2008). Classroom literacy: Toward a framework for reading and writing the
classroom text. Unpublished manuscript. University of South Florida, Tampa.
Edge, C. (2011). Making meaning with readers and texts: A narrative inquiry into
two beginning English teachers meaning-making from classroom events.
Graduate School Theses and Dissertations. http://scholarcommons.usf.edu/etd/3722/
Edge, C. (2015). On the nature of experience in the education of prospective teachers: A
philosophical problem. International Journal of Learning, Teaching and Educational
Research, 13(1), 29-41.
Eisner, E. W. (2005). Reimagining schools: The selected works of Elliot W. Eisner. New York:
Routledge.
Eisner, E. W. (1991). The enlightened eye: Qualitative inquiry and the enhancement of
educational practice. New York, NY: Macmillan.
Erickson, F. (1986). Qualitative methods in research on teaching. In M. Wittrock (Ed.),
Handbook of research on teaching (pp. 119-161). New York: Macmillan.
Gee, J. P. (2012). Social linguistics and literacies: Ideology in discourses (4th ed.). New York:
Routledge.
Graham, B. (2006). Conditions for successful field experiences: Perceptions of
cooperating teachers. Teaching and Teacher Education, 22, 1118-1129.
Greene, J. (1983). Teaching and learning: A linguistic perspective. Elementary School
Journal (33), 353-391.
Guyton, E., & Byrd, D. (Eds.), (2000). Standards: For field experiences in teacher
education.. Reston, VA: Association of Teacher Educators.
Hillocks, G. Jr. (1999). Ways of thinking, ways of teaching. New York: Teachers College
Press.
Holmes Group Inc. (1986). Tomorrows Teachers: A Report of the Holmes Group. Retrieved
February 10, 2008 from
http://www.holmespartnership.org/Tomorrows_Teachers.pdf.
Hurt, H.T., Scott, M. D., & McCroskey, J. C. (1987). Communication in the classroom.
Reading, MA: Addison-Wesley.
International Reading Association. (2012). Adolescent literacy (Position statement, Rev.
2012 ed.) (pp. 1-18). Newark, DE.
Langer, J. A. (1987). A sociological perspective on literacy. In J. A. Langer (Ed.), Language,
literacy, and culture: Issues of society and schooling (pp. 1-20). Norwood, NJ: Ablex
Publishing Corporation.
Lankshear, C., & Knobel, M. (2007). Sampling the new in new literacies. In M. Knobel,
& C.Lankshear (Eds.), A new literacies sampler (pp. 1-24). New York: Peter Lang.
Lortie, D. C. (1975). Schoolteacher: A sociological study. Chicago: University of Chicago
Press.
McIntyre, D. J., Byrd, D. M. & Foxx, S. M. (1996). Field and laboratory experiences. In J.
Silkula, T. J. Buttery, & E. Guyton (Eds.), Handbook of research on teacher
education (2nd ed., pp. 171-193). New York: Macmillan.
Maclellan, E. (2008). Pedagogical literacy: What it means and what it allows. Teaching and
Teacher Education, 24, 1986-1992.
National Center for Education Statistics. (2006). Nations report card: Reading 2005.
Washington, DC: U.S. Department of Education. Retrieved December 10, 2007
from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2006451.
National Center for Education Statistics. (2007). Nations report card: Reading 2007.
Washington, DC: U.S. Department of Education. Retrieved December 11, 2007
from http://nces.ed.gov/nationsreportcard/pdf/main2007/2007496.pdf
National Council Teachers of English. (2007). Mapping diverse literacies for the twenty-
first century: Opportunities, challenges, promising new directions. New York,
National Council Teachers of English.

54
National Council Teachers of English (NCTE) (2008). The NCTE definition of 21 st century
literacies. NCTE Position Statement. Urbana, IL: National Council Teachers of
English. Last retrieved April 27, 2009 from
http://ww.ncte.org/positions/statements/21stcentdefinition.
Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A typology of mixed methods sampling
designs in social science research. The Qualitative Report, 12(2), 281-316.
Onwuegbuzie, A. J., & Leech, N. J. (2007). Sampling designs in qualitative research:
Making the sampling process more public. The Qualitative Report, 12(2), 238-254.
Retrieved March 20, 2008 from http://www.nova.edu/ssss/QR/QR12-
2/onwuegbuzie1.pdf.
Perl, S. (1994). Composing texts, composing lives. Harvard Educational Review, 64(4), 427-
449.
Popper, K. (1962). Conjectures and refutations. New York: Harper and Row.
Richardson, L. (2000). Writing: A method of inquiry. In N. K. Denzin and Y. S. Lincoln
(Eds.), Handbook of qualitative research, 512-29. Thousand Oaks, CA: Sage.
Rogers, A. (2000). Literacy in the information age: Final report of the International Adult
Literacy Survey. International Review of Education, (46)5, 467-473.
Rosenblatt, L. (1938/1995). Literature as exploration. New York: MLA.
Rosenblatt, L. (1978). The reader, the text, the poem: The transactional theory of the literacy
work. Carbondale, IL: Southern Illinois University Press.
Rosenblatt, L. (2005). Making meaning with texts: Selected essays. Portsmouth, Heinemann.
Smagorinsky, P., Cook, L. S., Moore, C., Jackson, A. Y., & Fry, P. G. (2004). Tensions in
learning to teach: Accommodation and the development of a teaching identity.
Journal of Teacher Education, 55(1), 8-24.
Smagorinsky, P. & Whiting, M. E. (1995). How English teachers get taught: Methods of
teaching the methods class. Urbana, Illinois: National Council Teachers of English.
Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage.
Sudzina, M. R. & Knowles, G. J. (1993). Personal, professional and contextual
circumstances of student teachers who fail: Setting a course for understanding
failure in teacher education. Journal of Teacher Education, 44(4), 254-262.
Tobin, L. (1991). Reading students, reading ourselves: Revising the teachers role in the
writing class. College English, 53(3), 333-347.
Tobin, L. (2004). Reading student writing: Confessions, meditations, and rants. Portsmouth,
NH: Heinemann.
Veal, M. L. & Rikard, L. (1998). Cooperating teachers perspectives on student teaching
triad. Journal of Teacher Education, 49(2), 108-119).
Wilson, E. K. (2006). The impact of an alternative model of student teacher supervision:
Views of the participants. Teaching and Teacher Education, 22, 22-31.
Witte, S. P. (1992). Context, text, and intertext: Toward a constructivist semiotic of
writing. Written Communication, 9, 237-308.
Zeichner, K. (2012). The turn once again toward practice-based teacher education. Journal
of Teacher Education, 63(5), 376-382.
Zeichner, K., & Beir, M. (2012). The turn toward practice and clinical experience in U.S.
teacher education. Beitrge zur Lehrerbildung, 30(2), 153-170.

55
Appendix A: Themes Emerging During Data Analysis
Table 1
Teri: Interview Themes
Theme Examples from Teris Case

Guiding Image or Sees herself as a Jedi Knight who is to learn the ways of the
Metaphor for force
Internship
Relationship to Concludes, They hate me; feels like she must be the cool
students one because she started off that way; thinks students view
her as a peer; in another class she must be the prison
warden; speaks of students as both a collective class unit
and as individuals
Relationship to Perceives her situation to be lucky in comparison to
Peers other interns; adopts struggling peer interns; seeks council
of experienced, practicing teachers as professional peers
Relationship with Dr. Grandma is both an experienced, credentialed,
Cooperating professional, who expects results, and a maternal figure to
Teacher turn to for professional support and guidance
Teacher Role Characterized by tension between who she wants to be
and the role she is currently in; Ideally a teacher is a
mentor who facilitates and provides opportunities and
choices for students to engage in personally meaningful
learning; a teacher is someone who provides students with
tools to become successful and to be something big; a
teacher is an agent of change
Primary Stance Student apprentice stance; inclination to master learning
Toward Internship opportunities and objectives; sees herself as a young Jedi
knight who wont go to the dark side
Primary Foci when 1. Classroom management
Teaching 2. Relating to students
3. Knowing when students are learning
Personal Learning Internship has helped her grow up a bit, but she still
questions [her] own maturity and struggles to establish
a balanced relationship with students
Definition of [F]ulfilling the requirements set by [the university],
Successful establishing a good rapport with students and colleagues,
Internship learning about the system from within
Self-Evaluation of Successful with mini failures scattered intermittently
Internship
Most Influential How students respond to her
Aspect of
Internship

56
Table 2
Teri: Synthesized Themes
Theme Examples
Tensions Between Sees her early notions of teaching as nave; feels conflicted
Ideal and Real by what she knows to do and what she actually does. I
know I shouldnt raise my voice; Proximity control is
supposed to work, but students keep talking even though
I stand right there; I know you told us not to threaten
students with writing, but I do
Focus on what She Asserts that she is an excellent planner, yet her focus is on
Struggles to her inability to manage student behavior and manage
Accomplish or student questions in fast-paced honors courses with
Needs to Master multiple projects going on at once
Role of Teacher Defines and differentiates her role in terms of how
Defined by different students respond to her, especially negative
Different Classes responses; in some classes she is the cool peer, the
Responses to Her facilitator or the mentor, and in others she is the
gatekeeper of knowledge or the prison warden
Relationship to Evaluates success of her own conception of teaching in
Others light of how others respond to her
Perception of the Her role is to learn as much as possible; her university
Triad supervisor is an evaluator and a source of information;
her cooperating teacher is a professional guide who is to
make sure specific things are accomplished, who provides
scaffolding for Teris learning, and who acts as a
nurturing source of support.
Stance as An overall synthesis of Teris responses reveals her focus
Student/Apprentice to be on mastering what she is unable to do; yet she is
in a Teaching frustrated by her not knowing how to use what she does
Situation know and how to learn what she does not know; she is a
Jedi Knight and a mentor to her peers
simultaneously; the possibility of teaching every grade
level by the end of the semester thrills her; revises her
prior understanding of teaching to assimilate new
experiences; her perception of a successful internship is
defined in terms of meeting the universitys objectives.

57

Vol. 16, No. 1, pp. 57-71, January 2017
Coping Strategies of Greek 6th Grade Students:

Their Relationship with Anxiety and Trait
Emotional Intelligence
Alexander- Stamatios Antoniou
Department of Special Education and Psychology,
Faculty of Primary Education,
National and Kapodistrian University of Athens,
Athens, Greece
Nikos Drosos
Psychology Department,
Faculty of Philosophy, Pedagogy, and Psychology
National and Kapodistrian University of Athens,
Athens, Greece
Abstract. The aim of this study was to investigate childrens coping strategies and
explore their relationship with anxiety and trait emotional intelligence using a sample of
245 Greek 6th Grade students. Coping strategies are estimated with Kidcope - Children
version (Spirito, Stark, & Williams, 1988). Trait Emotional Intelligence is measured with
the Trait Emotional Intelligence Questionnaire-Adolescent Short Form (TEIQue-ASF)
(Petrides, Sangareeau, Furnhum, & Frederickson, 2006) and anxiety is assessed with
State-Trait Anxiety Inventory for Children (STAIC; Spielberger, 1973). Students reported
wishful thinking and positive coping as their most frequently used coping
strategies, while blame and anger is the least used strategy. The results, also, show
that trait emotional intelligence is positively correlated with positive coping and
negatively correlated with social withdrawal, blame and anger, passive
acceptance/ distraction, and wishful thinking. On the other hand, both trait and state
anxiety were positively correlated with social withdrawal, wishful thinking, and
blame and anger, and positively correlated with positive coping. The results are
discussed in terms of their implications for future research and early adolescents
counselling.
Keywords: Childrens coping strategies, Trait Emotional Intelligence, State and Trait Anxiety
Introduction
Late childhood and early adolescence are associated with changes in
almost all domains of life: social, physical, cognitive and educational. These
changes are often the cause of heightened stress. There are also other issues that
contribute to childrens everyday stress, for example, negative events in their

58
environment and exposure by the media (economic crisis, refugee crisis, crimes,
wars, etc.), family issues (financial problems, marital problems, divorce, illness,
etc.), and school issues (peer interactions, academic achievements, bullying, etc.).
Transition to Junior High school is a significant source of potential stress (Elias et
al., 1992; Munsch & Wampler, 1993). This transition is accompanied with major
changes in both social and academic contexts. Junior High school is much more
demanding and complex than Primary Education School, while parents
demands for academic success increase (Eccles et al. 1993). At the same time
children will have to enter a new learning and social environment, start
developing a new peer network and adjust to the demands of the new teachers.
In Greece, the economic crisis has affected all major sectors of Greek
social and economic life. According to Eurostat (10/2016) unemployment has
reached an extremely high rate of over 23% while in the same period of 2008 it
was reported as being under 8%, with an estimated 44% of the population living
below the poverty line. Apart from the decrease of employment rates, the
quality of employment has also deteriorated, as many individuals who remain
employed are often forced to accept reduced working hours, and/ or lower
salaries and benefits. An up to 40% reduction in salaries and pensions has
forced Greeks to change their life style and adjust to a new reality within a very
short period of time. These changes reflect an increase in psychological problems
(Economou et al, 2011; Giotakos, 2010; Giotakos, Karampelas, & Kafkas, 2011);
and have various negative effects on family life. As stressed by Takeuchi,
Williams, & Adair (1991) long-term financial difficulties have often major effects
on parents. They often exhibit symptoms of stress such as deterioration in
physical health, and marital problems, while their parental behavior declines.
The aforementioned issues are often felt by the children, thus increasing their
own feelings of anxiety and stress (Wagner et al., 2015).
Anxiety and stress

The various stressful situations that children face may lead to feelings of
anxiety. Anxiety can be defined as feelings of unease, worry, tension, and stress
(Nolen-Hoeksema, 2011). Anxiety is not directly caused by stressors, but it is
rather the reaction to the perception of stressors. Cattell and Scheier (1958, 1961)
proposed a distinction between state and trait anxiety, based on factor analytic
studies. Trait anxiety refers to a relatively stable and permanent characteristic of
an individual, while state anxiety refers to a transient condition that may vary
from day to day. In the same concept of ideas Spielberger and his colleagues
(1970) conceptualized trait anxiety as an acquired behavioral disposition that
reflects stable individual differences in anxiety proneness; and state anxiety as a
temporary emotional state. Anxiety and stress may manifest with both physical
symptoms and emotional and behavioral changes. Physical symptoms may

59
include headaches, fast heartbeat, muscle tension, upset stomach, back pain, cold
or sweaty hands and feet, chest pain, dry mouth, upset stomach, digestive
difficulties, and sweating (Frank, 2003). Emotional and behavioral changes may
include: changes in appetite, trouble sleeping, feeling of frustration, loss of
interest in previously interesting activities, poor self-esteem and self-efficacy,
concentration problems, excessive worry, fatigue, reduced school productivity,
self-medicating (abuse of drug or alcohol), hyperactivity, difficulty
concentrating, disturbing dreams/ nightmares, irrational anger, and outbursts.
Coping mechanisms
Coping refers to thoughts and/or behaviors that an individual takes in
order to minimize the negative effect that stressful events cause (Lazarus, &
Folkman, 1984). The aforementioned definition of coping includes both changes
that someone can impose on his/herself (e.g. cognitive restructuring) and
changes that someone can impose on his/ her environment in order to minimize
the negative emotional effects that he/she is experiencing. Attempting to
categorize the various coping strategies, most authors recognize two broad types
of coping: approach and avoidant strategies (e.g. Dempsey, Overstreet, &
Moley, 2000; Rosario, Salzinger, Feldman, & Ng-Mak, 2003). Whilst approach
strategies try to address the problem by managing the stressor, avoidant
strategies attempt to assuage the negative emotions by evading the stressor.
Exploring coping strategies used by children in Greece is of major
importance in order to design interventions for empowering them against the
most serious stressful events, such as the effects of the economic crisis. Such an
understanding necessitates an adequate measure of coping among children.
Given the relatively small number of available measures for assessing childrens
coping mechanisms, the present study will investigate the psychometric
properties of Kidcope (Spirito, Stark, & Williams, 1988), which is an instrument
which is utilized worldwide, for measuring coping. . To our knowledge, this is
the first time that this measure has been used among a Greek population.
Trait Emotional Intelligence

According to Coleman (2008) emotional intelligence (EI) is a complex
ability that includes monitoring ones own and other peoples emotions,
differentiating the various emotions and identifying them, and exhibiting the
appropriate thinking and behavior. People with high EI are able to link
intelligence, empathy and emotions; and subsequently to better understand the
dynamics of interpersonal relationships (Mayer, 2008). Nevertheless, there are
still substantial disagreements for the operational definition of EI. Two different
EI constructs were proposed by Petrides and Furnham (2000, 2001, 2003): (a) trait

60
EI (or trait emotional self-efficacy) and (b) ability EI (or cognitive-emotional

stability). Trait EI can be defined as a constellation of emotion-related self-
perceived abilities and dispositions and can be measured with the use of self-
report techniques. On the other hand, Ability EI can be defined as a constellation
of emotion-related cognitive abilities and can be measured with the use of
maximum performance tests.
Emotional Intelligence is considered to be closely linked with effective
coping (Matthew, & Zeidner, 2001; Snyder & Dinoff, 1999). As Furnham,
Petrides, and Spencer-Bowdage (2002) have found, EI seems to be associated
with healthy social coping styles. According to Gohm, Corse and Dalsky (2005)
there is a negative correlation between EI and avoidant coping strategies, such
as alcohol-drug engagement and behavioral disengagement, while there is a
positive correlation between EI and approach coping strategies, such as
seeking social support and religious coping styles. Siegling and his colleagues
(2015) had similar findings, although their study was focusing on the
psychometric characteristics of Trait Emotional Intelligence Questionnaire
Adolescent Short Form rather than exploring Trait EI relationship with coping
styles. Nevertheless, there have been some recent studies that showed no
correlation between ability EI and coping styles (Zeidner, Matthews, & Shemesh,
2016), and further research is needed.
The present study

The present study examines coping strategies that Greek 6th grade
students adopt in order to reduce anxiety. Furthermore, the present study aims
to investigate the level of students anxiety and trait emotional intelligence and
to explore the possible correlations between coping strategies, anxiety and trait
emotional intelligence.
Methodology
Participants
The sample included 245 6th Grade students, 107 (43.7%) of whom were
boys and 138 (56.3%) were girls. All students were enrolled in Greek public
schools. Schools that were selected to take part in the study were derived from
various regions of Athens to ensure that families of diverse socioeconomic status
were included in the sample. We chose to conduct this research among 6th grade
students because within the Greek educational system, sixth grade is the final
grade of Primary Education, whereby students need to prepare for their

61
transition to Junior High School where they will have to face new challenges.
This upcoming transition might function as an extra stressor for children.
Instruments
State-Trait Anxiety Inventory for Children (STAIC; Spielberger, 1973) was used to
measure childrens distress. STAIC is a self-report psychometric tool which
assesses state and trait anxiety of children aged between 8-18 years. STAIC
provides two different scores: state anxiety and trait anxiety. Each scale consists
of 20 items. Children are invited to report how they feel (e.g., calm, tense) at a
particular moment in time (state scale) and how they feel in general (trait scale)
rated on a 4-point Likert-type scale. STAIC was translated and adapted to Greek
by Psychountakis (1995). Both STAIC-State and Trait scales have shown very
good psychometric properties, and several studies with children support their
reliability and validity. Evidence for the construct validity of the STAIC-State
subscale has been presented (Spielberger, 1973). Additionally, STAIC-Trait scale
exhibits relatively high correlations with other psychometric tools that assess
similar constructs, which ensures its concurrent validity. Internal consistency
reliability coefficients in the present study were found to range from 0.82
(STAIC- trait scale) to 0.91 (STAIC- state scale).
Trait Emotional Intelligence Questionnaire Adolescent Short Form (TEIQue-ASF;
Petrides, Sangareeau, Furnhum, & Frederickson, 2006) was used to assess
childrens trait emotional intelligence. TEIQue is a self-report inventory that
assesses emotion-related self-perceived abilities and dispositions. The adult full
version of TEIQue encompasses four factors (well-being, self-control,
emotionality, and sociability) that can be further divided into 15 subscales.
It also provides a score for global trait EI (Petrides, 2009). The Adolescent Short
Form is a more simplified version of the same questionnaire. It includes 30 short
statements that assess the 15 trait EI facets, and can also provide an overall score
for global trait EI. Although scores for the 15 subscales can be calculated,
Petrides recommends the use of the global trait EI only for this version of the
questionnaire. In our study internal consistency reliability coefficient is 0.86.
According to its authors this form of the questionnaire can be addressed to
children as young as 11 years old. It was translated and adapted to Greek by
Petrides, Pita and Kokkinaki (2007). Several studies provided evidence
regarding it incremental validity (e.g. Davis & Humphrey, 2012; Siegling et al.,
2015).
Kidcope Children Version (Spirito, Stark, & Williams, 1988) was used to assess
childrens coping strategies. KIDCOPE is a brief checklist that assesses both
cognitive and behavioral coping strategies. There are two different versions of
Kidcope: one for adolescents (aged 13 to 16 years) and one for younger children

62
(aged 5 to 13 years). The child version that was used in the present study is
comprised of 15 items that measure the use of 10 coping mechanisms
(distraction, social withdrawal, cognitive restructuring, self-criticism,
blaming others, problem solving, emotional regulation, wishful
thinking, social support, and resignation). Although Kidcope is designed
to assess both frequency (i.e. How often did you do this?) and efficacy (i.e.
How helpful was it?) of the use of each strategy, in the present study only the
frequency was measured. A four-point Likert-type scale ranging from not at
all (1) to almost all the time (4) is used. As there are few available
instruments that assess childrens coping strategies, Kidcope is widely utilised,
although there are varying results regarding its psychometric properties and
factor structure. Several different factor structures have been proposed. Among
them, there are: a five-item single factor (Spirito, 1996), two factors (Cheng, &
Chan, 2003; Spirito, 1996), three factors (Spirito, Stark, & Tyc, 1994; Vigna,
Hernandez, Kelley, & Gresham, 2010), and four factors (Vernberg et al, 1996). It
should be noted that even the studies with the same number of factors did not
find the same factor structure. The aforementioned findings suggest that
Kidcopes factor structure is not stable and may vary across diverse samples.
Although the Kidcope measure has been used to measure coping strategies in a
variety of populations, to our knowledge it is the first time that it has been used
in Greece.
Procedure
Special permission to conduct the research in Greek public Primary
Education schools was granted by the Greek Ministry of Education.
Subsequently, members of the research team explained the aim of the study to
the school principals. The children that took part in the study were instructed to
answer all questions with sincerity stressing that the questionnaires are
anonymous, participation is voluntary, and the results will only be used for
research purposes. Questionnaires were answered exclusively in class, while
there were no time constraints.
Results
As Kidcopes factor structure is not well established, factor analyses were
performed in order to examine it. We chose to perform Exploratory factor
analysis (instead of Confirmatory), as previous studies had presented a plethora
of different factor structures in various populations.

63
EFA was performed to investigate Kidcopes internal structure. Factorability of

the 15 items was checked with the use of Kaiser-Meyer-Olkin measure of
sampling adequacy, and Bartletts test of sphericity. Both tests suggested
reasonable factorability [=0.62, for the Bartletts test of sphericity =
456,076 (df=105, p<.001)]. Factor analysis included 15 items for the 245 6th grade
students. Multiple solutions with varying number of factors were examined
using varimax and oblimin rotations as extraction methods. The final solution
(using varimax rotation) revealed six (6) factors, to which 61.67% of the variance
can be attributed; and was preferred because it provided the best interpretation
of the factors. Internal reliability coefficients were rather low (Table 1), but it was
expected due to the small number of items in each factor.
Table 1. Factor loadings based on PCA with varimax rotation for the 15 items of the Kidcope
Factors
Items
1 2 3 4 5 6
Try to sort the problem out by doing something

.75 -.27 -.05 -.11 -.01 -.04
or talking to someone
Try to sort out the problem by thinking of

.71 -.19 .16 .26 .04 .02
answers
Try to see the good side of things .55 .31 -.34 .14 -.13 -.18
Do nothing because the problem cannot be

-.06 .79 .05 -.10 -.02 .02
sorted anyway
Keep quiet about the problem -.26 .57 .18 -.29 -.19 -.21
Try to forget the problem .00 .47 -.16 .21 -.01 .12
Do something like watch TV or play to forget it -.13 .44 .17 -.32 .42 .04
Wish I could make things different .05 -.16 .78 -.15 -.11 .09
Wish the problem never happened -.03 .17 .75 .14 .01 -.09
Blaming someone else for causing the problem -.03 -.01 -.02 -.78 -.05 .02
Shout, scream and get angry -.07 .06 .03 -.75 .02 -.07
Stay on my own -.11 .21 .36 -.05 -.65 -.06
Try to feel better by spending time with others .53 .15 .04 .03 .60 .09
Try to calm down -.13 .01 .08 .12 .12 .85
Blaming myself for causing the problem -.26 -.14 .25 .19 .48 -.56
Percentage of Variance explained (total: 61.67%) 12.84 11.35 10.87 10.70 8.12 7.79
Internal Consistency Coefficients .52 .47 .55 .49 .40 .39

64
Based on the content of the items that constitute each component, they were
correspondingly labeled:
Component 1, labeled Positive Coping, consisted of items that refer to behaviors
focusing on solving the problem and on cognitive restructuring (e.g., try to sort
the problem by thinking of answers, or try to see the good side of things.
Component 2, labeled Passive acceptance/ Distraction, included items that refer to
behaviors of avoidance of the problem such as distraction and resignation (e.g.,
Try to forget the problem, do nothing because the problem cannot be sorted
anyway).
The third component, labeled Wishful Thinking, included items pertaining to
wishful thinking (e.g. wish the problem never happened).
The fourth component, labeled Blame and anger, consisted of items pertaining to
behaviors like blaming others and to emotional outburst (e.g. Blaming someone
else for causing the problem, Shout, scream and get angry).
Component 5, labeled Social withdrawal, consisted of items pertaining to
behaviors such as seeking loneliness, and not spending time with others (e.g.
Stay on my own, try to feel better by spending time with others). One item
that had a reversed factor loading was reverse scored.
Component 6, labeled Emotional Regulation, included items pertaining to
behaviors of remaining calm and not blaming one-self (e.g. try to calm down,
blaming myself for causing the problem. One item that had a reversed factor
loading was reverse scored.
Descriptive statistics were calculated for all variables (trait Emotional
Intelligence, trait Anxiety, state Anxiety and the 6 coping factors). Means and
standard deviations for each gender are presented in Table 2.
Table 2. Means and standard deviations for each gender for all measures
Males Females Total

Variables
M S.D. M S.D. M S.D. Min Max
Trait EI 144.86 27.23 143.14 25.98 143.81 26.43 63.00 203.00
Trait Anxiety 32.35 6.81 35.51 8.23 34.14 7.79 20.00 55.00
State Anxiety 30.71 6.47 32.37 7.90 31.63 7.33 20.00 54.00
Coping factors
Positive Coping 2.98 0.64 2.91 0.70 2.94 0.67 1.00 4.00
Passive
2.50 0.96 2.18 0.65 2.32 0.82 1.00 4.00
acceptance/Distraction

65
Wishful Thinking 2.82 0.83 3.18 0.81 3.02 0.84 1.00 4.00
Blame and Anger 1.85 0.86 1.72 0.70 1.77 0.77 1.00 4.00
Social Withdrawal 2.15 0.80 2.18 0.85 2.17 0.83 1.00 4.00
Emotional Regulation 2.60 2.22 2.43 0.85 2.50 1.60 1.00 4.00
Note. Anxiety (State): Min possible: 20, Max possible: 60. Anxiety (Trait): Min possible: 20, Max
possible: 60, Trait Emotional Intelligence: Min possible: 30, Max possible: 210, Coping factors:
Min possible: 1, Max possible: 4.
One-way MANOVA was conducted with gender as the independent factor

and all measurements as depended variables. There was a statistically significant
difference in measurements scores based on students gender, F (9, 193) = 3.08,
p = .002; Wilk's = 0.874, partial 2 = .15. Gender has a statistically significant
effect on trait anxiety (F (1, 201) = 7.46; p < .01; partial 2 = .04), on passive
acceptance/ distraction (F (1, 201) = 7.34; p < .01; partial 2 = .04), and on wishful
thinking (F (1, 201) = 9.45; p < .005; partial 2 = .05). With regard to trait anxiety,
boys presented lower scores (M=32.35, S.D.=6.81) than girls (M=35.51,
S.D.=8,23). Regarding coping strategies, boys had higher scores for passive
acceptance/ distraction (M=2.50, S.D.=0.96) than their female classmates
(M=2.18, S.D.=0.65), while girls had higher scores for wishful thinking (M= 3.18,
S.D.=0.81) than boys (M=2.82, S.D.=0.83).
Subsequently, correlation coefficients (Pearson r) among the 6 coping factors
and Trait EI, Trait Anxiety and State Anxiety were calculated (Table 3). Trait EI
has a moderate positive correlation with Positive Coping (r=.47), a moderate
negative correlation with Social Withdrawal (r=-.41), and low negative
correlations with Blame and Anger (r=-.36), Passive acceptance/ Distraction (r=-
.31), and Wishful Thinking (r=-.14). Trait Anxiety has a moderate positive
correlation with Social Withdrawal (r=.45), a low negative correlation with
Positive Coping (r=-.25), and low positive correlations with Wishful Thinking
(r=.26), and Blame and Anger (r=.15). State Anxiety has a moderate positive
correlation with Social Withdrawal (r=.44), a low negative correlation with
Positive Coping (r=-.31), and low positive correlations with Wishful Thinking
(r=.23), and Blame and Anger (r=.20).

66
Table 3. Correlation Coefficients (Pearson r) among the 6 coping factors and Trait EI,
Trait Anxiety and State Anxiety
Coping factors Trait EI Trait Anxiety State Anxiety
Positive Coping .47** -.25** -.31**
Passive acceptance/Distraction -.31** .07 .11
Wishful Thinking -.14* .26** .23**
Blame and Anger -.36** .15* .20**
Social Withdrawal -.41** .45** .44**
Emotional Regulation .02 -.02 .02

Note. *p<0,05, **=p<0,01,
Discussion
The purpose of the present study was two-fold: to explore coping
strategies of 6th grade Greek students, and to examine the relationship of coping
strategies with anxiety and trait emotional intelligence. Sixth grade students are
preparing for the transition to Junior High School, and at the same time they are
about to enter adolescence, both of which could function as major stressors.
In order to assess childrens coping strategies, we used Kidcope, a widely
accepted measurement of coping, and we examined its factor structure. Previous
studies had suggested that Kidcopes factor structure is not stable and may vary
across diverse samples (Cheng, & Chan, 2003; Spirito, 1996; Spirito, Stark, & Tyc,
1994; Vigna, Hernandez, Kelley, & Gresham, 2010; Vernberg et al, 1996). The
EFA of Kidcope yielded 6 factors: Positive Coping, Passive Acceptance/
Distraction, Wishful Thinking, Blame and Anger, Social Withdrawal, and
Emotional Regulation. The presence of 6 coping strategies might indicate that 6th
grade students coping strategies are quite differentiated. Although there were
six clear factors, reliability coefficients were relatively poor, which might be
attributed to the limited items that constitute each factor, and the childrens
young age (Altshuler, & Rumble, 1989).
The role of coping in child psychosocial adjustment has become the
epicenter of several studies over the last decades. As various studies have
shown, individuals who adopt active coping strategies, such as problem solving,
are better functioning, in comparison to individuals who prefer coping strategies
that are less active, such as social withdrawal (Endler & Parker, 1990; Peterson,
1989). According to our findings, Greek 6th grade students reported wishful
thinking and positive coping as their most frequently used coping strategies.
Emotional Regulation, Passive acceptance/ Distraction and Social Withdrawal
are used less often, while Blame ad Anger is the least used strategy. These

67
findings suggest that Greek students use mainly functioning strategies that can
result in reducing anxiety. Perhaps it would be interesting in future studies to
examine whether different stressors (e.g. in the family, in school, etc.) result in
the use of the same coping mechanisms.
Our hypothesis that children would demonstrate quite high levels of
anxiety because of the effects of the economic crisis and the transition to high
school and to adolescence was not supported. Students reported rather low
levels of both State and Trait Anxiety. Although the role of parental attitudes in
childrens anxiety is undisputed (e.g. Bogels, & van Melick, 2004; Rapee, 1997;
Sarason et al., 1960; Siqueland, Kendall, & Steinberg, 1996), further research is
needed in order to explore the relationship between economic crisis and parental
attitudes. It should be noted that the stressors adolescents experience may vary
as they originate from different socio-economic backgrounds. Furthermore, 6th
grade students are preparing for the transition to high school, but the transition
has not yet taken place, and they are still in a familiar environment. Perhaps the
transition related stress will manifest within the next year, when the children
actually begin attending high school. One of the possible explanations for the
low levels of anxiety could be associated to the functioning coping strategies that
are used.
There was no sex difference identified for State Anxiety, while for Trait
Anxiety females reported higher levels than their male counterparts. There are
several studies reporting preponderance of females in showing anxiety (e.g.
Cole, Martin, Peeke, Seroczynski, & Fier, 1999; Lewinsohn, Gotlib, Lewinsohn,
Seeley, & Allen, 1998). Multiple explanations have been proposed to explain this
difference. McCauley Ohannessian and her colleagues (1999) associated anxiety
with self-competence, and found evidence that the correlation between sex and
trait anxiety decreased when the variance explained by self-competence was
taken into account. Thus, these authors hypothesized that self-competence is a
partial cause for the observed sex differences in depression and anxiety in early
adolescents. Other studies (e.g. McLean, & Anderson, 2009) highlight the
complexity of factors (biological influences, temperamental factors, cognitive
factors, environmental factors) that might contribute to female predominance in
anxiety.
The current study suggests that trait Emotional Intelligence levels were
moderately high. This finding emerges in accordance to the low levels of
anxiety, and the high frequency of using functioning coping strategies. As
hypothesized, trait Emotional Intelligence was positively correlated with
positive coping, and negatively correlated with social withdrawal, blame and
anger, and Passive acceptance/ distraction. These findings are consistent with
previous studies (e.g. (Matthew, & Zeidner, 2001; Snyder & Dinoff, 1999;
Furnham, Petrides, and Spencer-Bowdage, 2002; Gohm, Corse and Dalsky, 2005)

68
that have suggested a positive relationship with healthy coping styles. The
negative correlation of Emotional Intelligence with anxiety was expected
because it is natural that trait emotional intelligence is not consistent with
feelings of unease, worry, tension, and stress.
On the other hand, anxiety was expected to be positively correlated mainly
to social withdrawal, which was the least frequently used coping strategy. This
finding is consistent to previous studies that have shown that poor adjustment at
the end of 6th grade was related to higher levels of school stress and less social
support from family and teachers. Anxiety was also positively correlated to
wishful thinking, blame and anger and negatively correlated to positive coping.
Future research
Our study highlights the necessity for further research in order to
understand the coping mechanisms that Greek children use. Although, students
reported the use of mainly functioning mechanisms, further investigation is
needed to examine whether they exhibit the same mechanisms in different
environments and stressors (e.g. school, family etc.). Students did not appear to
exhibit high levels of stress as we hypothesized (due to their soon-to-be
transition from Primary Education School to Junior High School). Future
research should investigate whether increased stress appears in the first grades
of Junior High School and whether students continue to have functioning coping
mechanisms. Finally, further investigation should focus in the effects of Greeces
current economic situation in parental attitudes and subsequently in the effects
of parental attitudes in their childrens anxiety levels and coping mechanisms.
References
Altshuler, J. L., & Ruble, D. N. (1989). Developmental changes in childrens awareness of

strategies for coping with uncontrollable stress. Child Development, 60, 1337-1349.
Bgels, S.M., & van Melick, M. (2004). The relationship between child-report, parent self-
report, and partner report of perceived parental rearing behaviors and anxiety in
children and parents. Personality and Individual Differences, 37, 1583-1596.
Cattell, R. B., & Scheier, I. H. (1958). The nature of anxiety: A review of thirteen
multivariate analyses composing 814 variables. Psychological Reports, 4
(Monograph Supplement 5), 351-388.
Cattell, R. B., & Scheier, I. H. (1961). The Meaning and Measurement of Neuroticism and
Anxiety. New York: Ronald Press.
Cheng, S. T., & Chan, A. C. (2003). Factorial structure of the Kidcope in Hong Kong
adolescents. Journal of Genetic Psychology, 164, 261-266.
Cole, D. A., Martin, J. M., Peeke, L. A., Seroczynski, A. D., & Fier, J. (1999). Children's
Over- and Underestimation of Academic Competence: A Longitudinal Study of
Gender Differences, Depression, and Anxiety. Child Development, 70(2), 459-473.
Coleman, A. (2008). A Dictionary of Psychology (3rd ed.). Oxford: Oxford University Press.

69
Davis, S. K., & Humphrey, N. (2012). The influence of emotional intelligence (EI) on
coping and mental health in adolescence: Divergent roles for trait and ability EI.
Journal of Adolescence, 35, 13691379. doi: 10.1016/ j.adolescence.2012.05.007
Dempsey, M., Overstreet, S., & Moely, B. (2000). Approach and avoidance coping
and PTSD symptoms in inner-city youth. Current Psychology, 19, 28-45.
Eccles, J. S., Midgley, C., Wigfield, A., Buchanan, C. M., Reuman, D., Flanagan, C., Mac
Iver, D. (1993). Development during adolescence: The impact of stage-
environment fit on young adolescents' experiences in schools and in families.
American Psychologist, 48(2), 90-101.
Economou, M., Madianos, M., Theleritis, C., Peppou, L. E., & Stefanis, C. N. (2011).
Increased suicidality amid economic crisis in Greece. The Lancet, 378, 1459.
Elias, M. J., Urbiaco, M., Reese, A. M., Gara, M., Rothbaum, P. A., & Haviland, M. (1992).
A measure of adaptation to problematic academic and interpersonal tasks of
middle school. Journal of School Psychology, 30, 41- 57.
Endler, N. S., & Parker, J. D. A. (1990). Multidimensional assessment of coping: A critical
evaluation. Journal of Personality Processes and Individual Differences, 58, 844-854.
Frank, K. (2003). The handbook for helping kids with anxiety and stress. Chapin, SC: Youth
Light Inc.
Furnham, A., Petrides, K., & Spencer-Bowdage, S. (2002). The effects of different types of
social desirability on the identification of repressors. Personality and Individual
Differences, 33, 119-130.
Giotakos, O. (2010). Oikonomiki krisi kai psichiki ygeia [Economic crisis and mental
health]. Psichiatriki [Psychiatry], 21, 195-204.
Giotakos, O., Karampelas, D., & Kafkas, A. (2011). Epiptosi tis oikonomikis krisis stin
psichiki ygeia [Effects of economic crisis in mental health in Greece]. Psichiatriki
[Psychiatry], 22: 109-119.
Gohm, C. L., Corser, G. C., & Dalsky, D. J. (2005). Emotional intelligence under stress:
Useful, unnecessary, or irrelevant? Personality and Individual Differences, 39, 1017-
1028.
Lazarus, R. S., & Folkman, S. (1984). Stress appraisal and coping. New York: Springer.
Lewinsohn, P. M., Gotlib, I. H., Lewinsohn, M., Seeley, J. R., & Allen, N. B. (1998).
Gender differences in anxiety disorders and anxiety symptoms in adolescents.
Journal of Abnormal Psychology, 107(1), 109-117.
Matthew, G., & Zeidner, M. (2001) Emotional intelligence, adaptation to stressful
encounters & health outcomes. In R. Bar-On & J. D. A. Parker (Eds.), The
handbook of emotional intelligence. San Francisco, CA: Jossey-Bass/Pfeiffer.
Mayer, J. D. (2008). Emotional intelligence: New ability or eclectic traits? American
Psychologist, 63, 503-517.
McCullough, M. E., & Snyder, C. R. (2000). Classical sources of human strength:
Revisiting an old home and building a new one. Journal of Social and Clinical
Psychology, 19, 1-10.
McLean, C. P., & Anderson, E. R. (2009). Brave men and timid women? A review of the
gender differences in fear and anxiety. Clinical Psychology Review, 29(6), 496-505.
Munsch, J., & Wampler, R. S. (1993). Ethnic differences in early adolescents' coping with
school stress. American Journal of Orthopsychiatry, 63, 633-646.
Nolen-Hoeksema, S. (2011). Abnormal Psychology (5th ed.). New York, NY: McGraw-Hill.
Peterson, L. (1989). Coping by children undergoing stressful medical procedures: Some
conceptual, methodological, and therapeutic issues. Journal of Consulting and
Clinical Psychology, 57, 380-397.
Petrides, K. V. (2009). Technical Manual for the Trait Emotional Intelligence Questionnaires
(TEIQue). London, England: London Psychometric Laboratory.
Petrides, K. V., & Furnham, A. (2000). On the dimensional structure of emotional
intelligence. Personality and Individual Differences, 29, 313-320.

70
Petrides, K. V., & Furnham, A. (2001). Trait Emotional intelligence: Psychometric

investigation with reference to established trait taxonomies. European Journal of
Personality, 15, 425-448.
Petrides, K., & Furnham, A. (2003). Trait emotional intelligence: Behavioural validation
in two studies of emotion recognition and reactivity to mood induction.
European Journal of Personality, 17, 39-57.
Petrides, K. V., Pita, R., & Kokkinaki, F. (2007). The location of trait emotional
intelligence in personality factor space. British Journal of Psychology, 98, 273-289.
Petrides, K. V., Sangareau, Y., Furnham, A., & Frederickson, N. (2006). Trait emotional
intelligence and childrens peer relations at school. Social Development, 15, 537-
547.
Psychountakis, M. (1995). To Erotimiatologio Agchous Katastasis kai Prodiathesis gia paidia
(STAIC): Prosarmogi se Elliniko plithismo [The State and Trait Anxiety Inventory for
Children (STAIC): Adaptation in Greek population]. Thesis. Ekpaidefsi
Psychologikon Aksiologiseon [Training in Psychological Assessment], Amoikto
Psichotherapftiko Kentro [Open Center of Psychotherapy].
Rapee, R. M. (1997). Potential role of childrearing practices in the development of anxiety
and depression. Clinical Psychology Review, 17(1), 47-67.
Rosario, M., Salzinger, S., Feldman, R. S., & Ng-Mak, D. S. (2003). Community violence
exposure and delinquent behaviors among youth: The moderating role of
coping. Journal of Community Psychology, 31, 489-512.
Sarason, S. B., Davidson, K. S., Lighthall, F. F., Waite, R. R., & Ruebush, B. K. (1960).
Parental attitudes. In S. B. Sarason, K. S. Davidson, F. F. Lighthall, R. R. Waite, &
B. K. Ruebush (Eds) Anxiety in elementary school children: A report of research, (pp.
189-233). Hoboken, NJ, US: John Wiley & Sons Inc
Siegling, A. B., Vesely, A. K., Saklofske, D. H., Frederickson, N., & Petrides, K. V. (2015,
June 26). Incremental Validity of the Trait Emotional Intelligence Questionnaire-
Adolescent Short Form (TEIQue-ASF). European Journal of Psychological
Assessment. Advance online publication. http://dx.doi.org/10.1027/1015-
5759/a000267
Siqueland, L., Kendall, P. C., & Steinberg, L. (1996). Anxiety in children: Perceived family
environments and observed family interaction. Journal of Clinical Child
Psychology, 25(2), 225-237.
Snyder, C. R., & Dinoff, B. L. (1999). Coping: Where have you been? In C. R. Snyder
(Ed.), Coping: The psychology of what works (pp. 3- 19). New York: Oxford
University Press.
Spielberger, C. (1973). STAIC preliminary manual. Palo Alto, CA: Consulting
Psychologists Press.
Spielberger, C. D., Gorsuch, R. L., & Lushene, R. E. (1970). Manual for the State-Trait
Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press.
Spirito, A. (1996). Commentary: Pitfalls in the use of brief screening measures of coping.
Journal of Pediatric Psychology, 21, 573-575.
Spirito, A., Stark, L. J., & Tyc, V. L. (1994). Stressors and coping strategies described
during hospitalization by chronically ill children. Journal of Clinical Child
Psychology, 23, 314-322.
Spirito, A., Stark, L. J., & Williams, C. (1988). Development of a brief coping checklist for
use with pediatric populations. Journal of Pediatric Psychology, 13, 555-574.
Takeuchi, D. T., Williams, D. R., & Adair, R. K. (1991). Economic stress in the family and
childrens emotional and behavioral problems. Journal of Marriage and the Family,
53, 1031-1041.
Vernberg, E. M., La Greca, A. M., Silverman, W. K., & Prinstein, M. J. (1996). Prediction
of posttraumatic stress symptoms in children after Hurricane Andrew. Journal of
Abnormal Psychology, 105, 237-248.

71
Vigna, J. F., Hernandez, B. C., Kelley, M. L., & Gresham, F. M. (2010). Coping Behavior
in Hurricane-Affected African American Youth: Psychometric Properties of the
Kidcope. Journal of Black Psychology, 36(1), 98-12.
Wagner, S. L., Cepeda, I., Krieger, D., Maggi, S., DAngiulli, A., Weinberg, J., & Grunau,
R. E. (2015). Higher cortisol is associated with poorer executive functioning in
preschool children: The role of parenting stress, parent coping and quality of
daycare, Child Neuropsychology, DOI: 10.1080/09297049.2015.1080232
Zeidner, M., Matthews, G. & Shemesh, D.O. (2016). Cognitive-Social Sources of
Wellbeing: Differentiating the Roles of Coping Style, Social Support and
Emotional Intelligence. Journal of Happiness Studies, 17: 2481-2501.
doi:10.1007/s10902-015-9703-z

72

Vol. 16, No. 1, pp. 72-83, January 2017
Active Learning Across Three Dimensions:

Integrating Classic Learning Theory with
Modern Instructional Technology
Thaddeus R. Crews, Jr.
Western Kentucky University
Bowling Green, KY 42101
Abstract. Many educational researchers have proposed moving away

from the traditional pedagogical model of long lectures with students as
passive learners, listening and taking notes. There is over a century of
pedagogical and cognition research that strongly suggests learning
occurs best when students take an active role in the construction of their
own knowledge. The call for change has only been accelerated by the
increasing availability of computer-based educational technology and
the exponential growth of digitally available information. This paper
combines the long history of educational research with the rapid
advancements of instructional technologies to build an Active Learning
Model with three active dimensions (teachers, students, and
technology). This model can help improve authentic learning within
multiple disciplines and pedagogies. A Case Study is presented
illustrating how the model helps improve student satisfaction as
reported using standardized teaching evaluation metrics.
Keywords: Constructivism; pedagogy; active learning; problem based

learning; educational technology
Introduction
A century ago John Dewey said, Give the pupils something to do, not something to
learn; and the doing is of such a nature as to demand thinking, or the intentional noting
of connections; learning naturally results (Dewey, 1916, p. 181). Many educators
agree with Dewey that students become more engaged in the learning process
when the learning activity is dynamic, significant, and relevant to their lives.
Cognitive scientists have an explanation for this, reporting that neural pathways
in the brain are rewarded by the release of neurochemicals (such as dopamine)
when student learning is active, meaningful, and authentic (Doyle, 2012). In this
way, neuroscience is independently confirming the findings of multiple
generations of educational researchers in the areas of effective teaching and
learning.
Technology is doing more than just confirming classic research. Modern

educational technologies are providing new opportunities to support the efforts
of improving teaching and learning. This report extends a classical model for

73
engaged active learning by adding a third dimension that represent the

increasingly important role of technology.
Classic Educational Research: The Roles of Teacher and Learner

Educational research has a rich history of investigation into instructional designs
for improving teaching effectiveness. Reflections on education go back at least to
the Greek philosopher Socrates who is often recognized as the first great teacher
(Nelson, & Brown, 1949). The Socratic method is a form of inquiry and debate
based on asking and answering questions to stimulate critical thinking and to
illuminate ideas. The Socratic method placed emphasis on the status of the
individual to challenge the polity of the state, and the consensus of society. It
was a thus a precursor of modern western intellectual individualism (Anthony,
2006). The skills of critical thinking and of challenging assumptions are just as
useful today (if not more so) as they were 2500 years ago when Socrates
developed his students in these areas.
Another important cognitive theory that impacts education is constructivism,

based on the idea that learning improves when students construct their own
understanding through experiencing things and reflecting on those experiences.
According to the theory, when a person encounters something new, it has to be
reconciled with the persons previous ideas and experiences. This reconciliation
might cause the learner to change what they previously believed, to discard the
new information based on previous experience, or to refine their knowledge if
the new experience is compatible with prior experience and understanding
(Anzai & Simon, 1979). In the classroom, the constructivist view of learning
leads to a variety of teaching practices that encourage active techniques for
creating new knowledge and reflection to explain how ones understanding is
changing. Similar to a Socratic method, the teacher may guide the learner
through the activity by asking foundational questions and encouraging the
student to then build on them.
One advantage of active learning is that it avoids the problem of inert

knowledge, which is information the learner possesses but cannot apply. To
understand inert knowledge, consider a student who learns
Distance/Rate/Time formulas (e.g., D = R * T and T = D / R and R = D / T)
during math class through drill-and-practice on a worksheet of word problems,
pulling numbers from the paragraph and plugging them in as the (hopefully)
appropriate variable in the (hopefully) appropriate formula. But later in the day
when learning about Charles Lindbergh and the Spirit of St. Louis in history
class, the student is asked to figure out how long it took to complete the first solo
non-stop transatlantic flight. When the student says she does not know, the
teacher asks What information would you need to determine how long the
flight lasted? The student does not realize to ask for the distance travelled or
the rate of the plane because the Distance/Rate/Time formulas were learned as
memorized inert information rather than active knowledge (Whitehead, 1929;
Gick & Holyoak, 1980; Bransford, Franks, Vye, & Sherwood, 1989).

74
Active Learning
The best way to avoid inert knowledge is to acquire knowledge as part of an
active learning process. In the process of learning, the learner's dynamic
cooperation is required (Gragg, 1940). Such cooperation from students does not
arise automatically, however. It has to be provided for and continually
encouraged. (Duncker, 1945). Active learning goes beyond memorization of facts
and builds on the students' preexisting conceptions (Bonwell & Eison, 1991).
Active learning activities involve opportunities for students to view information
as means to important ends, which in turn helps students learn about the
conditions under which knowledge is useful (Simon, 1980). When knowledge is
learned actively, it increases the chances of spontaneously using that knowledge
to solve new problems that are confronted later on (Bransford et al., 1990).
Modern neurological science further supports the idea of active learning:
Fifteen years of neuroscience, biology, and cognitive psychology research
finding on how humans learn offer this powerful and singular conclusion: It is
the one who does the work who does the learning. (Doyle, 2012, p. 7)
One example of active learning is problem based learning which often involves
a process similar to the four problem solving stages as stated by Polya: (1)
understanding the problem, (2) making a plan, (3) executing the plan and (4)
reviewing the solution (Polya, 1945). Problem-Based Learning (PBL), as a general
model, was refined in medical education in the early 1970's and since that time it
has become common practice in most medical schools where PBL is used in the
first two years of medical science curricula, replacing the traditional lecture
based approach to anatomy, pharmacology, physiology, etc. (Savery and Duffy,
1995). The model has been adopted in an increasing number of other areas
including Business Schools (Milter & Stinson, 1995), Schools of Education
(Bridges, 1992; Duffy, 1994); Architecture, Law, Engineering, Social Work (Boud
& Feletti 1991); and high school (Barrows & Myers, 1993). As Bransford et al.
state, Problem-oriented acquisition helps students appreciate the value of
information (Branford et al, 1990, p.121).
Great Teachers in 1966

The educational concepts of active learning and problem based learning are old
ideas that have been successfully applied in a wide variety of classrooms over
the last century. On May 6, 1966, Time Magazine did a cover story on 10 highly
effective college professors from ten different universities and who taught in ten
different academic disciplines (Figure 1). These professors may not have been
aware of the active learning insights of James (1899), Thorndike (1913), and
Dewey (1916), and they certainly were unaware of the future insights from
neuroscience regarding active learning from the likes of Goswami (2006), Willis
(2010) and Sigman et al. (2014). Nonetheless, these Time magazine professors
were successful as teachers because they knew the critical importance of
engaging their students with the material.

75
Figure 1. Time Magazines Great Teachers Cover, 1966
These Best Teachers of 1966 were each unique in their situations, teaching
different subject disciplines to different student populations at different
universities. What they had in common was intentional efforts to create a space
where students would be engaged in highly interactive and authentic learning.
This space can be captured visually in two dimensions where one axis represents
the activity level of the teacher and the second axis represents the activity level
of the learner (see Figure 2). In this model, the lower-left quadrant represents the
Traditional Classroom where the student is a passive learning (listening and
taking notes) and the teacher is also a passive participant (non-stop lecturing).
This is not meant to be viewed as an indictment of lectures or the traditional
classroom model. Rather it is a visual summary of the opportunities to make
students more active in their education process, and also to make the instructor
more active as well.

76
Figure 2: Active Learning Model for Great Teachers (circa 1966)
It is important to note that Figure 2 does not suggest any specific approach to
active or authentic learning. Indeed, there is no one size fits all
recommendation because effective learning depends on many factors, including
the content being learned, the knowledge level of the learners, the class size, and
the learning objective just to name a few. Specific techniques might work better
in some situations than others. The good news is that there are many techniques
designed to help participants (students and teachers alike) become more active
in the learning process. If a teacher were to plot a specific instance of a course
onto the Figure 2 chart, the location of that class would not be static, but would
make many small moves over time based on the dozens of choices made in the
design and refinement of the course, each decision moving the class slightly
toward the upper-right quadrant or slightly toward the lower-left quadrant, or
somewhere in-between.
Computing Technology in Education

Research efforts to explore and understand the impact of computing technology
in education has a longer history than one might think. Before a man walked on
the moon, Atkinson and Wilson edited a book of research efforts in the area of
computer-assisted instruction (Atkinson & Wilson, 1969). Over a decade before
the introduction of the IBM personal computer, Seymour Papert and others were
developing tools to allow children to develop hands-on computer programming
skills at a time when computers were the size of refrigerators and cost tens of
thousands of dollars (Papert & Solomon, 1971). In the early 1980s, Lepper
analyzed four sets of research issues raised by the rapid intrusion of
microcomputers into the lives of children, including the use computers as a
vehicle for intrinsic motivation, the study of the instructional effectiveness in
educational software, contrasting philosophies of instruction in different designs

77
of computer-based educational programs, and the effects of the computer on the

goals of formal education (Lepper, 1985). These and similar investigations at the
time are particularly impressive as they occurred before it was clear that
computing technology was certain to play some type of role in education, and
well before the World Wide Web came into existence in 1991 and the resulting
exponential growth of information in the two-plus decades that have followed.
Todays computing technology is becoming ubiquitous, and as such we cannot

leave it out of our discussion about Active Learning and Great Teachers. An
interesting thing happens when technology is added as a third dimension in the
Active Learning model (see Figure 3). As with the teacher and student
dimensions, emphasizing interaction with technology moves the class away
from the passive traditional classroom and toward a more authentic and
personalized learning experience.
Figure 3. The 3D Active Learning Model in Three Dimensions
This model has implications for a variety of high profile educational technology
issues. For example, enrollments in online classes have grown significantly
(Clark & Mayer, 2007) and many traditional institutions are expanding their
capacity for online courses (Allen & Seaman, 2013). Likewise, hybrid courses
that blend face-to-face and online instruction are one the rise (Bonk & Graham,
2006; Keengwe, Onchawari, & Oigara, 2014). Too often faculty are not provided
sufficient guidance on how to integrate technology into these new online or
hybrid courses.
It is possible to replace face-to-face lecture with recorded lecture, but that does
not move the learning experience out of the lower-left quadrant of the Active
Learning Model. Using technology to replicate the traditional classroom might
be cost effective, but it does not improve learning. When technology is

78
introduced with the goal of increasing the interactivity of the learning

experience, educational technology begins to fulfill its promise. This is consistent
with a study by Crews and Butterfield where students identified interaction
through class discussions and other types of active learning activities as the
single greatest predictor of success for a face-to-face course (Crews, Butterfield
2014).
CASE STUDY: Applying the 3D Active Learning Model to a Computer

Literacy Course
The 3D Active Learning Model has been implemented and revised over multiple
semesters in a college of business service course on computer literacy. The class
covers traditional literacy content on computer hardware, computer software,
and computer networks. The course also requires students to complete
numerous hands-on projects with Microsoft Word, Microsoft PowerPoint and
Microsoft Excel. This is significant amount of material for a single class, with
some schools having required textbooks that total over 700 pages of reading
material. Furthermore, students enter the class with diverse backgrounds
regarding computer knowledge and skills, which makes it even more
challenging to teach in a way that the majority of the students are challenged
and engaged.
For this case study, technology has been incorporated into this course according
to the 3D Active Learning model with the goal of moving the course toward the
highly interactive and authentic learning space. The first active technology
introduced in this course was SAM Projects, an expert system for grading
student projects in Microsoft Word, PowerPoint and Excel (Cengage, 2016). This
technology is a type of expert system that allows students to get detailed
feedback on their hands-on Microsoft Office projects. SAM Project grades the
students work and provides detailed feedback almost immediately regardless of
the time of day. When using SAM, a student may submit an assignment at 2:00
am, and the student will receive immediate detailed expert feedback while the
project is still fresh in the students mind. Furthermore, students are able to use
the feedback to resubmit their work multiple times (as determined by the faculty
member) and to improve their work based on the expert feedback.
A second technology element used in this revised course is recorded lectures,

which are increasingly common in a variety of online and flipped classrooms
(Thompson, 2011). Recorded lectures themselves are not more interactive than a
classroom lecture, but they lecture delivery to occur outside of class, which
allows face-to-face classroom time to be repurposed to support more active
learning activities. Using technology outside of the classroom to increase the
human interaction within the classroom is consistent with the book Teaching
Naked: How Moving Technology out of your College Classroom Will Improve
Student Learning (Bowen, 2012) that won 2013 Best Book on Higher Education
from the American Association of Colleges and Universities.
A third technology element incorporated in this Case Study was to increase

interactivity through online practice quizzes. After each assigned chapter

79
reading, an online practice quiz was made available to the students containing a
pool of questions first drafted by the publishers and edited and pruned by the
faculty member. These questions are made available for a period of time (e.g., 1
week) and the students could take the practice quiz multiple times during that
week.
Strategic technology adoptions are only part of this case study. Lectures became
shorter and more interactive, emphasizing the major challenges and
opportunities that students will be facing. Student participation was increased.
Small group projects, discussions, debates, class-wide problem solving activities,
and in-class demonstrations were all considered based on the main learning
objectives for that material and the technique that seemed most intuitive to the
faculty member.
The 3D Active Learning model is not a one-size-fits-all solution. It involves

treating each learning objective uniquely with the goal of increasing interactivity
by the student, the faculty member and the available technology. It involves a
series of small steps that cumulatively help move the course toward the desired
furthest quadrant where active learning is at its highest.
Results
The course revisions over three semesters based on the 3D Active Learning
Model have had a demonstrably positive effect. Students showed an increase in
attendance, alertness, participation, and attitude. Class meetings were more
engaged and pleasant.
In addition, students reported higher satisfaction with various elements of the

course as captured by the universitys standardized Student Instructor Teaching
Evaluation (SITE) scores. Students rated the professor of the 3D Active Learning
course above college and departmental averages across all questions in the SITE
evaluation over each of the past three semesters.
Furthermore, student evaluations show a positive increase over the past three
semesters while college and departmental averages remain relatively flat (with
spring 2015 included as a baseline). By increasing engagement, students felt the
learning was more authentic and in line with the course objectives (Figure 4),
feedback (Figure 5), teacher effectiveness (Figure 6), and student learning
(Figure 7).

80
Figure 4. Improved SITE evaluations regarding learning objectives
Figure 5. Improved SITE evaluations regarding feedback

81
Figure 6. Improved SITE evaluations regarding teacher effectiveness
Figure 7. Improved SITE evaluations regarding student learning
Conclusion
There is a long and rich history of educational research showing improved
student learning through increased interaction and engagement. Each teacher,
each student, each topic, and each setting is unique. But with each of these
unique challenges, there are multiple opportunities to make small changes in
course design and implementation with the goal of moving students toward the
most authentic and interactive learning space possible.
The 3D Active Learning Model provides a visualization for faculty looking to

increase the level of active learning in their course. The model is especially
helpful for faculty incorporating technology in the form of online or flipped

82
classes. The 3D Active Learning Model does not recommend any specific activity
or event or technology. Rather it is a model that encourages incremental change
away from a traditional classroom toward an environment of more authentic
learning involving increased interaction across three dimensions: teacher,
learning, and technology. It is a journey over time that requires a series of
pedagogical decisions where each choice is guided by the goal of increasing
active learning. Collectively these small changes have the effect of creating a
learning space that is well suited for better teaching and more authentic
learning.
References
Allen, E., & Seaman, J. (2013). Changing Course: Ten Years of Tracking Online Education in
the United States, Babson Survey Research Group. Retrieved from
http://www.onlinelearningsurvey.com/reports/changingcourse.pdf
Anthony, M. (2006). A genealogy of the western rationalist hegemony. Journal of Futures
Studies, 10(4), 25-38.
Anzai, Y. & Simon, H.A. (1979). The theory of learning by doing, Psychological Review, 86,
124140.
Atkinson, R. C., & Wilson, H. A. (1969). Computer-Assisted Instruction: A Book of Readings.
New York: Academic.
Barrows, H. S., & Myers, A. C. (1993). Problem-based learning in secondary schools.
Unpublished monograph. Springfield, IL: Problem-Based Learning Institute, Lanphier
High School and Southern Illinois University Medical School.
Bates, A., & Poole, G. (2003). Effective Teaching with Technology in Higher Education. San
Francisco: Jossey-Bass/John Wiley.
Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as
effective as one-to-one tutoring. Educational researcher, 13(6), 4-16.
Bonk, C., & Graham, C. (2006). The Handbook of Blended Learning Environments: Global
Perspectives, Local Design. San Francisco: Jossey-Bass/Pfeiffer
Bonwell, C.C. & Eison, J.A. (1991). Active learning: Creating excitement in the classroom.
ASHE-ERIC Higher Education Report No. 1. Washington, DC: ERIC Clearinghouse
on Higher Education, The George Washington University.
Boud, D., & Feletti, G. (1997). The challenge of problem-based learning. Psychology Press.
Bowen, J. (2012). Teaching naked: How moving technology out of your classroom will improve
student learning. Hoboken.
Bridges, E. M. (1992). Problem Based Learning for Administrators. ERIC Clearinghouse on
Educational Management, University of Oregon, 1787 Agate Street, Eugene, OR
97403-5207.
Bransford, J. D., Franks J. J., Vye, N. J., Y Sherwood, R. D. (1989). New approaches to
instruction: Because wisdom cant be told. In S. Vosniadou & A. Ortony (Eds.),
Similarity and analogical reasoning (pp. 470-497). New York: Cambridge
University Press.
Bransford, J. D., Sherwood, R. D., Hasselbring, T. S., Kinzer, C. K., & Williams, S. M.
(1990). Anchored instruction: Why we need it and how technology can help. In
D. Nix & R. J. Spiro (Eds.), Cognition, education, and multimedia: exploring ideas in
high technology. Hillsdale N.J.: L. Erlbaum.
Cengage Learning (2016). SAM Instructor User Manual. Retrieved from
http://assets.cengage.com/pdf/gui_sam-inst-comp-user-guide.pdf
Clark, R., & Mayer, R. (2007). eLearning and the Science of Instruction. San Francisco:
Pfeiffer

83
Crews, T. & Butterfield, J. (2014). Data for Flipped Classroom Design: Using Student
Feedback to Identify the Best Components from Online and Face-to-Face
Classes. Higher Education Studies; 4, 3, 38-47.
Dewey, J. (1916). Democracy and Education: An Introduction to Philosophy of Education.
Macmillan.
Doyle, T. (2012). Learner-centered teaching: Putting the research on learning into practice.
Stylus Publishing, LLC..
Duncker, K. (1945). On problem-solving. Psychological monographs, 58(5).
Duffy, T. M. (1994). Corporate and Community Education: Achieving success in the
information society. Unpublished paper. Bloomington, IN: Indiana University.
James, W. (1899). Talks to teachers on psychology: And to students on some of life's ideals. New
York: Henry Holt and Company
Keengwe, J., Onchawari, G., & Oigara, J. (2014). Promoting Active Learning through the
Flipped Classroom Model. Hershey PA: IGI Global.
Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive psychology,
12(3), 306-355.
Goswami, U. (2006). Neuroscience and education: from research to practice?. Nature
reviews neuroscience, 7(5), 406-413.
Gragg, C. I. (1940). Because wisdom can't be told. Harvard Business School Publishing.
Lepper, M. R. (1985). Microcomputers in education: Motivational and social issues.
American Psychologist, 40(1), 1.
Nelson, L., & Brown, T. K. (1949). Socratic Method and Critical Philosophy Selected
Essays.
Milter, R. G., & Stinson, J. E. (1995). Educating leaders for the new competitive
environment. In Educational innovation in economics and business administration
(pp. 30-38). Springer Netherlands.
Papert, S., & Solomon, C. (1971). Twenty things to do with a computer.
Polya, G. (1945). How to Solve It. Princeton University Press.
Savery, J. R., & Duffy, T. M. (1995). Problem based learning: An instructional model and
its constructivist framework. Educational technology, 35(5), 31-38.
Sigman, M., Pea, M., Goldin, A. P., & Ribeiro, S. (2014). Neuroscience and education:
prime time to build the bridge. Nature neuroscience, 17(4), 497-502.
Simon, H. A. (1980). Problem Solving and education. In D. T. Tuma & R. Reif (Eds.),
Problem solving and education: Issues in teaching and research (pp. 81-96). Hillsdale,
JR: Lawrence Erlbaum Associates.
Thompson, C. (2011). How Khan Academy is changing the rules of education. Wired
Magazine, 126, 1-5.
Thorndike, E. L. (1913). Educational psychology: the psychology of learning. New York:
Teachers College Press.
Willis, J. (2010). The current impact of neuroscience on teaching and learning. Mind, brain
and education: Neuroscience implications for the classroom, 45-68.
Whitehead, A. N. (1929). The aims of education. New York: MacMillan.

84

Vol. 16, No. 1, pp. 84-102, January 2017
The Effects of Cram Schooling on the Ethnic

Learning Achievement Gap: Evidence from
Elementary School Students in Taiwan
Yu-Chia Liu
Department of Hospitality & Tourism, Taiwan Hospitality & Tourism College,
Taiwan R.O.C. and Department of Curriculum Design and Human Potential
Development, National Dong Hwa University, Taiwan R.O.C.
Chunn-Ying Lin, Hui-Hua Chen

Department of Early Childhood Education, National Dong Hwa University,
Taiwan R.O.C.
He Huang
Department of Clinical Psychology, Chang Gung University, Taiwan R.O.C.
Abstract. For the last three decades, many studies have found an
obvious achievement gap between non-minority students and minority
students, which was mainly associated with a lower socioeconomic
status and a deficiency of family learning resources, such as learning
after school, of minority students. In many countries, cram schooling is
the most commonly extra learning activity which is believed to have
positive effects on learning achievement. However, there are few
empirical studies to explore the relationships between cram schooling
and the learning achievement gap of different ethnic students. This
study used 630 fifth-grade students in Taiwan as samples and carried
out a hierarchical regression analysis to discover the effects of cram
schooling on the ethnic learning achievement gap of Taiwans young
children. The results showed that cram schooling participation has
non-linear effects, first ascending and then descending, on students
learning achievement. In addition, those students who participated in
privately tutored classes with higher fees charged demonstrated better
learning achievement, but students enrolling in after-school programs in
cram schools might not show the same outcomes. Further analysis
indicated that minority children had fewer and poorer cram schooling
resources than non-minority children. In addition, most minority
children have lower socioeconomic status, giving them less opportunity
to participate in cram schooling activities and after-class programs in
schools, so their learning achievement was significantly lower than
non-minority children.
Keywords: cram schooling, ethnic learning achievement gap,

elementary school students, hierarchical regression analysis

85
Introduction
Over the past few years, a number of researchers have concentrated on
analyzing the academic achievement of students in different ethnic groups, and
discovered that the learning gap between minority students and their
counterparts of non-minorities remains constant (Brown-Jeffy 2009; Byun and
Park 2012; Rowley and Wright 2011). Some of them suggest that the minorities
learning performance falls off because of the lack of extracurricular and learning
activities after school. Among the researches focusing on the relationship
between extracurricular activities and the ethnic learning achievement gap, the
function of academic cram schooling has aroused sincere consideration. Cram
school is believed to have a positive effect on academic achievement, and has
become a very visible worldwide phenomenon (Bray 2013; Bray, Zhan, Lykins,
Wang and Kwo 2014; Kim and Park 2010; Kuan 2011; Kenayathulla 2013; Zhan,
Bray, Wang, Lykins and Kwo 2013 ).
Cram schools have also existed in East Asian countries for a long time (Bray et
al. 2014; Fung 2003; Lee and Shouse 2011), particularly those countries
influenced by Confucianism, such as Taiwan, mainland China, South Korea and
Japan. Traditional education values may make cram schools more crediable as
an fundamental and much needed social organization (Liu 2012). Moreover,
cram schooling activities may be influenced by parental socioeconomic status,
thus affecting plurality and deprived students to face greater learning
achievement gaps when compared to those who attend cram schools (Kim and
Park 2010; Lin, Hsieh and Chen 2015; Lee and Shouse 2011).
Aboriginal students are a minority group of the population in Taiwan. The

learning weakness of aboriginal students has been proven and is under constant
analysis (Kuan 2011; Liu 2012). In the past 20 years, the population composition
in Taiwan had changed enormously. This is the result of Taiwanese males
marrying females from mainland China and southeast Asia due to their low
socioeconomic status or other reasons. Therefore, Taiwan has a considerable
number of children of cross-national marriage in all levels of schools. These
children are more likely to fall behind their classmates in learning performance,
but still little research has considered such possibilities and the effects of
cross-national marriages. As a result, the academic field has limited
understanding of how the family composition of cross-ethnic or cross-national
marriages may affect a childs learning outcome.
Based on the above considerations, this study used 630 fifth-grade Taiwanese
students as samples, and carried out a hierarchical regression analysis. The
researchers incorporated ethnic cross-national marriages, family background,
and participation in academic cram schooling (time spent, expenditures, and
patterns) as the mediating variables. These factors were then used to clarify the
influential mechanisms of ethnic and cross-national marriages differences on
students academic achievement. Hopefully this investigation will fill in the gaps
in current research on the education achievement of different ethnic groups so

86
that we can better understand both the theoretical and practical implications of
ethnic achievement gaps.
Literature Review
Cram schooling is similar to shadow education (Bray et al. 2014; Kuan 2011; Lee
and Shouse 2011). This is due to its coexistence with the mainstream schooling
and how it mimics the regular school system, i.e. it duplicates the curriculum,
the regulations, even the purposes of formal schooling. Cram schooling offers
supplementary instruction aiming to assist students to catch up, keep up, or
make headway of their peers. Furthermore, it is a fee-paying service for extra
learning activities after school or during summer vacation, rather than the
unpaid tutoring provided by family members or teachers in after-class
programs. Additionally, cram schooling mainly focused on how to enhances
students performance in terms of academic subjects (Liu 2012; Zhan et al. 2013),
rather than extra-curricular activities to foster their cultural capital, i.e., music,
art, sports, and so on (Shih & Yi 2014). In this study, cram schooling includes
languages (Chinese and English), mathematics, science, and other academic
subjects that feature in mainstream school examinations. All of these subjects are
important branches of learning in the senior high school and university entrance
examinations. In this respect, cram schools consist of outside school instruction,
provided by profit-oriented organizations (Liu 2012), aiming to help students
master academic subjects of school curriculum and improve academic
performance in school (Byun and Park 2012; Lin et al. 2015), earn admission to
elite schools (Fung 2003) and benefit in their future occupations and the social
status (Stevenson and Baker 1992).
Cram schools are also known as Buxiban in Taiwan, Juku in Japan, Hagwon in
South Korea, and private tuition or a shadow education system in Western
countries (Bray 2013; Byun and Parker 2012; Kim and Park 2010; Kuan 2011).
Though cram schools have many different names, they share a common feature
in that they serve as a remedial or enrichment strategy. In Taiwan, two major
types of cram schools can be categorized: (a) the cram schools that help students
do schoolwork and prepare for entrance exams; (b) the institutes that provide
training in foreign languages (Liu 2012; Shih and Yi 2014). These two types of
cram schools can be regarded as academic cram schools due to the primary focus
on students educational advancement, such as languages, writing, and
mathematics, with the primary objectives of improving students abilities for
taking tests, helping them gain higher grades, and enter prestigious high schools
and universities. Attending cram schooling can increase academic performance,
thereby opening up more opportunities for higher education.
Some empirical studies reveal that cram schooling has a positively significant
influence on students academic achievement (Dang 2007; Kuan 2011; Liu 2012).
Rhy and Kang (2013) illustrated that the true effect of private tutoring on
academic performance remains, at most, modest. Specially, Chen and Hwang
(2011) found a non-linear influence, first increasing and then decreasing, on the
hours spent in cram schools. They explained that students fatigue caused by
long cramming hours may decrease the learning efficiency, and that the long

87
hours spent cramming may decrease the time expended on schoolwork.

However, Zhang (2013) doubted the effectiveness of private tutoring. In his
study, he found trivial association between private tutoring and students
academic performance. Therefore, the relationship between academic cram
schooling and achievement still needs to be investigated thoroughly.
It needs clarification that cram schools in Taiwan are usually private and
authorized by the local government. Their objectives are not only to assist
students with their learning, but also to earn a profit. These cram schools are
consequently fee-based. Moreover, students in Taiwan need to go to different
types of cram schools for better scores on tests, leading to a vast household
investment in cram schooling. The more famous cram schools produce better
student performance, attract more parents and their children, and charge higher
tuition fees, causing a heavy economic burden on less wealthy families (Lin et al.
2015). Accordingly, whether children can participate in private cram schools and
how many subjects they can take closely relate to the socioeconomic status of the
families.
For instance, several studies have found that the higher the parental education
level, the more academic subjects their children take at cram schools (Bray et al.
2014; Jung and Lee 2010; Kenayathulla 2013). Moreover, children from high
socioeconomic status families are more likely to take part in academic cram
schooling activities. Their families are also more likely to spend money on
private tutoring (Bray et al. 2014; Jung and Lee 2010; Kim and Park 2010;
Kenayathulla 2013; Lin et al. 2015; Shih and Yi 2014; Stevenson and Baker 1992).
It may be that if there were more cram schools, the opportunities for students
from lower socioeconomic status to attend cram schools and take cram courses
would be greater.
There are minority ethnic groups in Taiwan: aborigines and cross-national

marriage families with the mothers from mainland China and several
southeastern Asia countries. These cross-national marriage families are named
new inhabitants. The population of these minorities is less than ten percent
and generally possesses lower socioeconomic status compared to their
counterparts. In particular, most of the aborigines inhabit mountains or remote
areas, engage in manual labor jobs, earn less income, possess less education, and
therefore can provide their children with less educational resources. Similar to
Western research results of learning racial/ ethnical gap (Brown-Jeffy 2009;
Rowley and Wright 2011; Whitley, Rawana and Brownlee 2014), aboriginal
students learning discrepancies in Taiwan have been verified, including lower
educational levels, lower learning achievement, and lower ratio of universities
enrollment, etc. (Lin and Hwang 2009 ).
Accordingly, we assume that aboriginal parents-most from lower socioeconomic

status families cannot afford extra academic cram schooling expenditures.
Moreover, based on the consideration of profits, few, even none, private cram
schools are located in aboriginal areas. Likewise, some Taiwanese males choose
to marry females from developing Asian countries, such as Vietnam, Indonesia
and so forth. The primary reason for such cross-national marriage families is that

88
these low socioeconomic status Taiwanese males might marry females with the
same socioeconomic status by means of foreign marriage matchmaking. The
economic status of these cross-national marriage families might be relatively
low, similar to the aboriginal groups. In addition, these mothers from
cross-national marriage families cannot get jobs without Taiwanese resident
identification until several years after getting married, and they may have
difficulties in cultural adaption and oral communication. These difficulties
probably cause disadvantages to cross-national marriage children regarding
learning performance, including not being able to afford extra academic cram
schooling expenditures.
Because of these circumstances, the Ministry of Education in Taiwan promoted a

new project beginning in 2008. The project is named the Night Angel
Illumination Program (Ministry of Education 2009). Its purpose is to help
underachieving students to decrease the learning performance gap between
minority and non-minority students (Lin & Ou 2010). Two dimensions of this
project are After-School Care Program and Hand-in-Hand Program.
After-School Care Program offers after-school services, aiming to look after
elementary school and kindergarten children, help with homework assignments,
and guide these students learning. This program provides academic guidance in
classrooms after school or during vacations. Generally speaking, children and
their families have to pay a fee, but for physically and mentally handicapped
students, and students from low-income and aboriginal families, this program is
free of charge. Regarding the Hand-in-Hand Program, the students are from
aboriginal families, single-parent, grandparent, new-resident families, or those
from low socioeconomic families. Such programs are non-profit in orientation
and aim to enhance learning outcomes of disadvantaged students and decrease
the learning gap. Additionally, some academic guidance is sponsored by charity
groups, religious groups, or foundations established by private enterprises.
Financially disadvantaged university and college students or people with
teaching experience are hired to provide extra after-school academic guidance.
Such programs have low tuition fees, or are even free. The students participating
in these programs are primarily minority or underprivileged students. However,
there is no existing relevant research to verify the effectiveness of such academic
guidance services, compared to private academic cram schools.
Based on the above considerations, the cram schooling activities in Taiwan can
be grouped into three categories: (a) private profit-oriented cram schools,
regarded as Buxiban; (b) fee-based after-school programs provided by schools;
and (c) free after-school programs sponsored by non-profit organizations or
schools. According to relative viewpoints and empirical results (Brown-Jeffy
2009; Dang 2007; Kuan 2011; Lin et al. 2015; Liu 2012 ; Rowley and Wright 2011;
Whitley et al. 2014), this study assumes that minority groups cannot afford
private cram schooling expenditures due to their low socioeconomic status and
hence, their children possess lower academic performance. However, with few
relevant studies, some questions need to be addressed. First, few studies have
been conducted to compare the learning environment and learning performance
of cross-marriage children with aboriginal groups and non-minority groups.

89
Second, the effectiveness of free academic guidance services provided by schools

or non-profit organizations, and their effect on the academic achievement of
ethnic and cross-marriage children need more investigation.
Method
Participants
The analysis sample for this study includes children between age of 10 and 11
who had valid questionnaire data and their grade achievement assessment at the
time of the survey in 2013. In total there were 630 pieces of valid student and
parent data. Among the subjects, 49.4% were boys and 50.6% were girls. The
average length of education for the fathers and mothers was 12.07 years (SD =
2.23) and 12.07 years (SD = 2.00), respectively, which was the equivalent of a
senior high school graduation. The family economic status for the subjects were
given 1-3 points according to the level of wealth, and the mean was 2.66 (SD =
0.50).
Measures
Table 1 shows the design and scoring methods for each variable in this study.
All variables used in this study are grounded into two types: (1) student
individual characteristics, such as ethnicity, gender, and academic achievement,
(2) family background and characteristics, including parent educational level,
parent occupation, family economic conditions, family structure, sibling size,
and childrens participation in cram schooling.

90
Table 1 Variable measurement

Variable Description Metric
Ethnicity The measures were divided Non-minorities = 0 (as
into three categories: reference category)
Non-minorities, aboriginals
(minorities), and
cross-national marriage
(minorities).
Parents Convert the specified Elementary school = 6 years,

educational parents educational level Junior high school = 9 years,
level into education years. Senior high school = 12 years,
Junior college = 14 years,
College =16 years, Graduate
school = 18 years.
Parents What are fathers and professional and
occupation mothers occupations? The semi-professional = 0 (as
measures include four reference category)
categories: professional and
semi-professional, trading
and service, as well as
labor-related work and
low-tech jobs.
Family The economic status of the Item ranged from 1 to 4. 4 =

economic status families when the child Wealthy families, 1= poor
studied at the fifth grade. families
Family structure Family structure is measured Single-parent and other

with a set of binary household = 0 (as reference
indicators reflecting whether category)
the student lived in the
two-parent, single-parent or
other household.
Students
What is the students 1= boy, 0 = girl
gender gender?
Sibling size How many siblings does the Item ranged from 1 to 6. 6 =
child have? six or more siblings, 1= only
one child in family
Academic cram How many hours do you 0 = lowest hours; 20 = highest
schooling spend each week hours
Hours participating in cram
schooling?
Expenditures How much do you spend on NT$ 0 = lowest amount; NT$

(NT$/thousand) participating in cram 20,000 = highest amount

91
schooling each week?

After-school Whether have you taken part 1= yes 0 = no (as reference
programs in in after-school programs in category)
schools schools this semester?

tutorial classes in after-school tutorial category)
(private cram classes (private cram
schools) schools) this semester?

programs in after-school programs category)
outside outside schools ( cost-free)
Schools this semester ?
(cost-free)
Academic The achievement estimation The achievements of each

achievement values the average scores subject were based on
from the first semester different scoring criteria in
(including Chinese, math, each class. Therefore,
social, science, and English) choosing a class as a group,
in fifth grade students this study standardized
survey. students learning
achievements, then made
linear transformation in
scores with the mean value 85
and the standard deviation
10. The approximate
distribution range for the
achievements of each subject
is as follows:
Chinese: lowest score = 10.44
and highest score = 95.55;
Math: lowest score = 39.74
Social: lowest score = 30.89
Science: lowest score = 25.08
and highest score = 95.01.
Analysis Strategy
A series of hierarchical regression analysis was used with different groups of
predictors in the model, first with ethnicity (including non-minorities,
aboriginals, and cross-national marriages), then with ethnicity and controls for
family background (such as parent educational level, parent occupation, family
economic status, family structure, and sibling size) and childrens gender, and
finally with childrens participation in cram schooling measures added. In this

92
study, we hypothesize that, when parental SES, family background, and

childrens personal characteristics are included, the remained academic
achievement gap between non-minorities and minorities would be reduced
when cram schooling is held constant. Namely, the main purpose of this study is
to examine the potential mediating pathways through which academic cram
schooling mediates the ethnic gap of students academic achievement in Taiwan.
Table 3 shows the effects of the predictor variables on the students grade
achievement. Table 4 and table 5 are the analysis on mediating variables such as
cram schooling hours, expenditures and participating types (as dependent
variables). Since the dependent variables shown in Tables 3 and Table 4 were
continuous variables, the researchers practiced multiple regression analysis.
However, we conducted logistic regression analysis in Table 5 due to the
dependent variables were binary scale.
Results
Descriptive Analysis
Table 2 displays the results of the descriptive analyses among the minority
group students and non-minority group students in the respects of
socioeconomic status, personal attributes and family background, participation
in cram schooling, and their academic achievement. The results indicated that
the parents from minority group families had a lower educational level than
their counterparts, especially the cross-national couples (M = 10.82, 9.48; SD =
3.06, 2.97, respectively), which was equivalent to a junior high school level.
Secondly, among the non-minority groups, many fathers were engaged in
professional or semi-professional work (47.1%), whereas among the aboriginals
and cross-national marriage, most fathers were engaged in labor-related work or
low-tech jobs, occupying 66.3% and 64.3%, respectively. Among the
non-minority group, the percentages of maternal jobs in the professional and
semi-professional areas, trading and service sectors, and low-tech sectors were
almost the same, over 30% each. However, in aboriginals and cross-national
marriage, the mothers were mostly engaged in labor and low-tech jobs (61.1%
and 74.1%, respectively).
An analysis of family economic status (based on a Likert 4-point scale) showed

that most non-minority families had middle-class income level (M = 3.36, SD =
0.71), while the aboriginals families had a below-average living standard (M =
2.86, SD = 0.83). Also, among the non-minority groups and cross-national
marriages, over 10% of the families were single-parent families or were headed
by grandparents, whereas 27.3% of the aboriginal people had formed single
parent families or families headed by grandparents. In all of the ethnic groups,
girls tended to occupy a higher ratio. The average number of siblings was
highest in the aboriginals families (M = 3.10, SD = 1.36).

93
Table 2 Descriptive statistics for family SES, family background, participation in cram schooling, and childrens test scores by ethnicity
Non-minorities Minorities Minorities
Ethnicity (aboriginal) (cross-national marriages)
Variable N % Mean SD N % Mean SD N % Mean SD
Parental educational level
Father 502 13.82 2.78 100 11.23 2.95 28 10.82 3.06
Mother 500 13.50 2.52 100 10.69 2.78 27 9.48 2.97
Fathers occupation
Professional and semi-professional 230 47.1 15 15.8 6 21.4
Trading and service 107 21.9 17 17.9 4 14.3
Labor-related work and low-tech jobs 151 30.9 63 66.3 18 64.3
Mothers occupation
Professional and semi-professional 159 32.1 12 12.6 2 7.4
Trading and service 141 28.5 25 26.3 5 18.5
Labor-related work and low-tech jobs 195 39.4 58 61.1 20 74.1
Family economic status 504 3.36 0.71 100 2.86 0.83 28 3.14 0.71
Family structure
Two-parent household 421 84.7 72 72.7 24 88.9
Single-parent and other household 76 15.3 27 27.3 3 11.1
Students gender
Boy 244 48.9 42 42.0 13 46.4
Girl 255 51.1 58 58.0 15 53.6
Sibling size 492 2.08 0.72 96 3.01 1.36 27 2.15 0.72
Cram schooling
Hours 502 7.75 6.97 100 1.30 3.07 28 5.61 6.80
Expenditures (NT$/ thousand) 501 5.14 4.45 100 0.96 2.77 27 2.96 3.56
After-class programs in schools 499 21.4 99 64.6 28 39.3
After-class programs outside schools 499 69.9 99 15.2 28 53.6
(private cram schools)
After-class programs outside schools 499 4.0 99 31.3 28 3.6
(cost-free )
Academic achievement (average) 490 85.86 7.47 99 79.88 8.77 28 84.59 7.78

94
In the aspect of cram schooling, the data showed that students from aboriginal
families experienced the lowest number of hours and spent the least amount of
money on cram schooling, followed by students from cross-national marriage,
while students from non-minority families had the highest number of hours and
largest expenditures spent on cram schooling. Furthermore, of the three main
types of after-class schooling, nearly 70% of the students from non-minority
families chose private cram schools that demand higher tuitions, whereas 64.6%
of the aboriginal students chose to take extra hours in tutorial classes organized
by schools. Meanwhile, 31.3% of the aboriginal students enrolled in tutorial
classes organized by non-profit organizations, which were free of charge.
Students from cross-national marriages, like their counterparts from
non-minority families, tended to choose private cram schools (53.6%) and
tutorial classes organized by schools (39.3%). Lastly, in the aspect of student
academic performance, non-minority students scored the highest (M = 85.86, SD
= 7.47), closely followed by students from cross-national marriage (M = 84.59,
SD = 7.78). The aboriginals students, however, scored lower than their peers (M
= 79.88, SD = 8.77).
Multivariate Analysis
This study made ethnic group a predictive variable, as shown in Table 3, Model
1, and found that among the minority groups, aboriginal students scored far
lower than non-minority students (B = -5.76, SE = 0.85, p < .05), yet there was no
discernible difference between cross-national marriage students and their
non-minority counterparts. In Model 2, students personal attributes and family
backgrounds were included as control variables. Compared with the results
showed in Model 1, the difference between aboriginal students and their
non-minority peers became smaller (B = -1.98, SE = 1.57, p <.05). It was also
discovered that students whose mothers worked as laborers or with low-tech
jobs had poorer academic achievement. In Model 2, the explanatory power (R
square) reached 0.20. In Model 3, hours of cram schooling and the quadratic
term of cram schooling hours were included as mediators to observe the impact
of cram schooling hours on academic achievement. The regression coefficient
showed that longer hours of cram schooling resulted in better academic
performance (B = 0.40, SE = 0.14, p <.05). However, the square of the hours spent
on after-school cram schooling did not have a remarkable influence on academic
achievement. By further taking hours in cram schooling into Model 3 for
analysis, the disadvantage of aboriginal students grew smaller (B value dropped
from -1.98 to -0.72), and the influence plummeted to 63.63%. This indicated that
aboriginal students poorer performance might come from shorter hours in cram
schooling. The explanatory power (R square) of Model 3 reached 0.22. Model 3
indicated that the cross-national marriage students, with hours in cram
schooling excluded, had better academic achievement than the other two
groups. When dealing with factors separately, such as personal attributes,
socioeconomic status and hours in cram schooling taken as predict variables, the
cross-national marriage students performed better than the non-minority
students. This is worth of further discussion and scrutiny in future research.
Model 4 and Model 5 displayed similar results.

95
Table 3 Estimates on the achievement score for 5th grade students

Model 1 Model 2 Model 3 Model 4 Model5
B SE B SE B SE B SE B SE
Ethnicity
Non-minorities(reference categories)
Minorities (aboriginal) -5.76 ** (0.85) -1.98 * (0.93) -0.72 (0.97) -.069 (0.96) -1.02 (1.02)
Minorities(cross-national marriage) -1.05 (1.51) 2.97 (1.57) 3.32 * (1.55) 3.47 * (1.58) 3.59 * (1.55)
Father educational level 0.53 ** (0.16) 0.50 ** (0.15) 0.46 ** (0.15) 0.47 ** (0.16)
Mother educational level 0.16 (0.16) 0.16 (0.16) 0.10 (0.16) 0.21 (0.16)
Fathers occupation
Professional and
semi-professional
(reference categories)
Trading and service 0.15 (0.87) 0.26 (0.86) 0.25 (0.85) 0.14 (0.86)
Labor-related work and low-tech jobs -1.10 (0.79) -0.95 (0.79) -0.95 (0.78) -0.94 (0.79)
Mothers occupation
Professional and
semi-professional
Trading and service -0.74 (0.82) -0.84 (0.82) -0.74 (0.81) -0.40 (0.82)
Labor-related work and low-tech jobs -1.78 * (0.77) -1.50 (0.76) -1.51 * (0.76) -1.71 * (0.77)
Family economic status 0.49 (0.46) 0.37 (0.46) 0.22 (0.46) 0.29 (0.47)
Family structure
Two-parent household 2.42 ** (0.86) 2.25 ** (0.86) 2.17 * (0.86) 2.06 * (0.86)
Single-parent and other
household (reference categories)
Boy (reference categories: Girl) -0.48 (0.60) -0.47 (0.60) -0.40 (0.59) -0.41 (0.60)
Sibling size -0.61 (0.37) -0.46 (0.36) -0.38 (0.36) -0.33 (0.37)
Cram schooling
Hours 0.40 ** (0.14)
Hours Hours -0.01 (0.01)
Expenditures 0.71 ** (0.18)
Expenditures Expenditures -0.03 * (0.11)
After-school tutorial classes -2.22 ** (0.76)
After-school programs outside 1.23 # (0.72)
school(private cram schools)
After-school programs outside 0.74 (1.21)
school(cost-free)
N 617 582 579 578 575
R square 0.08 0.20 0.22 0.23 0.22
# p < .10 * p < .05 ** p < .01

96
This study included expenditures in cram schooling and the quadratic term of
expenditures in Model 4 to explore the relationship between academic
achievement and expenditures of cram schooling. The result indicated that
higher expenditures of cram schooling resulted in better academic performance
(B = 0.71, SE = 0.18, p < .05). In the meantime, the impact made by the quadratic
term of expenditures on academic achievement showed an apparent negative
value. This further indicated that the positive impact of expenditures on
academic achievement showed diminishing marginal utility. In other words,
spending too much tuition on cram schooling was negatively correlated with
academic achievement. With expenditures as a factor, the disadvantage of
aboriginal students became less obvious (the B value of Model 4 dropped from
-1.98 to -0.69) by 65.15%. This finding showed that the poorer performance of
aboriginal students might be caused by their families economic condition. The
explanatory power (R square) of Model 4 reached 0.23, an increase of 15% over
Model 2.
This study further analyzed how different cram schooling types affected student
academic performance, as shown in Model 5. The tutorial classes organized by
the schools had a negative effect on academic achievement. Although the
government invested sufficient funds into the program that allowed students
from lower-income families not to pay tuition, the effect was less than
satisfactory. On the other hand, students who chose private cram schools often
scored higher in their studies. The positive impact on such students reached a
remarkable result of nearly .05 (B = 1.23, SE = 0.72, p = .06). Students who
participated in tutorial classes set up by non-profit organizations did not show
improvement in academic achievement. The explanatory power (R square) of
Model 5 reached 0.22, an increase of 10% over that in Model 2.
Mediators
Table 4 showed that the aboriginal students spent less tuition and had fewer
hours spent on cram schooling than their non-minority counterparts (B = -4.94,
SE = 0.83, B = -2.01, SE = 0.51). Aboriginal families provided apparently fewer
resources for learning, which affected their childrens academic achievement.
Meanwhile, the cross-national marriage couples provided a similar level of
resources for their childrens after-class education compared with non-minority
families. Table 5 showed that the aboriginal students had a higher percentage of
choosing after-school tutorial classes (B = 1.44, odds ratio = 4.23, p < .05) or
tutorial classes provided by non-profit organizations (B = 2.27, odds ratio = 9.67,
p < .05) over private cram schools (B = -2.21, odds ratio = 0.13, p < .05). By
compiling the analytical results of Tables 3 to 5, it was concluded that the less
satisfactory academic achievement of the aboriginal students was caused by
choosing to attend after-school tutorial classes organized by the schools rather
than attending private cram schools. However, their choice of free tutorial
classes offered by non-profit organizations seemed not to have affected their
academic achievement (see Table 3).

97

Hours Expenditures
B SE B SE
Ethnicity
Non-minorities (reference
categories)
Minorities (aboriginal) -4.94 ** (0.83) -2.01 ** (0.51)
Minorities (cross-national -1.48 (1.40) -0.86 (0.88)
marriage)
Father educational level -0.01 (0.14) 0.21 * (0.08)
Mother educational level -0.04 (0.14) 0.18 * (0.09)
Fathers occupation
Professional and
semi-professional
Trading and service -0.89 (0.76) -0.33 (0.47)
Labor-related work and low-tech -1.27 (0.70) -0.47 (0.43)
jobs
Mothers occupation
Professional and
semi-professional
Trading and service 0.56 (0.73) 0.36 (0.45)
Labor-related work and low-tech -1.49 * (0.68) -0.43 (0.42)
jobs
Family economic status 0.50 (0.41) 0.62 * (0.25)
Family structure
Two-parent household -0.03 (0.75) 0.47 (0.46)
Single-parent and other
household
Boy (reference categories: Girl) -0.18 (0.53) -0.25 (0.33)
Sibling size -0.78 * (0.32) -0.50 * (0.20)
N 595 594
R square 0.16 0.25
# p < .10 * p < .05 ** p < .01

98

After-school After-class After-class
tutorial classes programs programs
outside school outside school
(private cram (cost-free)
schools)
B Odds B Odds B Odds
Ratio Ratio Ratio
Ethnicity
Non-minorities
(Reference categories)
Minorities (aboriginal) 1.44 ** (4.23) -2.21 ** (0.13) 2.27 ** (9.67)
0.88 (2.41) -0.48 (0.62) 0.22 (1.25)
Minorities(Cross-natio
nal marriage)
Father educational level -0.13 * (0.88) 0.02 (1.02) -0.13 (0.88)
Mother educational level 0.10 # (1.11) 0.00 (1.00) 0.15 (1.16)
Fathers occupation
Professional and
Semi-professional
Trading and service -0.21 (0.81) 0.07 (1.07) -0.34 (0.71)
Worker and other 0.29 (1.34) -0.15 (0.86) -0.16 (0.85)
Mothers occupation
Professional and
Semi-professional
Trading and service 0.53 # (1.70) -0.14 (0.87) 0.42 (1.52)
Worker and other 0.18 (1.20) -0.48 # (0.62) 0.35 (1.42)
Family finance situation -0.35 * (0.70) 0.54 ** (1.71) -0.48 * (0.62)
Family structure
Two-parent household -0.05 (0.95) 0.18 (1.19) -0.11 (0.89)
Single-parent and
other household
Boy (Reference 0.32 (1.37) -0.04 (0.96) 0.25 (1.29)
categories: Girl)
Sibling size 0.32 ** (1.37) -0.29 * (0.75) 0.23 (1.25)
N 591 591 591
Cox & Snell R Square 0.16 0.22 0.11
Nagelkerke R Square 0.23 0.30 0.27
# p < .10 * p < .05 ** p < .01

99
Conclusion and Discussion

Over the years, many researchers have made an effort to explore the
relationships between after school learning activities (such as private tutoring,
cram schooling and so forth) and students academic achievement (Bray et al.
2014; Kim and Park 2010; Shih and Yi 2014). Students in East Asia countries have
long stressed cram schooling, and they consider it as an effective way to enhance
their ability to pass national entry examinations and achieve the schooling
successfully (Fung 2003; Kuan 2011; Liu 2012). However, the relationships
between diverse cram schooling (including time, expenditures, and types of
participation) and students academic achievement are still not clear and
insufficient. Therefore, this topic has a highly referential value for educational
policies, instructional affairs, and the education system, where credentials and
educational background still have an immense effect on an individuals career in
society.
This study discovered that aboriginal students had less satisfactory academic
achievement than non-minority students; yet the learning performance of
children from cross-national marriage families was similar to that of the
non-minority students. This indicated that the main division of achievement
only existed between aboriginal students and non-minority students. Secondly,
this study discovered that the number of hours spent on cram schooling was
positively correlated with academic performance. On the other hand, although
expenditures showed diminishing marginal utility, it still had a positive link
with academic achievement. This indicated that, for Taiwanese students, cram
schooling is a crucial factor in academic achievement, thereby supporting the
previous research results (Chen and Hwang 2011; Dang 2007; Kuan 2011; Kim
and Park 2010; Liu 2012). Moreover, the results showed that the less satisfactory
academic achievement of aboriginal students was due to the scarcity of
additional cram schooling provided by their families after school. This result
reflected that gaps in learning outcomes between ethnic or racial groups are
associated with the abundance of family learning resources and poor
educational environment (Brown-Jeffy 2009; Byun and Park 2012; Lin and
Hwang 2009; Rowley and Wright 2011).
There has been relatively little research in the link between after-class tutorial
classes and academic performance, especially in regards to what type of cram
schooling and organizations can achieve better academic performance, what
kind of cram schools ethnic groups will choose, and what kind of division exists
in student achievement. This study found that tutorial classes organized by
schools had a negative impact on students, and that the students attending these
classes were mostly from aboriginal groups. This contributed to the lagging
behind of aboriginal students. The authorities concerned have been putting great
effort and funding into after-school tutorial classes that held in school, as it is felt
that they are a great help to minority students in their learning performance.
However, this study found that, the governments well-intended policy is not
providing the desired effect (possibly due to a poorly-organized teaching plan
and poorly trained teaching staff). This poses a concern for how the Taiwan

100
government allocates educational resources, and it is advised to take immediate

action and revise its policies.
Meanwhile, this study found that private cram schools, which generally demand
higher tuition fees than schools and non-profit tutorial classes, can be a great
help to students academic achievement. Other than the remote areas, cram
schools can be found in almost every city and town in Taiwan. The business of a
cram school depends on how its students perform academically. Thus, many
parents are eager to pay higher tuition fees as long as their children make
improvements in their academic studies (Lin and Hwang 2009; Lin et al. 2015).
To attract renowned teachers, cram schools have to be set up in urban areas.
Aboriginal students from remote areas find it difficult to commute to these cram
schools, and their families face the challenge of higher extra tuition fees. The
only choice left to them is to choose after-school tutorial classes, yet such tutorial
classes are of little help to diminish the learning gap.
In recent years, a large number of enterprises and non-profit organizations in

Taiwan have tried to bridge the gap with less-privileged families by providing
considerable funds and manpower to hire college students to teach aboriginal
students in remote areas or through online classes. The results of these actions
have also proved less satisfactory than expected. As Table 5 shows, aboriginal
students have a higher percentage of choosing tutorial programs that are free of
charge. Therefore, it is important to enhance the operations of such free tutorial
classes so that aboriginal students can have the opportunity to compete with
their non-minority peers.
The above findings also showed that children from cross-national marriage
families have similar academic performance compared with other non-minority
peers, and that the tuition and hours they spend on cram schooling are similar to
their counterparts. They may be less likely to attend off-campus cram schools,
and they tend to participate in on-campus after-school classes. Many mothers in
cross-national marriage families come from mainland China and Southeast Asia,
and they often face the problems of obtaining an ID card and finding a job, not
to mention language difficulties and culture conflict. The above all pose issues
for their children. This study used fifth graders as the research samples, and
found that their learning was inhibited considerably.
Although our analytical approach provided new insights regarding the

association between cram schooling participation and students academic
achievement gap of ethnic and cross-national marriage, there are several
limitations that need to be considered and possibly addressed in the future
research. Firstly, multi indicators of cram schooling were selected as a mediator
in this study, and it did not examine other mediators that could affect a students
academic achievement or the indirect impact of cram schooling on different
academic groups. This study also did not discuss the relationship between
student age and academic performance in different educational stages.

101
Secondly, other questions left to discuss include: What are the effects of cram
schooling on students learning adaptation in the long term? Will it affect a
students career or income (Lin and Hwang 2009)? Does any difference exist
about the relative findings between Eastern and Western societies? Additionally,
the long hours spent on a cram schooling cause higher pressure and less time for
leisure activities. This might lead to a lower sense of happiness or
maladjustment at school (Chen & Lu 2009; Csikszentmihalyi and Hunter 2003).
Hence it is important to pay attention to the negative effects brought forth by
cram schooling.
Lastly, a striking discovery of this study was that, after excluding socioeconomic
factors and cram schooling, students from cross-national marriage families often
performed better than non-minority students. This study, however, did not
investigate a deeper understanding of this phenomenon. The above limitations
are all topics that are worth further exploration in future research to create better
strategies and educational policies.
Acknowledgement
This research was sponsored in part by the Ministry of Science and Technology,
Taiwan, under Grant MOST 103-2410-H-259-031-SS2
References
Brown-Jeffy, S. (2009). School effects: Examining the race gap in mathematics

achievement. Journal of Africa- American Study, 13, 388-405.
Bray, M. (2013). Shadow education: Comparative perspectives on the expansion and
implications of private supplementary. Procedia-Social and Behavioral Sciences, 77,
412-420.
Bray, M., Zhan, S., Lykins, C., Wang, D., & Kwo, O. (2014). Differentiated demand for
private supplementary tutoring: Patterns and implications in Hong Kong secondary
education. Economics of Education Review, 38, 24-37.
Byun, S. Y., & Park, H. (2012). The academic success of East Asian American youth: the
role of shadow education. Sociology of Education, 85, 40-60.
Chen, C. W., & Hwang, Y. J. (2011). A re-exploration of stratification and efficacy in cram
schooling: An extension of the Wisconsin model. Bulletin of Educational Research,
57(1), 101-135 (In Chinese)
Chen, S. Y., & Lu, L. (2009). After-school time use in Taiwan: Effects on educational
achievement and well-being. Adolescence, 44, 891-909.
Csikszentmihalyi, M., & Hunter, J. (2003). Happiness in everyday life: The uses of
experience sampling. Journal of Happiness Studies, 4, 185-199.
Dang, H. A. (2007). The determinants and impact of private tutoring classes in Vietnam.
Economics of Education Review, 26, 683-698.
Fung, A. (2003). Cram schooling in Hong Kong: The privatization of public education.
Asian Anthropology, 2, 179-195.
Jung, J. H., & Lee, K. H. (2010). The determinants of private tutoring participation and
attendant expenditures in Korea. Asian Pacific Education Review, 11, 159-168.
Kenayathulla, H. B. (2013). Household expenditures on private tutoring: Emerging
evidence from Malaysia. Asia Pacific Education Review, 14, 629-644.
Kim, J. H., & Park, D. (2010). The determinants of demand for private tutoring in South
Korea. Asia Pacific Education Review, 11, 411-421.
Kuan, P. Y. (2011). Effects of cram schooling on mathematics performance: Evidence

102
from junior high students in Taiwan. Comparative Education Review, 55, 342-369.
Lee, S., & Shouse, R. C. (2011). The impact of prestige orientation on shadow education
in South Korea. Sociology of Education, 84, 212-224.
Lin, C. C., & Ou, I. C.(2010). The implement condition of Night Angel Illumination
Program. School Administrators Research Association, 67, 122-149. (in Chinese)
Lin, C. Y., Hsieh, Y. H., & Chen, C. H. (2015). Use of latent growth curve modeling for
assessing the effects of summer and after-school learning on adolescent students
achievement gap. Asia Pacific Education Review, 16, 49-61.
Lin, H. M., & Hwang, Y. J. (2009). The study on relationship among the Aborigines, Hans,
cram schooling and academic achievement. The example of the eighth graders in
Taitung. Contemporary Educational Research Quarterly, 17(3), 41-81 (in Chinese).
Liu, J. (2012). Does cram schooling matter? Who goes to cram schools? Evidence from
Taiwan. International Journal of Educational Development, 32, 46-52.
Ministry of Education (2009). Night Angel Illumination Program. Retrieved August 2, 2014,
from http://english.moe.gov.tw/ct.asp?xItem=14748&ctNode=11435&mp=1.
Rhy, D., & Kang, C. (2013). Do private tutoring expenditures raise academic performance?
Evidence from middle school students in South Korea. Asian Economic Journal, 27,
59-83.
Rowley, R., & Wright, D. W. (2011). No White child left behind: The academic
achievement gap between black and white students. The Journal of Negro Education,
80, 93-107.
Shih, Y. P., & Yi, C. C. (2014). Cultivating the difference: social class, parental values,
cultural capital and childrens after-school activities in Taiwan. Journal of
Comparative Family Studies, 45, 55-75.
Stevenson, D. L., & Baker, D. P. (1992). Shadow education and allocation in formal
schooling: transition to university in Japan. American Journal of Sociology, 97,
1639-1657.
Whitley, J., Rawana, E., & Brownlee, K. (2014). A comparison of aboriginal and
non-aboriginal students on the inter-related dimensions of self-concept, strengths
and achievement. Brock Education, 23(2), 24-46.
Zhang, Y. (2013). Does private tutoring improve students national college entrance
exam performance? A case study from Jinan, China. Economics of Education Review,
32, 1-28.

103

Vol. 16, No. 1, pp. 103-119, January 2017
Teachers Self-Efficacy at Maintaining Order and

Discipline in Technology-Rich Classrooms with
Relation to Strain Factors
Eyvind Elstad
University of Oslo, Norway
Knut-Andreas Christophersen
University of Oslo, Norway
Abstract. Teachers operational control is challenged by pupils access to

computers with internet in the classroom. The teachers control of
operations will be diminished in a technology-rich classroom.
Professional growth is conditioned by teachers experience of mastery in
their work. Therefore, it is interesting to study strain factors that are
associated with teacher efficacy at maintaining order and discipline in
technology-rich classrooms. This study splices two theoretical lenses:
theory on teacher efficacy and strain factors and theory of teachers
learning orientation. The purpose of this article is to explore the factors
that are associated with teacher efficacy at maintaining order and
discipline in technology-rich classrooms. The data set comes from a
survey of 156 teachers in Norway. The analysis shows that the
perception of conflict is negatively associated with teacher efficacy.
Stress and perception of shortcomings in the ability of the teacher to
influence the pupils is associated in a slightly weaker manner associated
with teacher efficacy for classroom management. There is a strong,
positive relationship between stress and conflict, and there are positive
associations between teachers sense of shortcoming and both stress and
conflict. Teachers emphasis on practical knowledge rather than expert
recommendations is positively associated with shortcomings, stress and
conflict. Implications for practice are discussed.
Keywords: self-efficacy; classroom management; strain; technology-rich

classrooms; ICT; teachers work.
Introduction
The Norwegian educational authorities have defined using digital tools as a
core skill that is to be stimulated by schools (Ministry of Education and
Research, 2014). Computers are used in school as writing tools, for the gathering
of information and for communication, and students are expected to make

104
academic use of the Internet. Norwegian sixth-form colleges have thereby seen
an equipment revolution in terms of accessibility to computers in the classroom
(Elstad, 2016a). Each pupil has his own computer with internet access and some
school districts expect these machines to be used often.
Off-task activity on the part of pupils during lesson time is a recurrent problem
(Elstad, 2006; Blikstad-Balas, 2012) and represents a frequent challenge to
teachers, whose work is often in the form of the teacher going through new
material on the blackboard or guiding pupils in their individual or group-based
studies. The physical design of the classroom includes the classroom artefacts. In
the traditional classroom, the arrangement of the desks is physically designed to
allow monitoring of pupils academic work. Desks are generally laid out in rows
in order that each pupil can see the blackboard. This physical layout creates a
challenge in terms of the teacher's ability to see whether the pupils are using
their computers for the purpose of academic work or off-task behaviour. In the
technology-rich classroom, classroom control of this kind is harder to maintain
because it is more difficult to monitor the pupils behaviour. Classroom
management is the actions teachers take to create an environment that supports
and facilitates both academic and social-emotional learning (Evertson &
Weinstein, 2011: 4). It is interesting to explore the potential managerial
challenges that teachers confront in technology-rich classrooms. Professional
growth for teachers is an important ambition in Norwegian educational policy
(White Paper no. 11, 2008) and teacher efficacy in terms of classroom
management is important if the teacher is to increase skills and professional
growth in their work. A teacher's first experience of teaching can be a challenge:
situations are often complex and the teacher needs to take account of several,
often unforeseen, things happening at once. This can easily create stress and
working-memory overload among inexperienced teachers when decisions need
to be taken (Leinhardt & Greeno, 1986). Through experience, teachers learn to
tackle complex decision-taking situations in a better way: the skill becomes
procedural (Anderson, 1982). In this way, the cognitive overload experienced by
beginners becomes reduced. After the initial phase in which the goal is survival,
teachers attempt to consolidate their pedagogic repertoire and experiment with
ways of carrying out their roles in ways give them mastery over new challenges
(Berliner, 1986; Huberman, 1989; Chi et al., 2014). Several studies document that
professional growth among teachers allowing for certain nuances is related
to improved pupil performance (Rivkin et al., 2005; Rockoff, 2004).
Teacher efficacy is conceptualized as a teachers judgment of ones capabilities to

plan and enact specific tasks (Bandura, 1994). The study of teacher efficacy in
various teaching-related aspects shows a surprising curvilinear pattern:
expectations of mastery increase with experience but can sink again after many
years of teaching experience (Klassen & Chui, 2010). The same pattern is
observed in studies of the teacher's contribution to pupil learning performance:
performance has a positive association with the number of years of teaching
experience; (Rockoff, 2004) but learning improvement reaches a plateau after a
few years' experience or even sinks slightly after 5-6 years of practice (Rivkin et
al., 2005).

105
A premise of our study is that a teacher's experience of mastery of the job is a

condition for professional growth (Tschannen-Moran, Hoy & Woolfolk, 2007).
Too low a feeling of mastery can be unfavourable for a future professional career.
It is therefore interesting to study which factors we presume will have a
connection with teacher efficacy. The purpose of this article is to investigate
strain factors statistically associated with teacher efficacy in maintaining order
and discipline in a situation in which the pupils each have a computer and
internet access. The purpose is also to investigate the relationships between all
the factors that form part of the theoretical framework. We will therefore
investigate how a disparity in teacher efficacy for the maintenance of order and
discipline corresponds with a disparity in how these teachers perceive (i)
opportunities to influence pupils in the 15-19 age range (in the Norwegian
school system, these pupils have a say in a number of significant matters), (ii)
teachers self-perception of experience as a starting point for their own
professional development and their experience of (iii) stress and (iv) conflict
when they are to lead learning work in their own classes.
Theoretical assumptions
Two theoretical lenses are brought together in this study: (1) Bandura's self-
efficacy theory (1994) and (2) the theory of teacher orientation to learning
(Opfer, Pedder & Lavicza, 2011: 5). A self-efficacy belief is an assessment of a
persons capabilities to attain a desired level of performance in a given
endeavour. Bandura assumed that belief in one's abilities was a powerful
driving mechanism influencing motivation to act, the effort put forth in the
endeavour, and the persistence of coping mechanisms in the face of setbacks.
The other theory concerns teacher orientation to learning which is integrated set
of attitudes, beliefs and practices as well as the alignment of oneself and ones
ideas to circumstances and context. These learning orientations are context
dependent. A part of teachers orientations to learning remain unchanged
overtime, while the context, the phase of teacher experience and the pupils a
teacher has influence the orientation to what, how and why they learn as
professionals.
Zee and Koomen (2016) present a recent review of teacher efficacy studies. They
integrated four decades of teacher efficacy research to explore the consequences
of teacher efficacy for the quality of classroom processes, students academic
adjustment, and teachers psychological well-being. Their results suggest that
teacher efficacy shows positive links with students academic adjustment,
patterns of teacher behaviour and practices related to classroom quality, and
factors underlying teachers psychological well-being, including personal
accomplishment, job satisfaction, and commitment. Negative associations were
found between teacher efficacy and burnout factors. However, they do not
explicitly focus on teacher efficacy at maintaining order and discipline in
technology-rich classrooms.
A new kind of managerial challenge has appeared with the advent of free
Internet access in the classroom: motivational conflicts between immediate
rewards of electronic chatting, surfing and games and the long-term rewards of
academic achievement (Elstad, 2008). Van Acker et al. (2013) explored the role of

106
teacher efficacy, subjective norm and attitude in technology-rich situations. They

found that teachers ICT skill was the strongest predictor of teacher efficacy.
Subjective norm played only a limited role in the intention to use the educational
use of digital learning materials. Basing on the outcome of this study, the
authors regard persuasive communication focusing on positive outcomes and
skills based training seem appropriate interventions to promote a positive
attitude towards the educational use of digital learning materials and improve
teacher efficacy in using digital learning materials. However, these scholars do
not focus on teacher efficacy at maintaining order and discipline. Quite few
researchers focus explicitly on teacher efficacy at maintaining order and
discipline (Evertson & Weinstein, 2011:3). This is especially true for classroom
management and technology: Bolick and Cooper (2011) and Bolick and Bartels
(2015) address how little research exists on how 1:1 computers affects classroom
management. In this article we have attempted to fill this gap in the literature.
We argue that educators and policymakers need to understand better teachers

challenges of classroom management in technology-rich classrooms. In this
article we assume that factors such as teachers' general perception of the
limitations of what can be achieved through teaching, as well as stress and
conflict in connection with the high availability of technology in the classroom
can determine their self-efficacy in highly technological learning environments.
Self-efficacy is primarily influenced by experiences of mastery1 (Bandura, 1994).
Skaalvik and Skaalvik (2007) identify strain factors statistically associated with
self-efficacy among primary and lower secondary teachers. Burnout is among
the factors they find to be strongly associated with teacher efficacy, while
teachers' general perceptions of the limitations on what can be achieved through
teaching are indirectly associated with teacher efficacy. In other words, the study
provides confirmation that issues perceived as stressful help account for teacher
efficacy. Teacher stress originates in a conflict between pupil interests and the
teacher's need for control over pupils' academic activities in the classroom. We
connect experiences both of stress and conflict to teacher efficacy in classroom
management (hypothesis no. 1 and no. 2).
We anticipate a range of opinions regarding how teachers perceive, for instance,

conflict with pupils and personal stress when enforcing the rules for acceptable
use of technology. We will therefore also investigate the range of teacher efficacy
in terms of classroom management. In addition, our assumptions suggest that a
perception of inadequacy in this respect is negatively associated with teacher
efficacy in maintaining order and discipline (hypothesis no. 3).
There is relatively limited theoretical knowledge about how teachers should

tackle the non-academic use of PCs. However, it is interesting to contrast the
significance of practical skills for teachers' perception of expert knowledge and
to investigate connections between this and teacher efficacy in relation to
maintaining order and discipline (hypothesis 4).
1
According to Bandura (1997), expectation of mastery is based on five types of
information source, one of which is personal experience.

107
The high degree of access to technology in Norwegian schools provides

opportunities both for the use of adequate technology for the purposes of
learning (for instance investigating what takes place within an electrical circuit,
Zacharia & de Jong, 2014), but at the same time, various kinds of challenges such
as the non-academic use of technology in the classroom (Blikstad-Balas, 2012).
We assume that some people will experience this as a source of stress that can
influence teacher efficacy. Situations may also arise in which some pupils cast
doubt on the rights of the teacher to limit, for instance, off-task pupil behaviour.
In other words, teachers may experience both stress and conflict-laden situations
when they are dealing with rules, but also in situations in which technology is
used to promote learning: it is a stated political goal that teachers must be able to
teach using digital tools and integrate digital skills as one of the five so-called
core skills (Ministry of Education and Research, 2014). For this reason we
assume that both skills and conflicts have a positive relation with an experience
of shortcoming (named con) in terms of influence over pupils (investigative
assumption nos. 1 and 2). Most sixth-form pupils are in the 15-19 age range, in
which pupils as they grow older will claim rights to self-determination and
personal positioning (Koepke & Denissen, 2012). The rationale behind the
teacher's work is to exercise a positive influence on the pupil's learning process
(the teacher has in this respect a clearly-defined responsibility for pupil learning
outcomes; White Paper no. 11, 2008), but pupil engagement in learning processes
will nevertheless depend on their motivation and self-discipline when working
in technology-rich learning environments. The result depends on input from
several parties. In other words, the teacher's task of influencing pupils is
dependent on their self-discipline.
Teachers are expected to be concerned with their own professional development.

Since the 1980s, research has focussed on the significance of teachers' epistemic
beliefs upon various aspects of teaching work (e.g. Fang, 1996; Chinn, Buckland,
& Samarapungavan, 2011). Several studies have provided empirical support for
suggesting a connection between teachers' personal epistemology and their
actual actions in the classroom (e.g. Brown & Rose, 1995), but this research field
is still immature (e.g. Greene & Seung, 2014). A theory developed by Opfer,
Pedder and Lavicza (2011) indicates how teachers' approach to learning
(including learning of practices) influences their skills development (learning
change). This theory joins a succession of theoretical contributions emphasising
practical skills alongside theoretical knowledge as fundamental for personal
teaching development. All professional practitioners employ theory in the
exercise of their profession. Theoretical development may be experience-based
(and thus weak), but may also be based on, for instance, research (and thus
strong). The emphasis can be contextual. When teachers need to master new
classroom challenges they can look to both experience and theory as sources
from which to learn how to tackle the situation. Norway has a deliberate policy
of stimulating teacher networks and developing practical skills. It is therefore
interesting to investigate how teachers use their own experiences and those of
other teachers to develop their own classroom management in technology-rich
classrooms, in other words, to develop practical skills.

108
Research about teachers' sense of their shortcomings and their development of

practical skills has so far as we can see not come very far. If the development
of practical skills is effective for professional development, it is reasonable to
expect a negative association between practical skills and perceptions of
personal shortcoming. If practical skills do not contribute to effective
professional development in challenge solving, we can expect a positive
association between shortcomings in terms of influence over pupils and practical
skills (investigative assumption no. 3). We can make corresponding arguments
in terms of the connection between practical skills and stress (investigative
assumption no. 4).
IA4
Figure 1 Theoretical model of hypotheses (abbreviated H) and investigative

assumptions (abbreviated IA).
Method
In order to investigate our hypotheses and investigative assumptions, we carried
out a survey among 156 teachers at 3 Norwegian sixth-form colleges. The data
collection took place in the form of a paper-based questionnaire filled in by the
teachers in connection with a planning day. The advantage of this method is that
all the teachers were required to take part in the planning day. No teachers
declined to take part in the survey. This means that apart from those who were
off sick on the day in question, the selection we have analysed can be regarded
as complete. In other words the selection has no self-selection issues.2 The three
schools in question, however, cannot be regarded as representative of
Norwegian sixth-form colleges and their teachers. All three, which offer the
curriculum directed towards preparation for higher education, are regarded as
2
Seven sixth-form colleges were originally included in the selection but the participation
percent was too low at four of these schools. In order to avoid self-selection we have removed
these four schools from the material to be analysed.

109
fairly similar. Pupils' average grades on entry were 4.25 [on a scale of 1-6] and
the value-added measures of the three schools are relatively similar (averaging -
0.35) (Falch et al., 2016). In other words, the schools can be regarded as fairly
average in terms of pupil attainment on entry and pupil progress during their
time at the school (pupil entry statistics and school contribution indicators based
on year group 2008-2009). This aspect of the selection as we will describe later
is of significance in relation to the conclusions that can be drawn.
The purpose of the investigation was described at the beginning of the session in
which the questionnaires were distributed in plenum and the teachers were
given sufficient time to consider the alternatives while filling out the
questionnaire. We thus believe that the answers are relatively well considered.
In all three schools, the pupils each have a PC with internet access available and
it is up to the individual teachers to tackle the challenges involved in terms of
classroom management. To remind the teachers about this challenge we
introduced the questionnaire with the following text:
The government's strategy document The teacher promotion states that

'Teachers must be able to teach using digital tools and integrate digital
skills as one of the five core skills. Consider that someone has decided
that pupils are each to have their own computer and access to the
internet during the lessons that you are teaching.
After this text, the teachers were asked to indicate on a seven-point scale (with
named extremes) which alternative corresponded best to their own views. The
teachers were asked to indicate values for the following questions or statements:
Teacher efficacy classroom management (secm)
How certain are you that in such a situation you would be able to:
maintain discipline in classes of over 25 pupils.

persuade even the most games-addicted pupils to concentrate on
academic tasks.
persuade pupils who often switch between different social media to
follow classroom rules.
These questions on teacher efficacy are inspired by Skaalvik and Skaalvik's (2007)
Maintain discipline construct, but here adapted to the situation of technology-
rich learning environments. The extremes of the scales regarding teacher efficacy
were: 1= Not sure at all, 7= Absolutely certain. After these questions regarding
teacher efficacy, teachers were asked to consider the following statements (the
extremes of the scales were: 1=completely disagree, 7=completely agree):
Shortcomings in terms of influence over pupils (con)3
How much pupils learn at school through using ICT is primarily

determined by their self-discipline.
3
Items are inspired by, but not identical to, those that Skaalvik & Skaalvik (2007) term
their External Control Scale.

110
It is difficult for a teacher to improve pupil performance by using ICT

unless the pupil can be bothered working on the tasks.
Stress in technology-rich classrooms (str1)
Allowing pupils free access to the internet during lessons creates an

extremely stressful situation for me.
When pupils have the freedom to do as they wish on the internet, I lose
control.
Where technology creates conflict situations (str2)4
Many conflicts with pupils arise when pupils are allowed to use the
internet during lessons.
Some pupils are unable to maintain focus on academic tasks when they
use the internet during lessons.
Teachers' practical skills development as a source of improvement (sop)5
In terms of developing my own teaching, I trust my own practice

evaluations more than what the experts say.
In terms of what teaching techniques work well, I have greater trust in
experienced teachers than in experts.
Analysis
Structural equation modelling (SEM) and descriptive statistical analysis were
used to analyse the relationships between the variables. Structural equation
modelling is suitable for confirmatory factor analysis and path analysis.
Assessments of fit between model and data are based on the following indices:
root mean square error of approximation (RMSEA), Tucker-Lewis index (TLI),
goodness-of-fit index (GFI) and comparative fit index (CFI). RMSEA < .05 and
TLI, GFI and CFI > .95 indicate good fit and RMSEA < .08 and TLI, GFI and
CFI > .90 indicate acceptable fit (Kline 2005).
The measurement and structural models were estimated with IBM SPSS Amos
22. The values of RMSEA, TLI, GFI and CFI indicate that the structural model in
Figure 2 has acceptable fit.
4
Items are inspired by, but not identical to, those that Skaalvik & Skaalvik (2007) term
Strain Factors.
5
Items are inspired by, but not identical to, measurements of teachers' internal attitudes
to learning, developed by (Opfer, Pedder & Lavicza, 2011).

111
Results
Tabel 1: Mean, standard deviation, skewness, kurtosis, and Cronbachs alpha.

N = 156 Indicators Mean SD Skewness Kurtosis Alpha
w2 3.87 1.80 -0.08 -1.00
secm w3 3.12 1.52 0.22 -0.98 0.831
w4 3.44 1.57 0.10 -0.83
w33 5.43 1.28 -0.85 0.61
con 0.710
w34 5.53 1.33 -0.53 -0.69
w37 3.84 1.90 0.14 -1.06
str1 0.753
w39 3.60 1.78 0.25 -0.86
w41 3.50 1.68 0.28 -0.76
str2 0.876
w43 3.46 1.64 0.24 -0.77
w103 4.76 1.45 -0.35 -0.31
sop 0.801
w104 5.24 1.38 -0.76 0.28
Figure 2: Structural equation modelling by using AMOS 22. Constructs and

abbreviations: Self-efficacy for classroom management (secm), external control (con),
stress (str1), conflicts induced by ICT in classrooms (str2), and teachers practical
knowledge epistemology as a source for improvement (sop).

112
Table 2
Hypothesis Formulation Result

1 Stress has a negative association with teacher efficacy for The association is negative
classroom management. (b(str1secm)= -.09), but
relatively weak (not
significant).
2 Conflict has a negative association with teacher efficacy The negative association
for classroom management. (b(str2secm)= -.23) is
significant. Hypothesis
supported.
3 Perception of shortcoming is negatively associated with The association is positive but
teacher efficacy for classroom management not significant
(b(str2secm)=
-.16). Hypothesis not
supported.
4 Practical skills re positively associated with teacher Practical skills development is
efficacy for classroom management. not associated with an
expectation of mastery in
classroom management:
(b(sopsecm)=.00)
In respect of the investigative assumptions, we find the following:
Table 3
Formulation Result
1 We investigate the relationship between conflicts and The association (b(constr2)=
perceptions of shortcoming in terms of influence over -.04) is very weak.
pupil motivation and self-discipline
2 We investigate the relationship between the significance The association
of practical skills and shortcoming. (b(sopcon)=.13) is positive.
3 We investigate the relationship between the significance The association
of practical skills and stress. (b(sopstr1)=.29) is positive.
4 We investigate the relationship between stress and The positive association
perception of shortcoming in respect of influence over (b(str1con)=.35) is
pupil motivation and self-discipline significant.
Discussion
The purpose of the article was to investigate strain factors and the significance of
teachers' practical skills development as a source of improvement statistically
associated with teachers' self-efficacy for succeeding in classroom management,
as well as reciprocal associations between the factors. The analysis indicates that
a perception of conflict (str2) is negatively associated with expectations of
mastering classroom management (secm). Stress (str1) and a perception of
shortcoming in terms of influencing pupils (con) have a rather weaker negative
association with teacher efficacy for classroom management (secm). There is a
strong, positive association between stress (str1) and conflict (str2), as well as
positive associations between shortcoming (con) and both stress (str1) and

113
conflict (str2). Teachers' emphasis on practical skills in preference to expert

advice (sop) is positively associated with shortcoming (con), stress (str1).
If these path coefficients reflect causal processes, the causal direction6 may be in
several directions. Since this cross-section investigation is unable to prove causal
direction, we must be cautious not to draw too strong conclusions about
causality. Research incorporating sequential analysis or controlled experiments
may help resolve questions about causal direction, but such research approaches
are demanding to carry out. We also note the positive association between
shortcoming in terms of influence on pupil motivation and self-discipline and
perceptions of stress and conflict. Here too, the causal direction is uncertain
even though it seems most plausible to assume that perceptions of stress and
conflict cause the perception of shortcoming.
There are sometimes significant differences of opinion regarding the use of

technology in schools, both amongst teachers, so-called experts, politicians and
policymakers. This article focusses on teacher perceptions. If politicians and
policymakers are to succeed with their leadership signals regarding improving
pupil learning and using ICT, it is important to understand the opportunities
and challenges and to respect the work done by teachers in present-day sixth-
form colleges.
Relatively little research is to be found concerning the work situation of teachers

in sixth-form colleges and teachers' perceptions, beliefs, preferences and
managerial challenges. One reason may be that classroom management is
sometimes equated with a mechanistic, authoritarian orientation that minimizes
the importance of positive interpersonal relationships and maximizes control
and compliance (Evertson & Weinstein, 2011:4). However, managerial
challenge is a topic of enduring concern for teachers. Some opinions expressed
by teachers may suggest that a gap exists between perceptions of reality among
teachers, particularly in sixth-form colleges, and those who are in a position to
shape educational policy (Kval, 2014) 7 plus educational researchers (Elstad,
2005). For example, a teacher in a sixth-form college writes the following:
As a teacher in a sixth-form college I experience every single day the

problem of digital gadgets. PCs and a variety of apps may be very
helpful to strongly-motivated pupils, but for a large proportion of pupils
they are a distraction that removes focus from subject-based learning.
Pretty much all research confirms this. As a county-council employee,
however, I have to be loyal to the resolutions passed by county
6
Strictly speaking, the analytical design does not provide a basis on which to determine
causal direction; only the strength of statistical associations.
7
This is one example. One of the top civil servants in the Ministry of Education and Research,
Ole Briseid, condemned some years ago teacher-centered instruction and emphasized progressive,
student-oriented and activity-oriented teaching methods: The Ministry has wanted to reform the
teaching methods in schools (in Norway) for a long time. He (Briseid) wants more project
work, less traditional teaching using the blackboard as a visual aid and more problem-based
teaching. To a larger extent, pupils will work independently and in groups with topics they
develop themselves. The teachers will be more like supervisors than lecturers, and computer
technology forces the development of new teaching methods (Kluge 2001, emphasised here).

114
councillors and by the Norwegian Directorate of Education and Training.

And they have decided that not just PCs but also the use of portable
applications on mobiles and tablets should play a greater and greater role
in teaching, in all subjects and at all levels. I have to accept this. I do not
like this development, but during my 15 years in the school no-one has
asked me (or my colleagues) whether we want this tremendous focus on
digital tools (Stre, 2016).
A task for future research is to determine the extent to which this type of opinion
is widespread and explore how contextual factors influence teacher efficacy and
performance in technology-rich environments. If there is a widespread
perception of a gap between the managers and the managed, those who are
responsible for political and administrative decisions should be concerned about
how teachers can succeed in their work. For the sake of balance: opinions are
also expressed at the other end of the scale. Some teachers, for instance, claim
that in the digital classroom, access to and use of tablets is a fantastic
opportunity (Ramo, 2016).
There are several limitations in our study. This type of analysis has limitations
from a conceptual perspective and in terms of its methodological (cross-sectional)
approach. We acknowledge these limitations and argue that they can serve as
point of departure for future research.
Conclusions and implications for future research

The strong emphasis on the accessibility of technology in the school is
controversial among politicians, teachers and researchers (Ministry of Education
and Research, 2014). For more than three decades, political quarters have been
expressing great expectations for the use of technology in schools (Elstad, 2016b).
This belief has been dented in the wake of an OECD report that provides an
empirical basis for stating that those countries that have invested in high
availability of technology display disappointing results. The internet navigating
skills of Norwegian pupils are actually poorer than those in countries with
limited availability to technology (but in which they actually train in adequate
navigating). Nor do computers improve pupil learning results (OECD, 2015).
Hattie uses the term distraction politics to describe the faith in technology as a
magic bullet in educational policy (Hattie, 2015:30). It is likely that views on
educational technology in Norway are on the change. For more than three
decades, researchers, policy makers and school professionals have all harboured
great expectations towards the use of technology in schools. This belief has
received a hard knock. School professionals and policy makers are seeking
answers to the question of how schools ought to relate to challenges created by
the use of technology in the school based on a firmer grasp on reality.
Our study documents empirical associations between teachers' perceived

shortcoming, stress, conflict, and teacher efficacy in respect of classroom
management. We note that the average for teacher efficacy for classroom
management (3.48) is lower than the neutral mid-point of the scale (which is 4),
which provides an indication of the challenges encountered by teachers in these
schools when they are to master class management in technology-rich

115
surroundings. This should generate humility regarding teachers' work

challenges. There is a fairly high standard deviation to the various items used to
measure teacher efficacy (see Table 1). This reveals a considerable disparity in
how teachers view this question. As mentioned earlier, the selection is taken
from schools with a score of roughly the national average in terms of school
added value indicator and that admit pupils with average grades from
secondary school. Since we have found that the challenges of leading pupil
groups in classrooms in which each pupil has a computer with full internet
access are fairly different in schools whose pupils have high entry grades, we
would not like to claim that measurements which show how teachers at such
schools perceive mastery of teaching are representative of Norwegian sixth-form
colleges in general. More research is needed here based on a selection of schools
at which pupils have a generally high motivation for schooling and schools in
which pupils have a low educational motivation in technology-rich learning
environments. Our measurements strictly speaking are only valid for the three
schools included in the selection, but we believe that our estimates provide an
indication of how teachers perceive self-efficacy and similar at sixth-form
colleges that are located in the mid-stream of pupil performance. The three
schools had no previous history of ICT-supported teaching before the system of
one PC per pupil was introduced beyond the fact that this system was a
consequence of national and local management signals.
Indications exist that schools with a strong focus on ICT-supported teaching

attract teachers with a particular interest in technologically-supported teaching
(Hauge, 2016). The selection effect in terms of teachers' choice of workplace has
been documented (Bonesrnning, Falch & Strm, 2005) and as such our
estimates are not valid for with a predominance of this type of teacher. In
addition, this research should be followed up by studies relating to courses other
than the pre-academic study programme. For instance it seems plausible that
pupils on vocational programmes display other types of unwillingness than
pupils on the pre-academic study programme. This may not least depend on the
subject that the teachers are teaching. Our analysis explains some, but not all, of
the variance in the dependent variables. It is a matter for future research to
develop larger analytical data sets that will provide opportunities to analyse the
significance of subjects, programme and different types of pupil groups.
The challenges of the technology-rich classroom create pressure on teachers

when leading educational politicians state that: good classroom management
and very clear rules are required regarding how PCs are used in the school. It is
up to the teacher and the school to ensure that these exist (Nilsen, 2011). A
natural conclusion from such statements is that anyone who does not succeed is
quite simply not good enough. There is a high degree of non-academic activity
in the lessons (Hatlevik et al., 2013). That the teacher has a clearly-defined
responsibility for what the pupils learn at an age range at which many pupils
feel that they themselves should be deciding what they should do during lesson
times, creates pressure on teachers. If teacher efficacy for maintaining order and
discipline is contextual, it is unhelpful when responsible politicians express
themselves in this manner. For this reason we need more research about the
significance of context.

116
We also need more research to provide normative conclusions about how

teachers can succeed better with classroom management. An emphasis on
practical experience and skills has a negative association with self-efficacy for
teaching and a positive association with stress, conflict and a sense of
shortcoming. Future research should investigate whether an emphasis on
experience-based knowledge dissemination through teacher networks (Larsen &
Nilsen, 2012) is useful for promoting teachers' professional development in
technology-rich environments. However, the averages for items measuring
teachers' belief in experience rather than expert advice as a source of
improvement are fairly high (Table 1). A plausible interpretation is a low level of
faith in research providing information on pedagogic implications. A broader set
of terms and items is needed here in order to draw clear conclusions. Our survey
indicates no statistical association between teachers' belief in experience as a
source of improvement and their teacher efficacy for classroom management.
In another article we have attempted to deduce the implications for classroom

management of what we regard as research-based knowledge about teaching
and learning in technology-rich environments (Arnesen et al., 2016). There are
grounds for saying that even subtle differences in the physical setting for
interaction between teacher and pupils and the terms of the teacher's authority
(computers with internet access for all pupils, tightening up of the final
assessment principle and the removal of the absence limit in Norway are
examples from the period 2007-2014) can have significance in relation to the
opportunities a teacher have to exert influence. If teachers are to succeed in
exerting an influence to promote pupil learning in technology-rich classrooms,
the teachers need adequate tools for classroom management (Ifenthaler and
Schweinbenz, 2013). There has been too little awareness of this matter (Elstad,
2016c). We need more knowledge about the significance of the management
tools in one-to-one laptop environments.
Acknowledgement
This work was supported by the Norwegian Research Council under Grant
[number 218245]. Sources of support for the work, including sponsor names
along with explanations of the role of those sources if any in study design;
collection, analysis, and interpretation of data; writing of the report; the decision
to submit the report for publication; or a statement declaring that the supporting
source had no such involvement.
References
Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological review, 89(4), 369.
Arnesen, T., Elstad, E. & Christophersen, K.A. (2016). Antecedents of pupils self-
regulatory strength in technology-filled school environments. International
Journal of Learning, Teaching and Educational Research. 15 (3), pp. 218-241, March
2016. Retrieved from: http://ijlter.org/index.php/ijlter/article/view/457/pdf
Bandura, A. (1994). Self-efficacy. New York: John Wiley and Sons.
Berliner, D. C. (1986). In pursuit of the expert pedagogue. Educational Researcher, 15(7), 5-
13.

117
Blikstad-Balas, M. (2012). Digital Literacy in Upper Secondary SchoolWhat Do Students

Use Their Laptops for During Teacher Instruction?. Nordic Journal of Digital
Literacy, 7(02), 81-96.
Bolick, C.M. & Bartels, J.T. (2015). Classroom Management and Technology. In E. Emmer,
& E. Sabornie (eds.), Handbook of Classroom Management, New York: Routledge.
(pp. 479-495).
Bolick, C.M. and Cooper, J.M. (2011) Classroom Management and Technology. In
Evertson, C. M., & Weinstein, C. S. (Eds.), Handbook of classroom management:
Research, practice, and contemporary issues. New York: Routledge. (pp. 541.557).
Bonesrnning, H., Falch, T., & Strm, B. (2005). Teacher sorting, teacher quality, and
student composition. European Economic Review, 49(2), 457-483.
Brown, D.F., & Rose, T.J., (1995). Self-reported classroom impact of teachers' theories
about learning and obstacles to implementation. Action in Teacher Education, 17
(1): 20-29.
Chi, M. T., Glaser, R., & Farr, M. J. (2014). The nature of expertise. New York: Psychology
Press.
Chinn, C. A., Buckland, L. A., & Samarapungavan, A. L. A. (2011). Expanding the
dimensions of epistemic cognition: Arguments from philosophy and psychology.
Educational Psychologist, 46(3), 141-167.
Elstad, E. (2005). Om prosesstyring og resultatorientering i utdanningspolitikken [On
process steering and result orientation in educational policies]. Norsk Pedagogisk
Tidsskrift, 89 (2), 132-145.
Elstad, E. (2006). Understanding the nature of accountability failures in the technology-
filled classroom: disaffected students and teachers who give in. Journal of
Curriculum Studies, 38 (4), 459-481.
Elstad, E. (2008). Building self-discipline to promote learning: students volitional
strategies to navigate the demands of schooling. Learning Inquiry, 2 (1), 53-71.
Elstad, E. (2016a). Educational technology in schools. Policymaking and policy
enactment. In Elstad, E. (ed.). Digital expectations and experiences in education (pp.
48-57). Rotterdam/ Boston /Taipei: Sense Publishers.
Elstad, E. (2016b). Educational technology: expectations and Experiences. An
introductory overview. In Elstad, E. (2016). Digital expectations and experiences in
education (pp. 3-28). Rotterdam/ Boston /Taipei: Sense Publishers.
Elstad, E. (2016c). Why is there a wedge between the promise of educational technology
and the experiences of a technology-rich pioneer school? In Elstad, E. (2016).
Digital expectations and experiences in education (pp. 77-96). Rotterdam/ Boston
/Taipei: Sense Publishers.
Evertson, C. M., & Weinstein, C. S. (2011). Classroom Management as a Field of Inquiry.
In Evertson, C. M., & Weinstein, C. S. (Eds.). (2011). Handbook of classroom
management: Research, practice, and contemporary issues (pp. 3-15). New York:
Routledge.
Falch, T., Bensnes, S. & Strm, B. (2016). Skolekvalitet i videregende opplring. Utarbeidelse
av skolebidragsindikatorer og ml p skolekvalitet. [School Quality in upper
secondary schools. Value Added Estimation and measuring School Quality],
Trondheim: Senter for konomisk forskning rapport nr. 01/16.
Fang, Z. (1996). A review of research on teacher beliefs and practices, Educational
Research, 38 (1): 47-65

118
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for life in
a digital age: The IEA International Computer and Information Literacy Study.
International report. Cham: Springer.
Greene, J. A., & Seung, B. Y. (2014). Modeling and measuring epistemic cognition: A
qualitative re-investigation. Contemporary Educational Psychology, 39(1), 12-28.
Hatlevik, O.E., Egeberg, B., Gudmundsdottir, G.B., Loftsgarden, M. & Loi, M. (2013).
Monitor skole 2013. Om digital kompetanse og erfaringer med bruk av IKT i skolen
[Monitor School 2013. On digital competence and experiences with use of ICT in
schools], Oslo: Senter for IKT i utdanning.
Hauge, T.E. (2016). On the life of ICT and school leadership in a large-scale reform
movement: A case study. In Elstad, E. (2016). Digital expectations and experiences
in education. SENSE Publishers, Rotterdam/ Boston /Taipei, (pp. 97-116).
Huberman, M. (1989). The professional life cycle of teachers. The Teachers College Record,
91(1), 3157.
Ifenthaler, D., and Schweinbenz, V. (2013). The acceptance of Tablet-PCs in classroom
instruction: The teachers perspectives. Computers in Human Behavior, 29(3), 525-
534.
Klassen, R. M., and Chiu, M. M. (2010). Effects on teachers self-efficacy and job
satisfaction: Teacher gender, years of experience, and job stress. Journal of
Educational Psychology, 102(3), 741.
Kline, R. B. (2005). Principle and practice of structural equation modeling. New York: The
Guildford Press.
Kluge, L. (2001). Reformforsk i 700 skoler [Reform attempts in 700 schools], Aftenposten
July, 5th.
Koehler, M. J. & Mishra, P. (2009). What is technological pedagogical content knowledge?
Con-temporary Issues in Technology and Teacher Education, 9, 6070.
Koepke, S., & Denissen, J. J. (2012). Dynamics of identity development and separation
individuation in parentchild relationships during adolescence and emerging
adulthoodA conceptual integration. Developmental Review, 32(1), 67-88.
Kval, K.E. (2014). Det store skolesviket. En diagnose og 18 forslag til forbedringer [The large
school betrayal. A diagnosis and 18 suggestions for improvements], Oslo:
Dreyer.
Kvernbekk, Tone (2001). Erfaring, praksis og teori [Experience, praxis, and theory]. In T.
Kvernbekk (ed): Pedagogikk og lrerprofesjonalitet. Gyldendal Akademisk. s 146 -
163
Larsen, S.K. & Nilsen, D. (2012). Mener PC-bruk i skolen er lrerens ansvar [Believe
computer use in schools is the teacher's responsibility] VG 26 juni 2012.
Retrieved from: http://www.vg.no/nyheter/innenriks/skole-og-
utdanning/mener-pc-bruk-i-skolen-er-laererens-ansvar/a/10059849/
Leinhardt, G., & Greeno, J. G. (1986). The cognitive skill of teaching. Journal of educational
psychology, 78(2), 75-95.
Mulholland, J., & Wallace, J. (2001). Teacher induction and elementary science teaching:
Enhancing self-efficacy. Teaching and Teacher Education, 17(2), 243261.
Nilsen, D. (2011). Lrer: - Skolen er blitt som en internettkaf [Teacher: - The school was
as an internet cafe] VG 18.juli 2011.
http://www.vg.no/nyheter/innenriks/skole-og-utdanning/laerer-skolen-er-
blitt-som-en-internettkafe/a/10080543/

119
Ministry of Education (2014). Strategy: The teacher promotion. Pulling towards knowledge in
school. Oslo: Ministry of Education.
OECD (2015). Students, Computers and Learning. Making the Connection, Paris: OECD.
Opfer, V. D., Pedder, D. G., & Lavicza, Z. (2011). The role of teachers orientation to
learning in professional development and change: A national study of teachers
in England. Teaching and Teacher Education, 27(2), 443-453.
Ottenbreit-Leftwich, A., Glazewski, K., Newby, T. & Ertmer, P. (2010). Teacher value
beliefs as-sociated with using technology: Addressing professional and student
needs. Computers and Education, 55, 13211335.
Ramo, M.U. (2016) IKT i skolen, en fantastisk mulighet[ICT in schools: a fantastic
opportunity]. http://www.minervanett.no/ikt-i-skolen-en-fantastisk-mulighet/
Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic
achievement. Econometrica, 73(2), 417458.
Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence
from panel data. The American Economic Review, 94(2), 247252.
Scherer, R., Siddiq, F., & Teo, T. (2015). Becoming more specific: Measuring and
modeling teachers perceived usefulness of ICT in the context of teaching and
learning. Computers and Education, 88, 202214.
Senter for IKT i utdanning (2015). Hensiktsmessig bruk av IKT i klasserommet en veileder.
[Appropriate use of ICT in the classroom - a manual] Oslo: Senter for IKT i
utdanning. Retrieved from:
https://iktsenteret.no/sites/iktsenteret.no/files/attachments/veileder_hensikts
messig_bruk_bm_lav.pdf
Skaalvik, E. M., & Skaalvik, S. (2007). Dimensions of teacher self-efficacy and relations
with strain factors, perceived collective teacher efficacy, and teacher burnout.
Journal of Educational Psychology, 99(3), 611.
Strms, H. I., & Brten, I. (2010). The role of personal epistemology in the self-
regulation of internet-based learning. Metacognition and Learning, 5(1), 91-111.
Stre, B.O. (2016). Kostbar loking p Facebook [Pricey surfing on Facebook]. Retrieved
from https://disqus.com/by/bjornsatre/
Tschannen-Moran, M. Hoy, M. A. & Woolfolk A.E. (2007). The differential antecedents
of self-efficacy beliefs of novice and experienced teachers. Teaching and Teacher
Education, 23(6), 944956
Van Acker, F., van Buuren, H., Kreijns, K., & Vermeulen, M. (2013). Why teachers use
digital learning materials: The role of self-efficacy, subjective norm and attitude.
Education and Information Technologies, 18(3), 495-514.
Woolfolk, A.E. & Hoy, W.K. (1990). Teachers sense of efficacy and beliefs about control.
Journal of Educational Psychology, 82(1), 8191.
Zacharia, Z.C., & de Jong, T. (2014). The effects on students conceptual understanding
of electric circuits of introducing virtual manipulatives within a physical
manipulatives oriented curriculum. Cognition and Instruction, 32, 101-158.
Zee, M., & Koomen, H. M. (2016). Teacher Self-Efficacy and Its Effects on Classroom
Processes, Student Academic Adjustment, and Teacher Well-Being A Synthesis
of 40 Years of Research. Review of Educational Research, doi: 0034654315626801.

120

Vol. 16, No. 1, pp. 120-134, January 2017
Using Reflective Journaling to Promote

Achievement in Graduate Statistics Coursework
J. E. Thropp
Embry-Riddle Aeronautical University
Daytona Beach, FL
Abstract. By encouraging students to reflect on their own learning

processes and identify areas that they understand as well as areas that
require further improvement, their academic performance can improve.
This reflection promotes the development of metacognitive skills, and
may be especially beneficial for students facing new and challenging
material, notably statistics coursework. Students grades from two
different sections of a graduate-level introductory statistics course were
compared. In the experimental section, students were encouraged to
complete a weekly reflective journal about their learning progress and
related experiences in the course. The control section followed the same
course structure as the experimental section, but did not complete the
journals. Independent samples t-tests were conducted on assignment
and test scores as well as the final course grade for the experimental and
the control sections. The experimental section performed significantly
better than the control section on one assignment and one test. The
experimental section also had a marginally higher average than the
control section on a second assignment. The results and analysis of
journal content show support for the use of reflective journaling to
improve the student learning experience.
Keywords: Journals; statistics education; learning management systems.
Background
Instructors strive to encourage and support students to enhance their learning
outcomes and to set them on a path to succeed not only in their coursework, but
also in their objectives outside of the classroom. Motivation is undoubtedly a
key component to college student success, yet can be hindered when the
material to be learned is associated with negative affective components, evokes
anxiety, and is perceived as difficult. Statistics is one such course that often fits
the aforementioned description. This paper discusses how keeping weekly
reflective metacognitive journals may have improved the learning experiences
and academic performance of students in a graduate-level statistics course.
Factors Influencing Students Statistics Attitudes and Performance

Coursework in statistics has been generally described by students as non-
engaging and difficult (Pan & Tang, 2004), oftentimes resulting in generally
adverse experiences (Murtonen & Lehtinen, 2003). A review of the literature

121
reveals a wide variety in potential factors underlying the reasons why many
students harbor negativity towards statistics and exhibit difficulty in performing
the coursework.
Seeing the value of statistics in ones career and other applications beyond the
classroom is a factor influencing attitudes towards statistics (Fullerton &
Kendrick, 2013; Ramirez, Schau, & Emmioglu, 2012; Schau, Millar, & Petrocz,
2012). Interest level in statistics also plays a role in statistics-related anxiety.
Macher et al. (2013) reported a negative correlation between the two variables, as
individuals tend to devote greater time and effort to objectives that interest
them. Anxiety and lack of intrinsic motivation have been linked with students
procrastination of statistics tasks (Dunn, 2014). Indeed, task-engagement and
seeing the value in a task are important motivational components that underpin
learning (Ryan & Deci, 2000).
Academic self-concept (Marsh & Martin, 2011) involves students perceptions of

their own academic skill level and knowledge in a specific domain. Macher et
al. (2013) found a negative relationship between mathematics self-concept and
anxiety about statistics. Low self-efficacy in mathematics and negative prior
experiences with mathematics can be linked to anxiety specific to statistics
coursework (Zeidner, 1991). Students who feel theyve performed well in
previous mathematics coursework also tend to have positive attitudes towards
statistics (Hannigan, Hegarty, & McGrath, 2014), yet an important consideration
is that mathematical and statistical abilities, while related in terms of numerical
manipulation, also differ in that statistical ability draws upon additional
domains, such as verbal reasoning and use of logic (Fullerton & Kendrick, 2013;
Johnson & Kuennen, 2006).
Statistics anxiety is a situation-specific form of anxietythat occurs when an

individual is exposed to statistics content, problems, instructional situations, or
evaluative contexts and is an enduring, habitual type of anxiety (Macher et
al., 2013, p. 536). It can interfere with the process of learning course material,
thereby resulting in poor grades (Cook, 2010; Onwuegbuzie & Wilson, 2003;
Williams, Payne, Hodgkinson, & Poade, 2008; Zeidner, 1991).
In the development of the Statistics Anxiety Rating Scale, a six-factor model of

statistics anxiety comprised of two dimensions emerged (see Cruise et al., 1985).
The first dimension, which pertains to anxiety, has three factors: a) interpretation
anxiety, b) test and class anxiety, and c) fear of asking for help. The second
dimension, which pertains to attitudes, also has three factors: a) worth of
statistics, b) computational self-concept, and c) fear of statistics teachers. This
model has been supported using several diverse populations of statistics
students over the years (e.g., Mji & Onwuegbuzie, 2008; Papousek et al., 2012;
Teman, 2013), and recently by DeVaney (2016) who validated it with graduate
level statistics students.
Macher et al. (2011) indicated that students with statistics anxiety tend to exert
less effort in studying and procrastinate in their coursework as well as in

122
studying for exams. Decremented intrinsic motivation has also been linked with
students statistics anxiety and task procrastination (Dunn, 2014). At the
graduate level, statistics anxiety can extend into other aspects of academic
performance, such as comprehending research articles involving statistical
results (Onwuegbuzie, 1997), and performing research-related activities that
require statistical knowledge (Onwuegbuzie, 2003).
Various individual differences in personality and disposition have also been

linked with performance in statistics coursework. Furnham and Chamorro-
Premuzic (2004) documented three such influential factors: The Big Five
personality traits of conscientiousness and extraversion (see Costa & McCrae,
1992), and spatial ability. They reported a positive relationship between
conscientiousness and statistics exam scores at the college level, a negative
relationship between extraversion and statistics performance, and a positive
relationship between spatial ability and statistics performance. Individual
differences in mathematical orientation have been evaluated with the preference
for numerical information (PNI) construct. Viswanathan (1993) describes PIN as
a disposition for the use (or disuse) of numerical information in general, and that
it reflects students willingness to learn and use mathematics and statistics.
Additionally, trait anxiety, a stable disposition toward feeling anxious in general
(Spielberger, 1972) has been linked to statistics anxiety (Macher et al., 2013;
Papousek et al., 2012). Anxious individuals also generally tend to be pessimistic
about their future task performance (Eysenck & Derakshan, 1997), which could
likely lead to negative attitudes regarding statistics coursework.
Metacognition and Reflective Journals

According to Paris and Winograd (1990, p. 7), students can enhance their
learning by becoming aware of their own thinking as they read, write, and solve
problems. This awareness of ones own thinking process is referred to as
metacognition, and is the basis for promoting self-directed learning, which
encourages learners to become active participants in their own academic
pursuits. Flavell (1976) further describes metacognition as:
ones knowledge concerning ones own cognitive processes and
products or anything related to them, e.g., the learning-relevant
properties of information or data. Metacognition refers, among other
things, to the active monitoring and consequent regulation and
orchestration of these processes in relation to the cognitive objects or data
on which they bear, usually in the service of some concrete goal or
objective. (p. 232)
Cognitive self-appraisal is one component of metacognition. As observed by

Flavell (1978), this self-appraisal focuses on students abilities to address what
they know about a subject, how they process information about the subject, and
when and why they should apply related knowledge. Further, Douglass and
Morris (2014) suggest that an understanding of ones own learning style
promotes self-directed or more autonomous learning in which students develop
their own comprehension of the topic at hand. This knowledge enables a learner
to select and use appropriate metacognitive self-management strategies to

123
develop or modify plans for solving a problem in order to achieve his or her goal
(Paris & Winograd, 1990). According to the theory of self-determination,
intrinsic motivation to succeed academically is facilitated by a more autonomous
style of learning (Ryan & Deci, 2000), the benefit of which is often improved
performance in coursework (Giuffrida, Lynch, Wall, & Abel, 2013).
Understanding ones personal learning style is a result of the self-reflection of

intellectual and affective activities in which individuals engage to explore their
experiences in order to lead to new understandings and appreciations (Boud,
Keogh, & Walker, 1985, p. 19). Students can use journals to document these
experiences and synthesize them with their other prior knowledge to enhance
learning (Boud, 2001). Moon (1999) describes several ways in which journal-
keeping can promote proactive learning through reflection, such as developing
critical thinking and greater inclination towards inquiry about a topic, increasing
active participation and ownership in the learning process, promoting self-
empowerment, and developing an alternative means of communication when
self-expression is difficult. A journal can also aid the writer in identifying his or
her intentions and goals as they relate to a situation, and is especially helpful
when the situation is complex and unfamiliar. Further, through reflection, a
writer can identify questions that need to be addressed in order to be productive
in unfamiliar situations (Boud, 2001).
Journaling in Learning Mathematics

Writing to Learn (WTL) is a movement in education that emphasizes the benefit
of language as a key component in knowledge acquisition, regardless of
curricula. WTL asserts that students develop an understanding of material
through their own written thoughts and observations (Bazerman et al., 2005).
Writing about course material is an active, engaging process that encourages
students to reconstruct the meaning of the material in their own relatable terms
(Borasi & Rose, 1999). Particularly supportive of learning is expressive writing,
in which writers probe their own thoughts and ideas for reflection (Britton,
Burgess, Martin, McLeod, & Rosen, 1975).
Writing to Learn Mathematics (WTLM) is a branch of WTL that focuses on

writing activities to enhance understanding of mathematics. Examples of such
activities include writing to explain topics in terms that are more relatable to the
student and writing about attitudes regarding the topic. Barker et al. (2004, p.
16) assert that requiring students to write about mathematics helps them learn
and gives instructors valuable insight into the nature of their understandings.
Indeed, by reading students journals, instructors can gain an awareness of their
students perceptions and in turn, this can give them opportunities to evaluate
their own pedagogical styles (Kenny, Schoffner, & Norris, 2014). Further, Meel
(1999) describes how writing about mathematics helps to personalize otherwise
unfamiliar concepts to the students.
Borasi and Rose (1999) showed how WTLM is an expressive form of writing that
can take the form of journals. In their study, undergraduate algebra students

124
used journals to document their feelings about mathematics, and reflecting on

their own learning processes bolstered their knowledge about the course
material. In a study by Bagley and Gallenberger (1992), high school
mathematics students kept daily journals to help develop problem-solving
strategies. They described this activity as a means of knowing what we think
a means of shaping, clarifying, and discovering our ideas (p. 660). Loud (1999)
found that undergraduate mathematics students who kept a weekly journal
exhibited not only higher final exam scores, but also more positive attitudes
regarding the course.
Writing activities have also been shown to aid in reducing statistics-related

anxiety and improve statistics course performance (Chiou, Wang, & Lee, 2014).
In the one-minute paper strategy, students anonymously document and submit to
their instructors their responses to the following questions: What is the most
important concept you learned in class today? and What questions remain unanswered?
This exercise allows students to identify concepts that require further
understanding and describe difficult concepts using their own words.
Research Questions and Hypotheses

Given the current body of knowledge, the researcher sought to address whether
keeping a reflective, online journal could improve the learning experiences of
students enrolled in a statistics course, specifically at the graduate level. In
doing so, an exploratory study was conducted in which students from one
section of the course were offered an optional journaling task, and their average
grades were compared to those from a section in which journaling was not
offered. It was hypothesized that students who completed the journals would
have higher mean scores on the evaluation items in the course (e.g., grades on
assignments and tests) as well as the final average in the course as compared to
students who did not complete the journals. If results indicated that keeping a
journal was associated with higher average grades, then a more widespread
implementation of journaling activities in future statistics courses might be
supported. Additionally, the content of the journals was evaluated in the
interest of investigating trends in student perceptions of the course and self-
assessments of their own knowledge, skills, and affective states. The researcher
was also interested in the nature of the questions and concerns students had
throughout the course, as well as any evidence of motivational factors that could
be driving academic behavior and performance.
Methodology
Study design. This study used a mixed-methods quasi-experimental design to
investigate the effect of reflective journaling on student performance in a
statistics course. Students enrolled in two different sections of a graduate-level
introductory statistical analysis course were recruited as participants. This
course is a 16-week, in-person, core requirement in a Master of Science degree
program in the university at which this study was conducted.
In the experimental section of the course, the students (n = 13) had the option of
volunteering to keep online journals to document their learning experiences in

125
the course throughout the semester. The control section (n = 14) was conducted
in the same manner as the experimental section, with the exception that the
students were not offered the option to complete the journals. Participation was
voluntary, and approval to conduct the study from the institutions ethics
review board was obtained.
Participant description. Students enrolled in both sections of the course were

comprised of a variety of backgrounds; they were a mixture of traditional and
nontraditional types, various nationalities, and differing levels of prior
mathematics coursework. While all of the students had completed at least one
introductory level mathematics course at the undergraduate level, only a portion
of them had completed any undergraduate statistics coursework. All of the
students were familiar with the Blackboard online learning management system
(LMS) that was used in the course, but none had expressed previous use of the
journal tool available on it.
Procedure. On the first day of class in the experimental section, the instructor
announced the option of participating in the study to the students, explaining
that they had the option of completing an online journal regarding their learning
progress each week. The journals were made available through a journal tool
embedded in the Blackboard LMS site for the course. Students in the
experimental section were asked to document briefly their weekly learning
experiences (not including weeks in which tests were administered) using the
online journal regarding the following issues:
Concepts covered in class in the current week that they felt they had
difficulty understanding and might necessitate further clarification
Questions they had regarding the concepts covered in class that week
Information that they felt might be helpful to explain or clarify the
difficult concepts
How they planned to seek clarification of these concepts
Although these specific journal topics were posed to them, they were not
prohibited from writing about other topics pertaining to their experiences in the
course.
Students were expected to spend not more than approximately five to ten
minutes per week on the journaling task, and were advised to work on the
journal when it was convenient for them. They were told that participation was
strictly voluntary with no bearing on course grade and that the journals were to
be used as a learning aid and would not be read by anyone except the instructor.
They were also informed that the instructor only assessed the journals for
completion and honest effort, not for content. Neither course credit nor any
other compensation was offered in exchange for participation. The instructor
described the journaling activity as an optional task to the students in the first
week of the course, and listed it as an optional weekly task on the course
syllabus. The instructor did not remind them to do the journals afterwards;
considering the exploratory nature of this study, the instructor preferred to
observe the natural tendency for students to use the journal without being
prompted.

126
In both sections, students were evaluated on three assignments, two tests, and
one final exam in fulfillment of course requirements. These evaluation items are
described below.
Data analysis assignments. Students scores on three different data analysis

assignments were collected. The course lectures, textbooks, and other
instructional resources (e.g., checklists for proper formatting and requirements
for what to report in results sections, templates to use for the assignments)
instructed the students in how to successfully complete the assignments.
The Statistical Package for the Social Sciences (SPSS) was the statistical software
package students learned to use in the course. For each of the three data
analysis assignments using SPSS (i.e., SPSS Assignments 1, 2, and 3), students
were provided with a dataset and asked to test several different hypotheses
using the software. Each of the three SPSS assignments covered a different set of
statistical operations taught in the course (e.g., one assignment covered t-tests,
another covered correlations and regressions, etc.). To succeed in these
assignments, the students were required to know the proper type of statistical
test to execute, how to execute it using SPSS, how to interpret the SPSS output,
and how to write the results and discussions in the American Psychological
Association (APA) format.
Tests. There were two multiple-choice tests administered in the course. The
questions covered statistical concepts that were taught in the course lectures, but
did not cover the use of SPSS or APA formatting. Test 1 covered material taught
in the first half of the course, while Test 2 covered material taught in the second
half of the course.
Final exam. The final exam was cumulative, covering all statistical operations
taught in the course. For the final exam, students received a dataset and were
asked to test several hypotheses using SPSS. Successful performance on the final
exam required students to be able to determine which statistical operation to
execute for each hypothesis test and explain why they chose each particular
operation, report the results of each hypothesis test, and briefly explain what the
results of each hypothesis test meant.
For both the experimental and control sections, student grades were collected
from the LMS at the end of the semester. Independent samples t-tests were
conducted on grades for all evaluation items to compare the experimental and
control sections. For the experimental section, the content from the online
reflective journals was collected from the journal tool in the LMS.

127
Results
Student performance. Grades on three SPSS assignments, two multiple-choice
tests, one cumulative final exam, and the final course average were compared.
The experimental section performed significantly better (p < .05) than the control
section on SPSS Assignment 1 and Test 1. The experimental section had a
marginally higher mean score (p = .07) than the control section on SPSS
Assignment 2. Although not significantly different, the experimental section
also had numerically higher mean scores than the control section on the
remaining evaluation items. See Table 1 for the significant differences.
Table 1: Significant Differences between Experimental and Control Sections

Evaluation Experimental Control t-statistic and
Item Section Section Cohens d (where
Mean Score Mean Score significant)
(SD) (SD)
SPSS Assignment 1 173.75 157.79 t(24) = 2.416,
(200 points possible) (17.77) (15.14) p = .024*
d = .93 (large effect)
SPSS Assignment 2 183.46 173.79 t(25) = 1.884,
(200 points possible) (10.86) (14.97) p = .071
Test 1 86.15 77.32 t(25) = 2.377,
(100 points possible) (9.02) (9.88) p = .025*
d = .93
(large effect)
Note. * Indicates significant at the .05 level.
Although the mean scores of the experimental section were numerically higher
than those of the control section in the remaining evaluations (SPSS Assignment
3, Test 2, and the final exam), the differences were not significant (all p > .05).
There was also no significant difference in the final course averages between the
two sections (p = .096).
A closer look was then taken at the students in the experimental section. Six of
the 13 participants opted to complete a journal entry more than once. The
performance of these six journal writers was compared to that of the students
who opted to not use the journal more than one time. Performance was
significantly higher among journal users on SPSS Assignments 2 and 3 (see Table
2). Although not significantly different, the journal users also had numerically
higher mean scores on the remaining evaluation items than those who did not
opt to use the journal.

128
Table 2: Significant Differences between Journal Users and Non-Journal Users within
the Experimental Section
Evaluation Journal Users Non-Journal t-statistic and
Item Mean Score Users Cohens d (where
(SD) Mean Score significant)
(SD)
SPSS Assignment 189.86 176.0 t(11) = 2.759,
2 (9.55) (8.56) p = .019*
(200 points d = 1.53 (large
possible) effect)
SPSS Assignment 189.43 178.67 t(11) = 3.603,
3 (6.13) (4.27) p = .004*
(200 points d = 2.04 (large
possible) effect
Note. * Indicates significant at the .05 level.
Analysis of students journal contents. The contents of the journals from the
experimental section were analyzed. Of the 13 students enrolled in the semester
in which the metacognition journals were introduced, seven students used the
journal at least once (six of whom used the journal more than once), creating a
total of 25 entries. Six of the students refrained from using the journal
altogether. The number of journal entries was greatest in the first five weeks of
class, and then declined thereafter (see Figure 1).
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Figure 1. Number of journal posts by week.
Some journal entries documented multiple topics. The content was categorized
according to whether it featured a question or a statement, and then by the topic.
The content of the entries in terms of the specific topics posed to the students is
as follows. Students made eleven statements about their learning of statistical
concepts, four statements about using SPSS, and one statement about using APA
format. Students also used the journal to ask the instructor a total of seven

129
questions regarding statistics material: four questions were about statistical

concepts, two questions were about how to do an assignment, and one question
was asked about how to use APA format.
In addition to the specific topics posed to them, students also used to the journal
to express general motivational factors; there were ten instances of this type of
content. One student (i.e., Student A) regularly used the journal to document
her feelings about the course: how she was progressing, external pressures that
added to her workload, her desire to perform well in the course, questioning her
ability at times, and also expressing confidence in herself. Some of her selected
quotes are as follows:
Student A: The hard work really paid off! I still cant believe how well I did on
this last paper. Now I just need to keep up the work and do well on the next
paper and test.
Alright, so I had a lot of things going on at work and in my life so I was not
able to take as much time for the paperthen again, I didnt realize how much
time was really needed.
I am happy to see the other students from our class tutor each otherit shows
how dedicated we all are to try to do well in this class.
Some examples of the entries made by other students are as follows:
Student B: This week in class the professor went over the basics of SPSS. I
understood most of the topics covered. However, I will likely review on my own
with the SPSS software to become comfortable with the processes of re-coding
data, analyzing it, and creating charts to represent the data visually.
Student C: Pretty much I understand everything covered in class, however this

[sic] are the things I might bump into later:
Forgetting specific definitions and concepts

Remembering the rules for the whole course
The plan? Keep going back to do quick readings of notes and search for material
on the internet that reviews the topics... regarding the rules my idea is to keep in
touch with the TA and read the syllabus...
Other thing for me was that many things were new to me since I haven't been in
ERAU before or for that matter in a University in the U.S.A.
Student D: I knew the day would come when the material becomes more
difficult. I still understand most things in class but when I started working on
the paper, APA and the table made it so difficult. I will definitely seek help and
try to see if the TA has time to help.

130
Instructor reply to Student D: Formatting tables is one of the most difficult

parts of APA. Do your best on the tables for now and come see us for help. You
will get it right with practice.
Discussion and Conclusion

In general, the experimental section that used the journals executed better
academic performance in the course assignments and tests than the control
section. These differences were significant for SPSS Assignment 1 and Test 2,
and marginal for SPSS Assignment 2. The differences were not significant in the
other course evaluation items. There was also no significant difference in final
course averages.
Though completing the weekly journals was entirely voluntary, some students
participated somewhat regularly. At the beginning of the semester, participation
was more frequent; however, participation declined as the semester continued,
possibly due factors such as students forgetting to participate and reduced
novelty of the task. To maintain a naturalistic environment, the instructor did
not regularly remind students to complete their journals. This hands-off
approach may have contributed to the decline in journal entries as a function of
time. Similarly, Borasi and Rose (1989) reported that students journaling in a
mathematics course waned when instructors did not prompt them to do so.
While it cannot be determined why some of the students refrained from using
the journals, Boud (2001) describes some of the factors that can discourage
individuals from keeping journals. Writing about ones perceived deficiencies,
sources of unease, and problems can be prohibitive, especially if the writer is
concerned about the identity of the reader and perceived negative consequences
that could result. Knowing that the instructor would be reading about students
criticisms of not only themselves, but also possibly of the course itself, could
have been deterrents to participation.
Students used the journals to document what they had learned that week mainly
in statistical concepts; however, some also documented what they had learned
about the SPSS software and APA formatting. Statements about what they had
learned outnumbered questions they had about the course material. Questions
about statistics were the most frequent, followed by questions about the
assignments and one question about APA format. According to Douglass and
Morris (2014, p. 16) talking with professors to identify gaps in learning
facilitates student self-directed learning. The content of students journals
showed that the journals indeed served as a means of communicating with the
instructor about where their perceived knowledge needed improvement.
Another positive use of the journal was students reflecting on their own learning
experiences. In some cases, students had been anticipating having a difficult
time with statistics, yet were surprised that they were learning effectively and
pleased with their own progress in the course. These statements likely reflected
their intrinsic motivation to succeed in the course. One student regularly used
the journal as a means to express her motivation-related feelings; sometimes

131
they were positive and sometimes negative, but the instructor used her writings
as an opportunity to encourage her perceived self-efficacy. As Herman (2012)
states, students intrinsic motivation can improve when they feel their
instructors believe in their abilities.
Aside from reflecting on their own learning, the journals may have also
provided another means for students to communicate with the instructor. While
students were encouraged to express themselves in class, and visit and email the
instructor for assistance during the course, the journal provided an additional,
perhaps less formal means of communication between the students and the
instructor. The instructor noticed that some students who did not vocalize
during class did write journal entries; in accordance with Moon (1999), perhaps
the journal served as a more comfortable format in which communicate.
This study was exploratory in nature and thus it only investigated two different
sections of a graduate-level statistics course. A larger sample size may have
produced more powerful hypothesis testing. With only 14 participants in the
control section and 13 in the experimental section, the study could have
benefited from larger sample sizes, but may still provide a glimpse into the
advantages of journaling in statistics coursework. Collecting more data from
future sections could increase the ability to detect achievement benefits as well
as trends in how the students are choosing to use the journals.
In conclusion, the practice of keeping a journal may have been a positive

influence on student performance in graduate-level statistics coursework. It
may have improved some aspects of student performance in terms of grades,
and also served as an effective and comfortable venue for students to
communicate with the instructor and receive positive feedback on their learning.
Keeping reflective journals in statistics coursework is a practice that is worthy of
continued exploration. Many online LMSs currently provide journal platforms
that can be employed for documenting students reflections on their coursework
in statistics and other quantitative methods. Future work can address the
impact of reflective journaling using a larger sample size as well as possibly
reminding students about participating in the journal multiple times throughout
the semester in order to increase the number of entries made.
References
Bagley, T. & Gallenberger, C. (1992). Assessing students dispositions: Using journals to
improve student performance. Mathematics Teacher, 85(8), 660-663. Retrieved
from
http://webapp1.dlib.indiana.edu/virtual_disk_library/index.cgi/4273355/FID
1736/journals/enc2205/2205.htm
Barker, W., Breesoud, D., Epp, S., Ganter, S., Haver, B., & Pollatsek, H. (2004).
Undergraduate programs and courses in the mathematical sciences: CUPM curriculum
guide 2004. Washington, DC: Mathematical Association of America.
Bazerman, C., Little, J., Bethel, L., Chavkin, T., Fouquette, D., & Garufis, J. (2005).
Reference guide to writing across the curriculum. West Lafayette, IN: Parlor Press.
Retrieved from http://wac.colostate.edu/books/bazerman_wac/wac.pdf
Borasi, R. & Rose, B.J. (1989). Journal writing and mathematics instruction. Educational
Studies in Mathematics, 20(4), 347 365. doi: 10.1007/BF00315606

132
Britton, J.N., Burgess, T., Martin, N., McLeod, A., & Rosen, H. (1975). Development of
writing abilities: Ages 11 18. New York: Macmillan.
Boud, D. (2001). Using journal writing to enhance reflective practice. In L.M. English &
M.A. Gillen (Eds.), Promoting journal writing in adult education (pp. 9-18). San
Francisco: Jossey-Bass.
Boud, D., Keogh, R., & Walker, D. (1985). Promoting reflection in learning: A model. In
D. Boud, R. Keogh, & D. Walker (Eds.), Reflection: Turning experience into learning
(pp. 18-40). New York: Nichols.
Chiou, C.C., Wang, Y.M., & Lee, L.T. (2014). Reducing statistics anxiety and enhancing
statistics learning achievement: Effectiveness of a one-minute strategy.
Psychological Reports, 115(1), 297-310. doi:10.2466/11.04.PR0.115c12z3
Cook, A. (2010). Improving the success rate in statistics (No. 415). Brisbane: University of
Queensland, School of Economics. Retrieved from
http://www.uq.edu.au/economics/abstract/415.pdf
Costa, P. T. & McCrae, R. R. (1992). NEO PI-R Professional Manual, Odessa, FL:
Psychological Assessment Manuals.
Cruise, R.J., Cash, W., & Bolton, D.L. (1985, August). Development and validation of an
instrument to measure statistical anxiety. Paper presented at the Annual Meeting
of the American Statistical Association, Las Vegas, NV.
Devaney, T.A. (2016). Confirmatory factor analysis of the Statistical Anxiety Rating Scale
with online graduate students. Psychological Reports 118(2), 565-586. doi:
10.1177/0033294116644093
Douglass, C. & Morris, S.R. (2014). Student perspectives on self-directed learning.
Journal of the Scholarship of Teaching and Learning, 14(1), 13-25. doi:
josotl.v14i1.3202
Dunn, K. (2014). Why wait? The influence of academic self-regulation, intrinsic
motivation, and statistics anxiety on procrastination in online studies. Innovative
Higher Education, 39, 33 44. doi:10.1007/s10755-013-9256-1
Eysenck, M. W. & Derakshan, N. (1997). Cognitive biases for future negative events as a
function of trait anxiety and social desirability. Personality and Individual
Differences, 22, 597-605.
Flavell, J.H. (1978). Metacognitive development. In J.M. Scandura & C.J. Brainerd (Eds.),
Structural/process theories of complex human behavior. The Netherlands: Sijthoff &
Noordoff.
Flavell, J.H. (1976). Metacognitive aspects of problem solving. In L.B. Resnick (Ed.), The
nature of intelligence. Hillsdale NJ: Lawrence Erlbaum Associates.
Fullerton, J.A., & Kendrick, A. (2013). The math problem: Advertising students
attitudes towards statistics. Journal & Mass Communication Educator, 68(2), 134-
149. doi: 10.1177/1077695813478181
Furnham, A., & Chamorro-Premuzic, T. (2004). Personality and intelligence as
predictors of statistics examination grades. Personality and Individual Differences,
37(5), 943-955.
Guiffrida, D.A., Lynch, M.F., Wall, A.F., & Abel, D.S. (2013). Do reasons for attending
college affect academic outcomes? A test of a motivational model from a self-
determination theory perspective. Journal of College Student Development, 54, 121-
139. doi: 10.1353/csd.2013.0019
Hannigan, A., Hegarty, A.C., McGrath, D. (2014). Attitudes towards statistics of
graduate entry medical students: The role of prior learning experiences. BioMed
Central Medical Education. doi: 10.1186/1472-6920-14-70
Herman, G.L. (2012). Designing contributing student pedagogies to promote students
intrinsic motivation to learn. Computer Science Education, 22, 369-388.
doi: 10.1080/08993408.2012.727711

133
Johnson, M., & Kuennen, E. (2006). Basic math skills and performance in an introductory
statistics course. Journal of Statistics Education, 14(2).
Retrieved from ww2.amstat.org/publications/jse/v14n2/johnson.html
Kenny, R., Shoffner, M., & Norris, D. (2014). Reflecting on the use of writing to promote
mathematical learning: An examination of preservice mathematics teachers
perspectives. The Teacher Educator, 49(1), 28 - 43. doi:
10.1080/08878730.2013.848002
Loud, B.J. (1999). Effects of journal writing on attitudes, beliefs, and achievement of students in
a college mathematics course (Doctoral dissertation). Retrieved from ProQuest
Dissertations Publishing. (Accession No. 9923962)
Macher, D., Paechter, M., Papousek, I., & Ruggeri, K. (2011). Statistics anxiety, trait
anxiety, learning behavior, and performance. European Journal of Psychology of
Education, 27(4), 483-498. doi: 10.1007/s10212-011-0090-5
Macher, D., Paechter, M., Papousek, I., Ruggeri, K., Freudenthaler, H., & Arendasy, M.
(2013). Statistics anxiety, state anxiety during an examination, and academic
achievement. British Journal of Educational Psychology, 83, 535-549. doi:
10.1111/j.2044-8279.2012.02081.x
Marsh, H.W., & Martin, A.J. (2011). Academic self-concept and academic achievement:
Relations and causal ordering. British Journal of Educational Psychology, 81(1), 59-
77. doi: 10.1348/000709910X503501
Meel, D.E. (1999). Email dialogue journals in a college calculus classroom: A look at the
implementation and benefits. The Journal of Computers in Mathematics and Science
Teaching, 18(4), 387 413.
Mji, A., & Onwuegbuzie, A.J. (2004). Evidence of score reliability and validity of the
statistical anxiety rating scale among technikon students in South Africa.
Measurement and Evaluation in Counseling and Development, 36, 238251.
Moon, J. (1999). Reflection in learning and professional development. London: Kogan.
Murtonen, M. & Lehtinen, E. (2003). Difficulties experienced by education and sociology
students in quantitative methods courses. Studies in Higher Education, 28, 171-
185. doi:10.1080/0307507032000058064
Onwuegbuzie, A.J., (1997). Writing a research proposal: The role of library anxiety,
statistics anxiety, and composition anxiety. Library & Information Science
Research, 19, 5 - 33.
Onwuegbuzie, A.J. (2003). Modeling statistics achievement among graduate students.
Educational and Psychological Measurement, 63, 1020-1038.
Onwuegbuzie, A.J., & Wilson, V.A. (2003). Statistics anxiety: Nature, etiology,
antecedents, effects, and treatments a comprehensive review of the literature.
Teaching in Higher Education, 8(2), 195-209. doi: 10.1080/1356251032000052447
Pan, W., & Tang, M. (2004). Examining the effectiveness of innovative instructional
methods on reducing statistics anxiety for graduate students in the social
sciences. Journal of Instructional Psychology, 31, 149-159. Retrieved from
http://search.proquest.com.ezproxy.libproxy.db.erau.edu/docview/213906873
?accountid=27203
Papousek, I., Ruggeri, K., Macher, D., Paechter, M., Heene, M., Weiss, E., & Freudenthaler, H.
(2012). Psychometric evaluation and experimental validation of the statistics anxiety
rating scale. Journal of Personality Assessment, 94(1), 8291.
doi:10.1080/00223891.2011.627959
Paris, S.G., & Winograd, P. (1990). Promoting metacognition and motivation of
exceptional children. Remedial and Special Education, 11(6), 7-15.
Ramirez, C., Schau, C., & Emmioglu, E. (2012). The importance of attitudes in statistics
education. Statistics Education Research Journal, 11(2), 57-75.
Ryan, R.M., & Deci, E.L. (2000). Self-determination theory and the facilitation of intrinsic
motivation, social development, and well-being. American Psychologist, 55, 68-78.
doi:10.1037/0003-066X.55.1.68

134
Schau, C., Millar, M., & Petocz, P. (2012). Research on attitudes towards statistics.
Statistics Education Research Journal, 11(2), 57-75.
Spielberger, C.D. (1972). Anxiety as an emotional state. In C.D. Spielberger (Ed.), Anxiety:
Current trends in theory and research. London: Academic Press.
Teman, E.D. (2013). A rasch analysis of the statistical anxiety rating scale. Journal of
Applied Measurement, 14, 414434.
Viswanathan, M. (1993). Measurement of individual differences in preference for
numerical information. Journal of Applied Psychology, 78(5), 741-752. doi:
10.1037/0021-9010.78.5.741
Williams, M., Payne, G., Hodgkinson, L., & Poade, D. (2008). Does British sociology
count? Sociology students attitudes toward quantitative methods. Sociolgy, 42,
1003 1021.
Zeidner, M. (1991). Statistics and mathematics anxiety in social science studentssome
interesting parallels. British Journal of Educational Psychology, 61, 319328.

135

Vol. 16, No. 1, pp. 135-160, January 2017
Competence and/or Performance - Assessment

and Entrepreneurial Teaching and Learning in
Two Swedish Lower Secondary Schools
Monika Diehl and Tord Gran Olovsson
Ume University
Ume, Sweden
Abstract. Entrepreneurial teaching and learning, thus entrepreneurial

education corresponds well with formative assessment/assessment for
learning. Both are characterised by an approach to education, teaching,
and learning, which puts pupils in the centre of their own learning.
Learning aims to go deep and generate real learning where
competencies rather than measurable results are the focus. Both
entrepreneurial education, and assessment for learning are promoted by
the Swedish National Agency for Education. Entrepreneurial education
has been inscribed in the national curriculum for Swedish compulsory
schools since 2011. The same curriculum and syllabuses also focus on
several knowledge requirements, which form the basis for assessing
pupils performances. Thus, the Swedish national curriculum can be
said to send two rather disparate messages. This research focuses on
lower secondary school and the broad approach of entrepreneurial
education and uses Basil Bernsteins theory of performance and
competence models to elaborate on entrepreneurial teaching and
learning in relation to assessment. Observations along with interviews
with teachers and pupils in two Swedish lower secondary schools
provide the empirical basis for the research. The results reveal some
differences between the schools but indicate that both teachers and
pupils are relating to the prevailing dominance of performance models
and thus encounter difficulties when trying to adopt entrepreneurial
education and assessment for learning.
Keywords: entrepreneurial education; assessment for learning; lower

secondary school; performance model; competence model.
Introduction
Entrepreneurial education (EE) aims to develop an entrepreneurial mind-set
among pupils and, thus, the future labour force (European Commission, 2002;
2006; 2010; 2012; 2013; 2016; OECD, 1989; 1998; Swedish Government Office,
2009). This means that pupils abilities such as creativity, curiosity, and problem-
solving are to be encouraged. Aspects such as interdisciplinary collaboration,
involvement with the surrounding society as well as pupils participation,
influence, activity, and experiences are advocated, thus entrepreneurial teaching
and learning is to be perceived as an educational approach (Erkkil, 2000; Kyr,

136
2005; Lackus, 2015; Re degrd, 2014). The need for EE has been expressed in
national and international policy text for decades; in 2011, it was inscribed in the
Swedish national curriculum (SNAE, 2011a).
In Sweden, there has also been an increased focus on assessment for learning
(AfL), (also referred to as formative assessment)1, where assessment is used as a
tool in the ongoing learning process. The state has encouraged implementation
of AfL in Swedish classrooms through some of its policies (e.g. SNAE, 2011b).
The motive behind this is to support pupils' learning and development. Among
others, Stobart (2008) claims that AfL is a process, but also an approach to
learning and teaching. It is obvious that EE can be easily linked to AfL. In
literature on entrepreneurial teaching and learning, AfL is advocated and seen
as a part of its means (Falk-Lundqvist, Hallberg, Leffler, & Svedberg, 2011; 2014;
Josefsson, Bostani, & Josefsson, 2009). In parallel with this, there is also an
increased summative focus in Swedish compulsory schools (Wahlstrm &
Sundberg, 2015). Summative results usually are considered as what counts
concerning pupils future possibilities, and, to a large extent, the focus on
summative results controls the mediation of knowledge (Stobart, 2008). Thus,
the message to teachers and pupils is somewhat contradictory as summative
assessment can be perceived as requiring other types of classroom practices and
ultimately other forms of knowledge rather than those promoted by EE and AfL.
Basil Bernstein (2000) uses the concepts of performance models and competence
models to distinguish between the knowledge forms that different pedagogic
practices generate. This study is a part of a continuing professional development
(CPD) programme on EE, including 25 schools across Sweden. It took place
between 2012 and 2015 and its goal was to combine school improvement and
research. The aim was to provide a better understanding of EE, further develop
it, and encourage its adoption. The present study involves two lower secondary
schools taking part in the programme. The study uses the concepts of the
performance and competence models to elaborate on teachers and pupils
understanding of knowledge, teaching, and learning in relation to EE and AfL.
There is a lack of empirical research on EE in relation to assessment in
compulsory schools. Furthermore, in the current research on AfL, there seems to
be a lack of thorough studies using direct examples from classroom observations
(Black, 2015). Therefore, this study aims to fill this gap.
Aim and research questions

The aim of this study is to elaborate on how different modes of teaching and
learning are associated to different forms of knowledge and assessment, and
1 In some research, the concepts of assessment for learning and formative

assessment are given somewhat different meanings (e.g., Bennett, 2011; Swaffield,
2011). In this article, the concepts are used synonymously, which is also common in
other research (Swaffield, 2011). Both concepts are used in the article: In most sections
assessment for learning is mainly used, but in the Results section, formative assessment
is used.

137
how this relates to entrepreneurial education in Swedish lower secondary

schools. The following research questions will guide the study:
How is teaching conducted and questions about knowledge and
assessment expressed in practice?
How do teachers express themselves about teaching, knowledge, and
assessment in relation to entrepreneurial learning?
How do pupils express themselves about and relate to teaching, learning,
and assessment?
Background
In this section, we place entrepreneurial education in an international and national
(Swedish) context. The Swedish assessment system in compulsory school will be
presented, followed by the means of summative assessment and assessment for
learning. Views on entrepreneurial learning in relation to assessment will conclude
this part of the article.
Entrepreneurial education
International and national policy documents have expressed the need for EE for
a long time (European Commission, 2002; 2006; 2010; 2012; 2013; 2016; OECD,
1989; 1998; Swedish Government Office, 2009). Globalisation and marketization
are the main incentives for introducing EE from an early age. In Sweden, this is
a thread throughout the educational system (Swedish Government Office, 2009).
EE is often delineated by a narrow and a broad approach. The narrow approach
is connected to teaching and learning about how to plan, start, and run
businesses. The broad approach, most commonly called entrepreneurial
learning, is about usurping an entrepreneurial mind-set (e.g. Erkkil, 2000; Jones
& Iredale, 2010; Leffler, 2006; Lindster & Norberg, 2016; Mahieu, 2006). In
Sweden, the narrow approach is primarily a question for upper secondary
school and higher education, whereas the broad approach permeates the entire
educational system. Education should provide the conditions for pupils to
develop an entrepreneurial mind-set, as the Swedish curriculum (SNAE, 2011a,
Lgr 11) says:
An important task for the school is to provide a general but

coherent view. The school should stimulate pupils creativity,
curiosity and self-confidence as well as their desire to explore their own
ideas and solve problems. Pupils should have the opportunity to take
initiative and responsibility, and to develop their ability to work both
independently and together with others. The school in doing this
should contribute to pupils developing attitudes that promote
entrepreneurship. (p.11, italics added).
In Swedish national syllabuses, the pupils abilities to analyse, reflect,

communicate, argue, and evaluate can be connected to entrepreneurial teaching
and learning.

138
Assessment in Swedish compulsory school

In Swedish compulsory school, a criterion-referenced assessment system has
been operating since the mid-1990s (Wikstrm, 2006). This system requires that
student performance is assessed in relation to knowledge requirements (criteria)
in each subject. Teachers assess their own pupils performance also regarding
summative assessment, which is the basis for selection for further studies.
Several assessment reforms have been implemented in recent years with
consequences for classroom assessment. Therefore, there is a greater emphasis
on assessment, particularly since the revised national curriculum Lgr 11 (SNAE,
2011a) was introduced in 2011. The new curriculum included new syllabuses in
all subjects, clearer requirements for what students should achieve, a new
grading scale with more levels, end-of-term grading in Grades 6 and 7 (instead
of starting in Grade 8, which was formerly the case), and national tests in more
subjects starting at earlier grades. The implementation of these reforms brought
about a greater focus on summative assessment (Wahlstrm & Sundberg, 2015)
in classroom practice, with teachers more carefully using the national steering
documents in teaching and assessment. The main purposes of the assessment
reforms were to improve student performance and school effectiveness through
better monitoring of pupils progress (SOU, 2013: 30) and to detect pupils
knowledge gaps at an early stage (Samuelsson, 2012).
Summative assessment
Summative assessment (assessment of learning) is usually described as a
summarising of learning after teaching is completed. For a number of years,
there has been an international trend towards more summative assessment for
accountability, which involves external (political, administrative) requirements
that are easily measured and provide clear results (Lundahl, Hultn, Klapp, &
Mickwitz, 2015; Stobart, 2008; Torrance, 2011;). This may cause tensions between
the kind of assessment that teachers perceive as meaningful and assessment
concerning accountability (Black, 2015; Gardner, 2010; Suurtam & Koch, 2014).
Public announcement of schools results based on students achievements in
terms of test scores and grade levels are now common and may counteract the
use of formative strategies in classroom practice (Brookhart, 2009; Natriello,
2009). Assessment methods commonly attributed to summative assessment are
standardised tests (in Swedish compulsory school: national tests) and grades.
The concept of high-stakes testing (e.g. Au, 2007) is usually used to describe
tests with major consequences for students. The national tests in Grade 9, in
lower secondary school in Swedish compulsory school, is generally perceived by
students as being high-stakes (Eklf & Nyroos, 2013). The national test scores
may have implications for final grades in Grade 9, which, in turn, give students
access to further studies in upper secondary school. National tests in Swedish
compulsory school are provided in Grades 3, 6, and 9.
Test scores and grades are often criticised for not contributing to real learning
(Stobart, 2008; Jnsson, 2013; Lundahl et al.,2015). Previous research on
summative assessment (e.g. Eklf, 2017) shows that a focus on test scores and
grades detracts from actual learning, which implies increased negative stress for
both pupils and teachers. One downside of standardised tests (Hkansson &

139
Sundberg, 2012) is that they limit teaching content and drive teaching to include
only what is coming on the test, a practice which is called teaching to the test
(Mehrens & Kaminski, 1989). However, summative assessments themselves do
not necessarily prohibit good teaching, rather it is how the summative results
are used. Good teaching and assessment involves assessment for learning and
assessment of learning, in interaction (Black, 2015; Olovsson, 2017; Tveit, 2014)
Assessment for Learning

Black and Wiliam (1998) argued that formative classroom assessment can
improve student performance. Their argumentation was built upon their idea
that the main purpose of assessment in education is to support pupils learning.
Wiliam and Thompson (2007) elaborated a framework for AfL using three key
processes of teaching and learning. The processes involve determining goals for
learners, what the learner already knows or can do, and what is required to get
the learner to the goal. In this process, feedback is the key element for effective
learning, and constructive, forward-looking feedback is what enhances learning
most effectively (Hattie & Timperley, 2007). However, criticism of the positive
impact of AfL has emerged. The criticism e.g. concerns a perceived instrumental
approach to teaching (Bennett, 20011; Torrance, 2007).
If feedback and instructions from teachers to students is too detailed, there is a

risk that students may become too dependent on others in their learning process
and take a more passive role (Groome, 2005; Jnsson, Lundahl, & Holmgren,
2015; Torrance, 2007; Torrance, 2012). In this context, Torrance (2007) argues that
criteria compliance replaces learning. If AfL becomes too teacher-centred and
aimed at supporting pupils to achieve a certain grade, it might lose a significant
part of its value: At the heart of it is a vision of active and self-regulating
learners who work to make sense of what they are learning . Peer- and self-
assessment play a key role in this self-regulation, as does feedback (Stobart,
2008, p. 169). In conjunction with this, Stobart discusses the spirit of AfL (see also
Marshall & Drummond, 2006), which implies a clear understanding of why
teaching is organised in certain ways. It is not just about formative techniques,
used without reflection in the classroom, a practice referred to as being the
letter of AfL (Marshall & Drummond, 2006; Stobart, 2008).
Entrepreneurial education and assessment

Literature and research on the broad approach of EE advocates using AfL as it is
and sees it as being a natural part of EE, and aligned with its means (Falk-
Lundqvist, Hallberg, Leffler, & Svedberg, 2011; 2014; Josefsson, Bostani, &
Josefsson, 2009; Lackus & Moberg, 2013; Lundahl, 2011). The connections
between EE and AfL are obvious in many ways. Primarily, they both concern the
recognition that learning is not about being able to reproduce what is already
known but is increasingly about creating new knowledge relevant to solve
problems (e.g. Gibb, 2000; Johannisson, Amundsson, & Kivimki, 2010; Lundahl,
2011; SNAE, 2015). EE is often linked to Dewey and progressivism, where the
learning process rather than measurable knowledge outcomes is the focus (e.g.
Re degrd, 2014), and the same applies for AfL (Lundahl, 2011). EE and AfL
are approaches based on pupils knowledge level; they represent a different

140
view of teaching and learning than what is seen as a traditional approach

(Erkkil, 2000; Gibb, 1993; Kyr, 2005; Lackus, 2015; Lundahl, 2011; Stobart,
2008). Nonetheless, not much research on assessment of EE in compulsory
school has been done. The problem of assessing complex abilities such as
creativity, fantasy, innovation, critical thinking, and problem-solving are
addressed by researchers within the fields of AfL and EE (Diehl, Lindgren &
Leffler, 2015; Diehl, 2016a; Diehl 2016b; Lundahl, 2011). Lackus, Lundqvist and
Middleton (2015) and Lackus (2016) offer suggestions based on a smartphone
app where students are able to report and reflect on critical learning events as
they happen. Other than that, Swedish literature on entrepreneurial learning
suggests different ways for teachers to conduct AfL by providing examples of
classroom situations and reflecting on them (Falk-Lundqvist, Hallberg, Leffler, &
Svedberg, 2011; 2014) or by offering examples of different scoring rubrics
(Josefsson Bostani & Josefsson, 2009).
Theoretical Framework
This study aims to investigate what kind of knowledge is focused on in schools
and classes taking part in a CPD programme on entrepreneurial learning.
Contradictions in curriculum and syllabuses and their implications on teaching
and learning have been described above. A way to explain the different
knowledge forms that emerge is to use Bernsteins (2000) pedagogic models,
performance and competence. These in turn have an impact on what kind of
consciousness and identities are shaped among teachers and pupils. Our
identities, or our sense of who we are, is realised or brought about through what
Bernstein calls three message systems: curriculum, pedagogy and evaluation
(South African Institute for Distance Education, SAID, 2010). Curriculum
concerns what is defined as legitimate knowledge (selection), pedagogy
concerns the form or mode used for transmission (how the selection is taught),
and evaluation concerns how knowledge is tested or assessed. Evaluation, thus
assessment, is dependent on curriculum and pedagogy (SAID, 2010).
Bernstein (1975) distinguishes between two different types of curriculum, one

working according to a collection code and one according to an integrated code.
In Sweden, as in many other countries, the national curriculum is a merging of
both to some degree and thus has many contradictory aspects. The introduction
of EE in many ways presupposes an integrated code, but is contradicted by very
strict knowledge requirements and required learning outcomes, which to a high
degree presupposes schoolwork according to a collection code. The codes in turn
are characterised by strong or weak classification and framing. Strong
classification means that each category (in this case subjects) has a unique
identity, voice, and rules for internal relationships. Weak classification indicates
less specialised discourses, voices, and identities. Framing concerns the
regulation of communication and interaction in different categories. It is about
the internal logic of a pedagogical practice and describes the nature of control
over sequencing and pacing, selection of communication, expected assimilation,
learning criteria, and the social context in a classroom. Framing, like
classification, can be stronger or weaker. Strong classification and strong

141
framing denotes a pedagogical practice characterised by a collection code, weak

classification and framing, thus an integrated code (Bernstein 1975). The
different codes consecutively generate contrasting pedagogic models, which
influence tests and evaluation/assessment.
The performance model

The performance model is characterised by strong classification.
Briefly, a performance model of pedagogic practice and context

places the emphasis upon a specific output of the acquirer, upon a
specific text the acquirer is expected to construct and upon the
specialised skills necessary to the production of this specific
output, text or product. (Bernstein, 2000, p. 44)
The pedagogic discourse thus is based on the specialisation of subjects, the rules
for legitimate texts are explicit, and pupils performances are graded. An
important part of teachers professionalism is to be explicit and to know the
grading procedures, and when pupils work is evaluated (assessed), the focus is
on what is missing. Accountability is facilitated by the objectivity of the
performance and thus outputs can be measured and optimised (Bernstein 2000,
p. 50). The grading system implies a potential repair service, and pupils are to
blame if they fail. Teachers essentially have control over selection, sequencing,
and pace, and the space for the pedagogic practice is clearly marked and
explicitly regulated. The modes of instruction imply disciplining regulation
where deviances are very visible and not much time is spent or needed on
personalised modes of control. In such pedagogic practice, the acquirer (pupil)
invisibly is positioned in the past. Thus, teaching is constructed on the past but it
is made visible that they learn for the future. Bernstein (2000) distinguishes
between introverted and extroverted modalities. In the case of introverted
modalities the future is the exploration of a specialised discourse itself as an
autonomous activity (p. 49) which allows for more autonomy for teachers and
learners within curriculum regulation. In the case of extroverted modalities the
future is likely to become dependent on external regulation, for example, the
economy or local markets (p.49), resulting in much less autonomy. The
performance model entails rather low costs due to the explicit structures of
transmission and of its progression (p. 50).
The competence model

The competence model is characterised by weak classification and in
competence theories there is an in-built procedural democracy, an in-built
creativity, an in-built virtuous self-regulation. And if it is not in-built, the
procedures arise out of, and contribute to a social practice, with a creative
potential (Bernstein, 2000, p. 43).
The pedagogic discourse bases its transmission on competencies that pupils

already possess or are thought to possess. It is group based, and pupils appear to
have a great deal of control over selection, sequence, and space. The pedagogy is
largely based on projects, themes, and experiences. Pupils have access to, and

142
considerable control over, the construction of many different spaces. Different

activities are not punctuated, and progression is implicit to the pupils. Pupils
experience the pedagogic practice in the present tense, and teachers understand
or read the pupils competence development in their products to be able to
facilitate further challenges. The consequence is that the meaning of an
acquirers sign is not available to the acquirer, only to the teacher (Bernstein,
2000, p. 48). Teachers focus on and discuss pupils products based on what is
present, and criteria for evaluation (assessment) are implicit and diffuse. Even if
positional and imperative control of pupils occurs, focus is on self-regulation
and individual communication with each pupil about intentions, dispositions,
and relationships, based on pupils own reflexivity. Competence models require
a wide range of autonomy, but there still needs to be a homogeneity in practice,
which reduces each teacher autonomy. When it comes to teaching material,
packages and textbooks are seldom used. Instead teachers construct their own,
which requires autonomy. Pupils products are difficult to evaluate objectively,
and the models are less amenable to public scrutiny and accountability.
Competence models are not connected to specialised futures. The costs for
competence models are rather high, as there are no time limits for pupils
learning and hidden costs in terms of time needed for teachers commitment,
planning, and socialising with parents are usual. Other than that, feedback to
pupils and constructing learning material is time consuming. The
() lack of recognition of hidden costs may lead to ineffective

pedagogic practice because of the demands of the practice, or, if
these are met, the lack of recognition may give rise to
ineffectiveness because of the fatigue of the teachers. (Bernstein,
2000, p. 49)
There are different modes of performance models and different modes of

competence models but also merges between them in pedagogic practices.
Bernstein (2000) argues that performance modes are normal across all levels of
education, and thus competency modes may imply an interruption or
resistance to this normality or may be appropriated by official education for
specific and local purposes (p.51). When the state introduced the broad
approach of entrepreneurial education into the Swedish national curriculum,
this constituted what can be seen as an appropriation for specific purposes.
Before looking into its realisation in practice and the assessment challenges
involved, a description of this studies methodology will be presented.
Methodology
Sample
Six of the schools taking part in the CPD program were lower secondary schools.
Out of these, two were chosen for data gathering due to their geographic
similarities (both situated in suburbs of major Swedish cities) and differences in
pupil demographics. Almost all the students in School 1 have a Swedish

143
background, and 1/3 of the students in School 2 have backgrounds other than
Swedish.
School 1 is situated in a wealthy middle-class area where housing mainly

consists of villas. It has about 150 pupils (grades 79, ages 1316) at the time of
data gathering. At the time of data gathering, some subjects where merged on
occasion, but the school work was primarily conducted within subject-specific
classes. School 2 is situated in a suburban community with mixed housing. It has
around 300 pupils (grades 69, ages 1216). The work was organised by subject.
Data collection
The data collection was guided by Bernsteins (1975; 2000) theoretical
frameworks. It included classroom observations and interviews with teachers
and pupils and went on for about three weeks at each school. The aim was to
interview the teachers whose lessons were being observed. The idea was to be
able to link the interviews to what had been observed, thus the observations
were conducted first.
All in all, 52 lessons were observed: 21 observations in School 1 and 31 in School

2. A structured observation schedule (Cohen, Manion, & Morrison, 2011) with
fixed categories was created. The schedule was based on Bernsteinian categories
to distinguish the degrees of teachers and pupils control of communication,
criteria, time, pace, sequencing, and order in the classroom. In addition, the
observation schedule included Swedish (and international) policy documents
(European Commission, 2002, 2006, 2010; Government Office of Sweden, 2009;
OECD, 1989, 1998), and the Swedish national curriculum and syllabuses, which
describe the required abilities for entrepreneurial education (SNAE, 2011a).
Additional field notes were made to absorb the situation and to describe events
and behaviours not covered by the structured part of the schedule. Situations,
actions, and expressions connected to assessment made up a considerable
amount of the field notes. Overall, the method can be described as a form of
participant observation (Cohen et al., 2011; Kvale, 1997).
The interviews were semi-structured and used prepared questions that matched
the observation protocol as well as questions on assessment, particularly
formative assessment, and its relationship to entrepreneurial teaching and
learning. The questions allowed scope for open-ended answers (Hannan, 2007).
The interview guide was changed to some extent; questions were added and
reformulated based on what had been observed during the time spent at the
schools. Altogether, eight teachers were interviewed, three in School 1 and five
in School 2, and the interviews lasted for 4090 minutes. Pupils from all grades
were interviewed in both schools, resulting in 14 group interviews and one
single interview. Nine interviews were conducted in School 1, and six in School
2. Altogether, 38 pupils took part in the interviews. The interviews lasted 3050
minutes. All interviews were recorded.

144
Data analysis
Verbatim transcriptions were made to help the researchers remember the details
in the context and situations in which the observations and interviews took place
(Kvale, 1997). The field notes were transcribed and categorised. For purposes of
this study, we focused on observations connected to assessment. The data was
analysed by school in order to identify similarities and differences between
them. The interviews were transcribed verbatim. The material was reviewed
with the research question in mind, and general patterns, regardless of school,
were searched for (generating natural units of meaning). The next step was to
discern pupils and teachers statements about assessment practices. This meant
organising the data into feasible and adequate categories (classifying,
categorising, and ordering the units of meaning). A search for similarities and
differences in the statements identified differences within and between the
schools, allowing us to formulate new variables and recognise sub-categories
and themes. Based on the concepts of entrepreneurial teaching and learning and
assessment, a text was formed and organised, structuring narratives to describe
the interview contents. Finally, the interviews were interpreted to create
meaning, together with the field notes (Cohen et al., 2011; Watt & Boolsen, 2007).
Ethical considerations
The research was part of a CPD programme, which the participants were aware
of. Still, all respondents had to be willing to be interviewed, and the teachers had
to be willing to be observed. On the occasion of the first observation in each
class, the researcher presented the reason for being in the classroom. Before
interviewing willing learners below 15 years of age, parental consent was
obtained by having them sign a paper with information about the research. All
respondents were informed that they could choose to end the interview at any
time (Swedish Research Council, 2002). The names and locations of the schools
and the names of the participants were anonymised. Due to the small amount of
teacher interviews, they are not specifically presented after the citations in the
Results section of the article. Gender and grade is presented regarding the
pupils, as they were many more of them and the information is seen as
important for understanding their statements.
Results
At the time of data gathering, both schools were engaged in the implementation
of the national tests. This meant that a lot of time was spent preparing and
conducting the tests. The first part of the results will start by presenting a
general picture of the teaching culture in the observed classrooms. This will be
followed by a description of classroom work and assessment when the schools
are preparing for the national tests. Along with this, other observations related
to assessment will be presented. After that, the teachers discussions of teaching,
testing, formative assessment, and entrepreneurial learning will be presented.
Finally, the pupils opinions about learning, tests, and assessment will be heard.
Entrepreneurial teaching, learning, and assessment in practice

School 1
On a general level, many of the observed lessons were characterised by a high
degree of creativity. Pupils were given the opportunity to work together, find

145
their own solutions, and at times work in an interdisciplinary manner. Often, but
not always, the pupils seemed to be aware of what they are supposed to learn.
Although many lessons were characterised by teachers urging the pupils to
reflect and think on their own, that is, teaching was directed towards more
complex learning goals, sometimes the right answers were given directly and
the pupils were told to do things in a certain manner to get it right.
Preparations for the national tests were usually a part of the classroom lessons.
One example is that part of lesson in Social Science was devoted to discussion
and practice tests from a previous year. The teacher urged the pupils to pay
special attention to the questions that required reasoning, reflection, and
analysis and told the class that they would later look at examples of how these
questions are assessed. On one occasion, one of the pupils said that he was
nervous about the upcoming tests, and the teacher said that he was too.
Regarding other observations related to assessment practice, elements of

formative assessment occurred quite frequently. One example is from a lesson in
Physics and Technology (the subjects are merged). During the lesson, the teacher
give support and feedback on what the pupils were working on at the moment
and frequently walked around the classroom, talking to individual pupils about
their results on former tasks and tests. The teacher told the pupils that they need
to be more analytic and that they have to develop their reasoning to get higher
grades. The teacher often used a scoring rubric, which helps her explain what
areas the pupils need to develop. As she said, The more the pupils see the
rubrics, the more used they will become to the way of thinking and can develop
their work.
In Social Science, the teacher encouraged the pupils to develop their language
and write complete sentences, to reason about advantages and disadvantages
regarding the issue they are writing about. In Home Economics, the pupils were
required to reflect on the processes they learned. In general, the teachers were
very encouraging towards the pupils. One example from the field notes is from a
Physics and Technology lesson when the teacher was facilitating a discussion
with pupils about their thought and different solutions. Following are some of
the teachers comments:
- This was really good, now you are close!
- What fun, then youve got a challenge today also!
- Lets try it this way!
Grading and assessment was explicitly discussed on many occasions during

everyday classroom work. The teachers often referred to upcoming tests when
they were teaching or commenting on pupils work. Pupils often asked how
their work is going to be assessed. One example is that some classes were
building a city in Mindcraft2 during a lesson in Geography, which was carried
out in smaller groups. One pupil asked the teacher how he was going to grade
their cities. He replied, How you have been thinking as a city planner, the
2 A computer programme that enables the user to build houses and cities.

146
thought behind it. If it looks good in Mindcraft or not does not give high grades.
Its how you have been thinking. Regarding behavioural matters, the teachers
repeatedly urged the pupils to put away their mobile phones or prodded them
to get started on their work. However, no teacher directly connected behaviour
with assessments and grades.
School 2
On a general level, the lessons in this school followed the same pattern. They
started out with an introduction, followed by the pupils working on tasks, often
answering fixed questions from the textbooks, while the teacher walked around
the classroom helping the ones asking for help. Often the end of the lesson
seemed to come suddenly as the teachers realised time was running out. It was
very seldom that pupils seemed to be aware of why they were working with
tasks, and what they actually learned. In handicrafts, a more apparent
entrepreneurial and formative approach in teaching could be observed: The
handicraft teacher worked according to a process-driven routine where the
pupils made a plan, decided how to execute the plan, and then evaluated their
work.
During many lessons, much time was spent preparing the pupils for the
upcoming national tests. In Mathematics for example, it was common for
teachers to hand out previous tests, and to urge the pupils to work out the
problems in them. The teachers checked whether the pupils were having
difficulty with assignments and spent time explaining to them individually or
for the whole class to hear. Teachers also spent a great deal of time explaining
how the previous tests were assessed and what the pupils have to consider
when taking the upcoming tests. The field notes reveal that The teacher urges
the pupils to notice how many points the different assignments generate, as it
gives them a hint about how much they need to answer and how advanced the
question is. The same teacher also informed the pupils that showing how they
think, the process, is more important than the actual answer.
A great deal of concern about the national tests can be heard among both
teachers and pupils. One pupil asked the teacher how the test results will affect
the grades. Another pupil asked if it is a good idea, as a preparation, to read all
the textbooks from lower secondary school in the subject being tested. One
teacher expressed concerns about the lack of time for preparation and revealed
that she has lately devoted more time to the subject that will be tested than to
other subjects she is teaching. The time issue is also obvious. Teachers often tell
the pupils to hurry up and to practice as much as possible during the lessons.
Along with this, the teachers are supportive of pupils learning and positive
towards the pupils: Are you doing fine? I think you are doing sooo fine, its
going to be fun to summarise your results later on.
On a general level, other observations of the assessment practice indicate a good

relationship between the teachers and the pupils. However, many of the teachers
reacted strongly when the pupils did not do as they were told, and the pupils
were often rebuked. They were often told to stay in their places and to be silent

147
and focused. Behavioural reprimands were often connected to grades and

assessment. One example noted when a boy is noisy: Stefan, now you have to
change places, you get high scores on the national test, and then you waste your
talent!
The teachers often referred to upcoming tests to motivate pupils to get started
and to work. If the class worked diligently and behaved well, they were
promised to be able to end the lesson a bit earlier than usual. Lesson attendance
was of utter importance and was related to grades and to behaviour.
Teachers comments of entrepreneurial teaching, learning, and

assessment
School 1
Relations between entrepreneurial teaching, learning, and assessment
One teacher said he was struggling to find a balance between the required
subject knowledge outcomes, which require a more teacher-led style of teaching,
and entrepreneurial teaching methods. He argued that pupils need a base of
knowledge to be able to understand, reason, and analyse. Entrepreneurial
education, on the other hand, he said, requires more interdisciplinary and
investigatory working methods where the pupils are active and have influence
over their learning. The challenges regarding these working methods are for
example: finding time for planning and enacting interdisciplinary work, getting
colleagues to join in on such working methods, and establishing fair assessment
of individual pupils. The teacher seemed stressed to ensure that all pupils
achieved the knowledge requirements. Even if he wanted to work more
entrepreneurially and intended to let the pupils have greater influence, the
national tests, the required learning outcomes, and fear of not being able to
provide fair grades, had a repressive influence on his teaching. Another teacher
confirmed that assessment controls the teaching to a great extent. She said that
she needs tests to be able to show parents why their children got certain grades,
and that grades are important to the pupils:
They are incentive for some. If one can show them, now you have
developed these competences, but if you want to get better, you
can do this and this. The grades clarify their learning, they get help
and can choose to work for a better grade.
Even this teacher expressed that the national tests are something that stresses
her, but thought that more interdisciplinary projects would provide an
opportunity to assess the pupils together with colleagues.
Teachers understanding of formative assessment

Two of the teachers said that formative assessment is about supporting pupils
learning. One said it is about the pupils ability to self-assess and to be able to set
their own goals. One teacher said:
I think it is difficult.I see formative assessment as sitting down

with the pupil and talk about their work and I dont really think I

148
have that time, but I can see that many need more of it. . You can
also view formative assessment as working digitally, that I give
comments on their work over the internetbut still I think that
meeting them, to sit down and talk to them is better.but that is
difficult with a whole class.
Even though this teacher saw supporting pupils during a lesson as a form of
formative assessment, he did not find the classroom situation a good
environment for this, because some pupils never ask for help. The teacher
considered different ways to achieve good formative assessment, but thought it
would be difficult to get it working.
Another teacher thought she developed her use of formative assessment in a

good way; she uses rubrics and evaluates each lesson together with the pupils.
That gives her good understanding about what the pupils have understood or
not, which does not make the results of the tests so crucial.
School 2
Relations between entrepreneurial teaching, learning, and assessment
One teacher argued that entrepreneurial learning is about motivating the pupils
to learn in a safe environment, for example, that pupils would dare to talk in the
classroom. She said that many pupils question the teaching content and
therefore she intends to motivate them, to persuade them that the knowledge is
important for them to be able to improve learning. The teacher suggested that
one way to motivate pupils is to be an enthusiastic teacher; she claimed the
enthusiasm would be reproduced in the classroom. The teacher gave an example
of a situation when she talked about an issue and it later turned up as a question
on the national test. She said, Its important how you mediate things. The
pupils may think that the issue will turn up as a question on the national tests.
This teacher thought she was teaching entrepreneurially in Technology and said
that the pupils are highly motivated during those lessons. The pupils were eager
to get good grades and even stayed after the lessons to finish their work. The
teacher tried having the pupils grade their own and others work.
Another teacher related entrepreneurial teaching to paying attention to every

pupil, engaging them in planning, and letting them have a say about when tests
should be conducted. Having good relations with the pupils is essential, she
said. She did not think that the national tests have a negative influence on
entrepreneurial learning but thought that the national testing system is wrong in
many aspects. First of all, they imply a lot of negative stress for the pupils.
Secondly, she thinks the tests signal a mistrust of teachers, she says that carrying
them out:
() underestimates us teachers. We already have grades (to assess

the pupils knowledge) one must trust that we know what we
do. I feel mistrusted, the tests are good but it feels like I am being
tested. The pupils want to have as good grades as possible, but
they dont even know what to practice.

149
.
Yet another teacher admitted they have focused very much on Physics this year,
as she thought it was going to be the science subject in the national test. Her way
of working entrepreneurially has been to engage the pupils in correcting their
own tests, which she found made the pupils committed and contributed to an
interest in finding out the right answers on failed questions. She also wanted to
improve in making pedagogic plans so that the pupils understand the
knowledge requirements, the means for progressing, and the reason for learning
a certain teaching content.
Teachers understanding of formative assessment

Two teachers connected formative assessment solely to how tests are assessed
and engaging pupils in correcting tests. One of these teachers said she always
tries to assess pupils working process and sometimes threatens pupils if, for
example, they talk too much, implying that they will be assessed for grading
based on their behaviour. She admitted that she assesses behaviour too, even if
she knows she should not. The handicraft teacher saw formative assessment as
guiding the pupils continuously. She also thought a lot about how formative
assessment could be enacted and its relationship to summative assessment. The
teacher initiated a working process where the pupils first consider what they
want to work with and how, then carry the project through and finally evaluate
their work. She discussed assessment in relation to this process, which means
that not only the final product could be assessed but also the idea and the
considerations about the projectin other words, the actual work, as well as the
pupils reflections about their work. She said she finds it very demanding to
grade and assess the implemented process. She regarded the described working
process to align with entrepreneurial learning, and as both formative and
summative assessment. The Social Science teacher meant that everything the
pupils do is to be assessed, to their advantage. One should assess when they do
good things and always look on their work positively. She said she tries to give
the pupils positive feedback, lead them in the right direction, talk to them and be
attentive to see if someone does not understand.
Pupils comments of teaching, learning, and assessment

School 1
The pupils expressed rather negative thoughts about studying for tests, which
they think has a negative influence on their learning:
I dont really like to read books and study for a test. Then I think it
feels like one just studies, studies, studies just crude study all
the time. Then you do the test, and the day after you have
forgotten it all. Sometimes it feels like you just do it because you
have to, not to learn sometimes it would be better to learn in
other ways like doing a project and a presentation or something
like that. Then, you learn more when you work together. (girl,
Grade 8)
Many pupils requested more practical work in school as they think they learn
better in that way. One boy (Grade 9) said, The pupils would learn more if

150
there was more practical work, (the knowledge) would settle more due to that
both the body and the brain work at the same time.
Regarding if the pupils were aware of the knowledge requirements, one pupil
(boy, Grade 8) said: Yes, we often get to take part of the knowledge
requirements. Its just that its not only those that count. Its also how one
behaves in lessons, how one hands in an assignment, how one has written the
assignment.
Many pupils felt negative stress regarding grades. This was most obvious
among 9th graders:
There is such a tremendous pressure now..we get homework in all

subjects, and then there are the national tests, and thus we have
tests in other subjects too. Like, this week, we had a national test in
Swedish and above that we had a test in Mathematics. (girl, Grade
9)
Overall, the amount of time required to keep up with other homework and
study for tests was experienced by pupils as stressful.
Not many pupils thought they had experienced formative assessment in the
sense of getting feedback during the semester about their learning progress.
Some said they never got feedback, and others said they experienced it on
occasion. Some pupils mention that the Science teacher has given them feedback
on their work and has pointed out what and how they can improve. The Science
teacher uses a scoring rubric, which makes the feedback clearer. Other teachers
give written feedback on tests and require pupils to save the tests in a binder.
The pupils can then review the comments before the next test to inform
themselves about what they need to improve. Yet other teachers, the pupils say,
just tell them that they should plod along but do not clarify what to do and how
to do it. He kind of doesnt say what I need to practice so I dont really know
what I should do or what I need to study, so it kind of never works (girl,
Grade 8). Even if the pupils try to get higher grades, the feedback they receive is
not seen as sufficient. Many would like to get more continuous feedback on their
performance and not only at the end of the semester when they receive grades.
Some think that the teachers do not think about their grades during the semester
but do a summative calculation only at the end.
Based on the pupils statements, it does not seem common for them to be part of
the assessment process. The only subject the pupils refer to is Swedish, where
they have given each other responses on written texts. That said, many
considered it is difficult to judge someone elses work or to be able to find
something to comment on.
School 2
In this school, some pupils prefer conventional tests rather than assignments that
require a lot of writing, thinking, and formulating long coherent texts. Even if
they are aware that this kind of studying has a negative effect on their learning,

151
they argue that it is easier just to read and memorise content and answer
questions in a test than to elaborate on an issue in a broader sense. They study
for the test and then forget everything, especially if they feel the subject is
uninteresting:
I think some tests, when you are not interested then you just
study sentences and wordsthen when there is a question then I
can answer it exactly, but at the same time I dont understand what
it is about, but I write exactly as it should be but I myself I
dont understand it and the day after the test I know nothing
about it I have just learned the words. (girl, Grade 9)
The interviews revealed that pupils occasionally are aware of the knowledge
requirements, as some of the teachers have shown them to the pupils. However,
the requirements generally seem to be rather unclear to the pupils.
The pupils expressed different degrees of stress connected to grades and

assessment. It is rather obvious that, on a general level, the feeling of stress is
related to age and grade. Pupils in Grade 9 are more stressed than those in
Grades 6 and 7. Pupils in Grade 9 had a lot of national tests during the time of
data gathering:
Now when we have the national tests it is very, very much in

one week! Now we have three (national tests) this week, two in
History and one in French it is very much . it becomes very
stressful. (girl, grade 9)
The pupils were not only worried about the national tests, they also had other
tests at the same time. Some of the pupils were aware that the purpose of the
national tests is to identify their subject knowledge, and that they are not meant
that to study a lot in beforehand. But they still study and seem to feel very
frustrated:
Ive heard the test will be about History there is endless much
history and I think it is meaningless to study think I wont have
time to learn everything! So I just choose a little its hard to
know what the test will be about. (girl, Grade 9)
The pupils were also asked if they were informed about how they were
performing and how to proceed to improve their grades. The most common
answer was that they get feedback from the teachers at the end of each semester,
when the grades were already set. Other than that, they also get feedback at the
individual evaluation talk (together with parents) every semester. Some pupils
said that it also is ones own responsibility to ask the teacher about feedback.
Some pupils experienced oral and written feedback on tests and other
schoolwork, but the most common feedback regarding grades was given in bad
situations. One girl, for example, was upset because the teacher contacted her
mother instead of talking to her when she was about to fail in a subject. The
interviews also revealed situations where teachers used grades to correct what

152
they saw as misbehaviour or laziness. Occasionally, the pupils experienced

involvement in the assessment process. In some subjects, they corrected their
own tests or commented on a classmates test or work.
Analysis and discussion
Practice
The observations revealed that the overall pedagogic model in both schools can
be linked to what Bernstein (2000) calls the performance model. The pedagogic
practices are characterised by isolation between the different subjects, thus
strong classification, even though the framing within the subjects differs to some
degree between the schools. In School 1, the teachers in the observed lessons can
be said to have a more profound understanding and interpretation of EE
(Backstrm-Widjeskog, 2010), and as a result, elements of the competence model
can be found. This is illustrated in that pupils have more room for creativity,
regulate their time and pace to a higher extent, cooperate and communicate, and
thus experience more self-regulatory elements (Bernstein, 2003).
Common in both schools were the preparations that were undertaken before the
national tests in connection with everyday teaching. David (2011) discussed the
narrowing of the curriculum, which means that the subjects or the content
included in tests has a more prominent role in regular everyday teaching than
other subjects or content. This is also something that is apparent in both schools.
The teachers, in different ways, expressed that the national tests control teaching
content, which implies a teaching to the test practice (Mehrens & Kaminski,
1989). This in turn entails a risk that scores and grades, instead of learning,
become the main focus (Eklf, 2017). In School 2, references to grades are not
seldom used in connection with pupils behaviour. It can be argued that grades
and tests to a certain extent control both teaching content and pupils behaviour
(Bernstein, 1971; Stobart, 2008). In the same school, grades and national tests are
used as tools to motivate pupils to work harder, and if they do so, they may quit
the lesson earlier, which can be seen as corresponding badly with the means of
entrepreneurial learning. When assessment is described in literature about
entrepreneurial learning, it is linked to AfL (Josefsson & Josefsson, 2009; Falk-
Lundqvist, Hallberg, Leffler, & Svedberg, 2011). Thus, formative or
entrepreneurial assessment can be especially found in the feedback to pupils
in School 1. Other than that, such assessment often takes the form of general
praise and is non-specific in relation to the progress of learning (c.f. Gamlem &
Smith, 2013). More examples of AfL are found in School 1, even if they mainly
seem to be used to meet summative requirements, and tests and grading criteria
control the formative activities in everyday teaching (Torrance, 2007).
In conclusion, summative requirements point out the direction for teaching and
assessment in the classrooms that were studied (c.f. Stobart, 2008). The
observations indicate that the teaching is to some extent directed towards
complex goals/knowledge requirements and that parts of entrepreneurial
learning (e.g. the ability of analysing and reasoning) are present, even if some

153
pupils have difficulty knowing what is expected of them regarding different

qualities and content in their work (Diehl, 2016b; Olovsson, 2015).
Teachers expressions
In School 1, it is obvious that teachers struggle to find a balance between the
curriculum merge of the performance and competence model (Bernstein, 2000).
The introduction of EE has exposed teachers to the contradiction between the
two models and induced worries about how to ensure that pupils meet the
knowledge requirements yet are given the opportunity to work
entrepreneurially. The performance model requires teachers to be explicit and
objective when grading pupils abilities, which is more feasible in a practice
characterised by strong classification and framing, thus a curriculum following a
collection code (Bernstein, 1975; 2000). However, the prerequisites for EE include
weak classification and framing, i.e. a competence model according to an
integrated code. Yet, this entails individual-assessment communication about
intentions, dispositions, and relations based on pupils own reflexivity
(Bernstein, 2000). One of the teachers has solved the problem by introducing
score rubrics, which she views as enabling her to be specific and clear with
pupils and herself regarding grades and at the same time allowing for what is
labelled entrepreneurial teaching and learning (SNAE, 2011a). However, the use
of score rubrics has occasionally been met with criticism, namely that it evokes
surface learning, leads to a more passive role for pupils, and entails a type of
AfL that can be seen as more teacher-centred (Jnsson, Holmgren & Lundahl,
2014). Stobart (2008) argues that there is a risk that AfL will be reduced to an
instrument operated by high-stakes accountability, simply offering preparation
for summative assessment. The dominance of external accountability pressures
implies that there is a risk that the letter rather than the spirit of AfL merges into
the accountability systems, and that AfL only becomes a means for achieving
accountability requirements (Black, 2015; Stobart, 2008). Most teachers in School
2 are not as concerned, for they link EE to assessment by letting the pupils form
part of the assessment process through correcting their own and/or their
classmates tests. In this school, it is also clear that not only the pupils
knowledge concerning the knowledge requirements are assessed. Skills such as
creativity, responsibility for ones own work, and aspects such as concentration,
behaviour, and diligence, form the basis for assessment (cf. Klapp, 2015). This is
in line with the performance model where the mode of instruction implies
disciplinary regulation (Bernstein, 2000).
Pupils expressions
Pupils in both schools talk about the lack of real and deep learning when
studying for tests; they prepare by crude studying, or even memorising, before a
test and then forgetting everything after. The testing culture which they are
experiencing can clearly be connected to the performance model (Bernstein,
2000). Even if national tests include questions aimed at abilities required by the
competence model, such as analysing, reflecting, and evaluating, many pupils
seem to perceive them as high-stakes (Eklf & Nyroos, 2013) and prepare them
in accordance with a performance model. Thus, The shadow of high-stakes
summative assessments operates as an obstacle through students as well as

154
directly for teachers (Black, 2015, p. 174). The pupils in School 1 seek more
projects and practical work as well as other methods for showing their
knowledge achievement. Such requests were not as common in School 2, where
some pupils preferred conventional tests requiring only short answers easily
identified as right or wrong. A possible explanation may be the lack of
experience with pedagogic modes connected to entrepreneurial learning or
learning in accordance with an integrated code, more of a competence model,
which the pupils in School 1, to some extent, have experienced (Bernstein, 2000;
Diehl, 2016b). The predominance of the performance model and its implications
as a repair service is revealed in School 2 in a pupils expression of
disappointment with the teacher, who contacted the pupils mother instead of
the pupil regarding the potential failure in a subject.
The empirical material does not provide information about how the
development of the complex competences and abilities are presented to the
pupils, but, at least in School 2, the pupils argue that their awareness of the
knowledge requirements is inadequate. This gives reason to suppose that
discussions about different qualities on reasoning and analysing do not occur to
any significant degree (c.f. Gyllander Thorkildsen & Erickson, 2016). Pupils
think that their involvement in the assessment process can be increased, even if
pupils in School 2 already have some relevant experience. More feedback from
the teachers regarding pupils progress in different subjects is asked for, as it is
seen as developing learning (e.g. Hattie & Timperley, 2007; Wiliam, 2011). More
detailed reflection of pupils awareness of the knowledge requirements and
what is assessed is found in pupils in School 1; these pupils find the information
rather clear in some subjects, though not in all subjects. For the pupils in School
2, it is more common not to be aware of the knowledge requirements. In relation
to the two knowledge forms (performance and competence) described by
Bernstein (2000), it can be concluded that School 1, to a greater extent than
School 2, merges the performance and the competence models. School 2 can be
said to work based on the performance model in all subjects except for
Handicraft. Hence, when it comes to the pupils awareness of the knowledge
requirements, they seem to be implicit and diffuse (as in the competence model)
in both schools. Yet, this may depend on how pupils view criteria. In School 1,
the pupils seem to know what to do to achieve the knowledge requirements but
not why, whereas the pupils in School 2 more often have a diffuse sense of why
but do not know, and can, on occasion, feel uncertain about how to meet the
knowledge requirements.
Conclusion
With a few exceptions, either teaching or learning in the two schools seems
largely aligned with whether AfL or EE. Teaching usually takes place within
specific subjects and is based on the knowledge requirements, most likely to ease
and facilitate assessment (Wahlstrm & Sundberg, 2015). Thus, there are clear
indications that the summative assessment is central to both schools, as
demonstrated by both observations and interviews with teachers and pupils.
Summative assessment can be linked to the knowledge form represented by the
performance model (Bernstein, 2000). This model is, in turn, associated with a

155
curriculum based on a collection code and a pedagogy that Bernstein refers to as

visible. A visible pedagogy is characterised by strong classification and framing
and emphasises pupils performance and their external products (Bernstein,
2000). According to Bernstein (2003), visible pedagogy is the standard European
practice and the one that leads to professional occupational placement. This
means that the message systems, curriculum, pedagogy, and evaluation
(assessment) are mainly influenced by the performance model, which
subsequently impacts teachers and pupils in terms of consciousness and
identity. The consciousness and identity, or the mind-set, that EE pursues
requires more of a competence model and can thus be seen slightly more in
School 1 than in School 2, even if the performance model is predominantly
featured in both schools. Through, among other aspects, the introduction of EE
along with strong OECD recommendations (Nusche, Halsz, Looney, Santiago
& Shewbridge, 2011) and efforts to implement AfL (Jnsson, Lundahl &
Holmgren, 2015), the Swedish curriculum can be said to fuse modes of both
performance and competence models. Bernstein (2000) argues that modes of
competence models can be seen as interrupts or resistance to the normal
performance model. This may explain the difficulties and dilemmas that both
teachers and pupils encounter and experience in relation to EE as well as AfL.
As the implications of EE and AfL may not fit pupils and teachers
understanding and socialisation of the means of schooling, teaching, and
learning.
The question of teacher autonomy in relation to national tests was raised during
the teacher interviews. The introduction of EE in educational policy and
curricula is intimately linked, nationally and internationally, to neoliberalism
and marketisation: Pupils are to be prepared for the demands of the global
labour market (e.g. European Commission, 2010; 2013; 2016; Government Office
of Sweden, 2009; OECD, 1998; SNAE, 2015). This, on one hand, implies a need
for pupils bound for the labour market to leave school with an amount of
complex skills, abilities, and competences that are advocated by EE and
supported by AfL. On the other hand, neoliberalism and marketisation underlie
competition and survival of the fittest, which may explain the increased focus
on measurement, performance, and accountability. Thus, it reflects the means of
summative assessment. Bernstein argues that Accountability is facilitated by
the objectivity of the performance and thus outputs can be measured and
optimised (Bernstein 2000, p. 50). Among the educational research society,
there is significant concern about what comes in the wake of the ideological shift
and its focus on performance assessment, measurement, quality assurance
(Apple, 2009; Englund, Forsberg & Sundberg, 2012; Lidman, 2011, Ozga, Dahler-
Larsen, Segerholm & Simola, 2011; Ozga, 2017), tests, and test results (Ball 2003;
Lindgren, Hult, Segerholm & Rnnberg, 2012; Lindgren, 2012 Power, 1999). For
example, there are concerns about the focus on teachers abilities in eliciting
certain outcomes replaces a discussion about the desirability (or lack thereof) of
the outcomes themselves (Biesta, 2009). In Bernsteinian (2003) terms, this
development would be connected to extroverted modalities within the
performance model, which result in less autonomy for teachers, instead of
external regulation such as economy and local market regulation of their work.

156
As such, this means that entrepreneurial teaching and learning, as well as the
means of AfL, includes an inherent policy and curriculum contradiction, for they
strive to foment real and deep learning in an aim to educate and develop
autonomous, confident, analytic, and reflexive citizens. Thus, it seems that the
letter rather than the spirit of AfL (Marshall & Drummond, 2006; Stobart, 2008) is
at use. This reasoning can be linked to Bernsteins (Bernstein, 1990) theory of
sacred and profane knowledge forms. The sacred can be seen as the spirit and
connected to bildung and thus to know-why knowledge, and the profane as
the letter, thus more practical know-how knowledge. In educational
practice, for teachers and pupils, the fusion of the performance and competence
models is quite a challenge!
Acknowledgements
This study is a part of a PhD project financed by the Swedish independent
research institution Innovation, Research, and Development in Schools (IFOUS)
and the Ume School of Education, Ume University, Sweden.
References
Apple, M.W. (2009). Foreword. In S. Gewirz, P., Mahony, In. Hextall, & A. Cribb (Eds.),
Changing Teacher Professionalism International trends, challenges and ways forward.
London & New York: Routledge.
Au. W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis.
Educational Researcher, 36(5), 258-267.
http://journals.sagepub.com/doi/pdf/10.3102/0013189X07306523
Backstrm-Widjeskog, B. (2010). Teachers thoughts on entrepreneurship education. In
K. Skogen, & J. Sjvoll, J. (Eds), Creativity and Innovation: Preconditions for
Entrepreneurial Education. (pp. 107-120). Trondheim: Tapir Academic Press.
Ball, S. (2003). The teachers soul and terrors of performativity. Journal of Education Policy.
18(2), 215-228.
http://www.tandfonline.com/doi/abs/10.1080/0268093022000043065
Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education:
Principles, Policy & Practice, 18, 525.
http://dx.doi.org/10.1080/0969594X.2010.513678
Bernstein, B. (1971). Class, codes and control. Volume I. Theoretical studies towards a sociology
of language. London: Routledge.
Bernstein, B. (1975). Class, Codes and Control, Volume 3, Towards a Theory of Educational
Transmission. London: Routledge & Kegan.
Bernstein, B. (1990). The structuring of Pedagogic discourse. London: Routledge.
Bernstein, B. (2000). Pedagogy Symbolic Control and Identity Theory, Research, Critique.
Bristol: Taylor & Francis.
Bernstein, B. (2003). Class, Codes and Control, Volume IV, The structuring of Pedagogic
Discourse. Oxon: Routledge.
Biesta, G., (2009). Good education in an age of measurement: On the need to reconnect
with the question of purpose in education. Educational Assessment, Evaluation and
Accountability. 21(1), 33-46. http://dx.doi.org/10.1007/s11092-008-9064-9
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in
Education: Principles, Policy & Practice, 5, 774.
http://dx.doi.org/10.1080/0969595980050102

157
Black, P. (2015). Formative assessment an optimistic but incomplete vision, Assessment

in Education: Principles, Policy & Practice, 22(1), 161-177. DOI:
10.1080/0969594X.2014.999643
Brookhart, S.M. (2009). Assessment and examination. In: L. J Saha & A.G Dworkin (Eds.).
International handbook on teachers and teaching. P. 2. New York: Springer.
Cohen, L., Manion, L. & Morrison, K. (2011). Research Methods in Education. London &
New York: Routledge.
David, J.L. (2011). Research says../ High-Stakes testing Narrows the Curriculum.
Educational Leadership. 68(6), 78-80.
http://www.ascd.org/publications/educational_leadership/mar11/vol68/nu
m06/High-Stakes_Testing_Narrows_the_Curriculum.aspx
Diehl, M., Lindgren, J., & Leffler, E. (2015), The impact of Classification and Framing in
Entrepreneurial Education: Field observations in Two Lower Secondary Schools.
Universal Journal of Educational Research. 3(8), 489-501.
http://dx.doi.org/1013189/ujer.2015.030803
Diehl, M. (2016a). From Entrepreneurship to entrepreneurial Education in Lower
Secondary School: Pedagogising by Means of Pedagogic Device. Journal of
Educational Issues. 2 (1), 36-56. http://dx.doi.org/10.5296/jei.v2i1.8748
Diehl, M. (2016b). Teachers and Learners Experiences of Entrepreneurial Education:
Practice and Challenges in Relation to Visible and Invisible Pedagogy. Problems
of Education in the 21st Century, 73 (73), 27-44.
http://www.scientiasocialis.lt/pec/?q=node/1029
Englund, T., Forsberg, E. & Sundberg, D. (2012). Vad rknas som kunskap
Lroplansteoretiska utsikter och inblickar i lrarutbildning och skola. Stockholm: Liber.
Eklf, H., & Nyroos, M. (2013). Pupil perceptions of national tests in science: Perceived
importance, invested effort and test anxiety. European Journal of Psychology of
Education, 28(2), 497510. doi:10.1007/s10212-012-0125-6
Eklf, H. (2017). Betygen i den svenska skolan. I A. Hult & A. Olofsson (Red.),
Utvrdering och bedmning i skolan (s. 62-76). Stockholm: Natur och Kultur.
Erkkil, K. (2000). Entrepreneurial Education. New York & London: Garland Publishing
Inc.
European Commission (2002). Entrepreneurship education at school in Europe National
strategies, curricula and learning outcomes. Brussels: Eurydice European
Commission.
Europakommissionen (2006). Promoting entrepreneurship in schools and universities.
Bryssel: Eurydice.
European Commission (2010). Europe 2020. A European strategy for smart, sustainable and
inclusive growth. Brussels: European Commission.
European Commission (2012). Entrepreneurship education at school in Europe National
strategies, curricula and learning outcomes. Brussels: Eurydice.
European Commission (2013). Entrepreneurship Action Plan. Brussels: European
Commission.
European Commission (2016). Entrepreneurship Education at school in Europe. Eurydice
Report. Luxembourg: Publications Office of the European Union.
Falk-Lundqvist ., Hallberg P-G., Leffler E. & Svedberg G. (2011). Entreprenriell
pedagogik i skolan drivkrafter fr elevers lrande. Stockholm: Liber.
Falk-Lundqvist ., Hallberg P-G., Leffler E., & Svedberg G. (2014). Entreprenriellt lrande
i praktik och teori. Stockholm: Liber.
Gardner, J. (Ed.) (2010). Assessment and learning. London: Sage.

158
Gamlem, S. M. & Smith, K. (2013). Student perceptions of classroom feedback.

Assessment in Education: Principles, Policy & Practice, 20(2), 150-169.
http://dx.doi.org/10.1080/0969594X.2012.749212
Gibb, AA. (1993). Enterprise culture and education: Understanding enterprise education
and its links with small business, entrepreneurship and wider educational goals.
International Small Business Journal, 11(3), 11-34.
https://www.researchgate.net/publication/247738528_Enterprise_Culture_and
_Education_Understanding_Enterprise_Education_and_Its_Links_with_Small_B
usinessEntrepreneurship_and_Wider_Educational_Goals
Gibb, AA. (2000). In pursuit of a new enterprise and entrepreneurship paradigm of
learning: creative destruction, new values, new ways of doing things and new
combinations of knowledge. International Journal of Management Reviews, 4(3),
233-269. http://onlinelibrary.wiley.com/doi/10.1111/1468-2370.00086/abstract
Government Office of Sweden (2009). Strategi fr entreprenrskap inom utbildningsomrdet.
Stockholm: Regeringskansliet.
Groome, T. H. (2005). Educating for life. New York, NY: Crossroad.
Gyllander Thorkildsen, L. & Erickson, G. (2016). If theyd written more- On students
perceptions of assessment and assessment practices. Education Inquiry 7(2), 137-
157. http://dx.doi.org/10.3402/edui.v7.27416
Hannan, A. (2007). Interviews in Education Research. [Online] Received from:
http://www.edu.plymouth.ac.uk/resined/interviews/inthome.htm 20161005
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational
Research,77, 81112.
http://journals.sagepub.com/doi/full/10.3102/003465430298487
Hkansson, J. & Sundberg, D. (2012). Utmrkt undervisning Framgngsfaktorer i svensk
och internationell belysning. Stockholm: Natur och Kultur.
Johannisson, B., Amundsson, A. and Kivimki, K. (2010) Training in entrepreneurship as
a many-sided struggle for growing insight. In Skogen, K., &J. Sjvoll (Eds.),
Creativity and Innovation: Preconditions for Entrepreneurial Education (pp.171-188).
Trondheim: Tapir Academic Press.
Jones, B & Iredale, N. (2010) Enterprise education as pedagogy. Education + Training,
52(1), 7-19. http://dx.doi.org/10.1108/00400911011017654
Josefsson Bostani, S. & Josefsson, R. (2009). Formativ bedmning vid entreprenriellt lrande
s hr gr du. Kristianstad: Me University AB.
Jnsson, A. (2013). Lrande bedmning. Malm: Gleerups.Jnsson, A., Lundahl, C &
Holmgren, A. (2015). Evaluating a large-scale implementation of Assessment for
Learning in Sweden, Assessment in Education: Principles, Policy & Practice, 22(1),
104-121. DOI: 10.1080/0969594X.2014.970612
Klapp, A. (2015). Bedmning, betyg och lrande. Lund: Studentlitteratur.
Kvale, S. (1997). Den kvalitativa forskningsintervjun. Lund: Studentlitteratur.
Kyr, P. (2005). Entrepreneurial learning in a cross-cultural context challenges previous
learning paradigms. In: P. Kyr & C. Carrier (Eds.). The Dynamics of Learning
Entrepreneurship in a Cross-Cultural University Context. Tampere: University of
Tampere.
Lackus, M., & Moberg, K. (2013). Entreprenrskapsutbildning Frn ABC till Phd.
Entreprenrskpsforum.
Lackus, M. (2015). Entrepreneurship in Education What, why, when, how. Entrepreneurship
360. Background paper. Paris: OECD Publishing.
Lackus M., Lundqvist, M & Williams Middleton, K. (2015). Opening the Black Box of
Entrepreneurial Eduction outline of an app-based research project. Peer

159
reviewed conference contribution, 3E Conference; Lueneburg Germany, April

22-24
Lackus, M. (2016). Value Creation as educational Practice Towards a new Educational
Philosophy grounded in Entrepreneurship. Doctoral Thesis, Chalmers University,
Gothenburg.
Leffler, E. (2006). Fretagsamma elever Diskurser kring entreprenerskap och fretagsamhet i
skolan. Doctoral thesis, Ume University.
Liedman, Sven-Erik (2011). Hets! En bok om skolan. Stockholm: Bonniers Frlag.
Lindgren, Joakim (2012). Judgment or evidence?: Knowledge in Swedish Schools
inspection, paper presented at the 40th Annual Congress of the Nordic
Educational Research Association, 8-10 March 2012, Copenhagen
Lindgren, J., Hult, A., Segerholm, C. & Rnnberg, L. (2012). Mediating school inspection
Key dimensions and keywords in agency text production 2003-2010, Education
Inquiry 3(4), 569-590. http://www.diva-
portal.org/smash/get/diva2:572937/FULLTEXT01.pdf
Lindster Norberg, E-L. (2016). Hur ska du bli nr du blir stor? En studie i svensk
gymnasieskola nr entreprenrskap i skolan r i focus. Doctoral Thesis, Ume: Ume
University.
Lundahl, C. (2011), Bedmning fr lrande. Stockholm: Norstedts.
Lundahl, C., Hultn, M., Klapp, A., & Mickwitz, L. (2015). Betygens geografi
forskning om betyg och summativa bedmningar i Sverige och internationellt
Stockholm: Vetenskapsrdet.
Mahieu, R. (2006). Agents of Change and Policies of Scale. Doctoral thesis. Institutionen fr
svenska och samhllsvetenskapliga mnen, Ume University.
Marshall, B., & Drummond, M.J. (2006). How teachers engage with assessment
for learning: Lessons from the classroom. Research papers in Education, 21(2), 133-
149.
http://dx.doi.org/10.1080/02671520600615638
Mehrens, W. A., & Kaminski, J. (1989). Methods for improving standardized test scores:
Fruitful, fruitless or fraudulent? Educational Measurement: Issues and Practice, 8(1),
14-22. http://onlinelibrary.wiley.com/doi/10.1111/j.1745-
3992.1989.tb00304.x/pdf
Natriello, G. (2009). High stakes testing and teaching to the test. In: L. J Saha & A.G
Dworkin (Eds.), International handbook on teachers and teaching. P. 2. New York:
Springer.
Nusche, D., Halsz, G., Looney, J., Santiago, P & Shewbridge, C. (2011). OECD Reviews of
evaluation and assessment in education: Sweden. OECD.
OECD (1989). Towards an Enterprising Culture: Challenge for Education and Training.
Paris: OECD/CERI.
OECD (1998). Fostering Entrepreneurship. The OECD Job Strategy. France: OECD.
Olovsson, T.G. (2015). Det kontrollera(n)de klassrummet Bedmningsprocessen i svensk
grundskolepraktik i relation till infrandet av nationella skolreformer. Doctoral thesis,
Departement of Education, Ume University.
Olovsson, T.G. (2017). Betygens infrande i rskurs 6. I A. Hult & A. Olofsson (Red.),
Utvrdering och bedmning i skolan (s. 170-184). Stockholm: Natur och Kultur.
Ozga, J., Dahler-Larsen, P., Segerholm, C. & Simola H. (Eds.) (2011). Fabricating Quality in
Education: Data and governance in Europe. London and New York: Routledge.
Ozga, J. (2017). Education policy should not be driven by performance data. Nature
Human Behviour, 1(0014). http://www.nature.com/articles/s41562-016-0014
Power, M. (1999). The Audit society: rituals of verification. Oxford: Oxford University Press.

160
Re degrd, I K. (2014). Pedagogiskt entreprenrskap i lrerutdanningen: en framtidsrettet

lringsstrategi, Norway: Cappelen Damm.
SAIDE (2010). Draft outline of Bernsteins concepts. South African Institute for distance
Education. http://www.saide.org.za
Samuelsson, P. (2012). Betygshistorik. In K. Nordgren, C. Odenstad & J. Samuelsson
(Eds). Betyg i teori och praktik. Malm: Gleerups.
SOU 2013: 30. SOU 2013:30. Det tar tid om effekter av skolpolitiska reformer. Stockholm:
Utbildningsdepartementet.
Stobart, G. (2008). Testing times. New York: Routledge.
Suurtamm C., & Koch M. J. (2014). Navigating dilemmas in transforming assessment
practices: experiences of mathematics teachers in Ontario, Canada. Educational
Assessment, Evaluation & Accountability, 26(3), 263-287.
http://link.springer.com/article/10.1007/s11092-014-9195-0
Swaffield, S. (2011). Getting to the heart of authentic assessment for learning.
Assessment in Education: Principles, Policy & Practice, 18, 433449.
http://dx.doi.org/10.1080/0969594X.2011.582838
Swedish Government Office (2009). Strategi fr entreprenrskap inom utbildningsomrdet.
Stockholm: Regeringskansliet.
Swedish National Agency for Education, (SNAE), (2011a). Lroplan fr grundskolan,
frskoleklassen och fritidshemmet. Stockholm: Skolverket.
Swedish National Agency for Education, (SNAE), (2011b). Kunskapsbedmning i skolan:
Praxis, begrepp, problem och mjligheter. Stockholm: Skolverket.
Swedish National Agency for Education, (SNAE), (2015). Skapa och Vga Om
entreprenrskap i skolan. Stockholm: Skolverket.
Swedish Research Council, (2002). Forskningsetiska principer inom humanistisk-
samhllsvetenskaplig forskning. Stockholm: Vetenskapsrdet. Available at
http://codex.vr.se/texts/HSFR.pdf ; 2016-01-15
Torrance, H. (2007). Assessment as learning? How the use of explicit learning objectives,
assessment criteria and feedback in post-secondary education and training can
come
to dominate learning. Assessment in Education: Principles. Policy & Practice, 14(3),
281-294. http://dx.doi.org/10.1080/09695940701591867
Torrance, H. (2011). Using assessment to drive the reform of schooling: Time
to stop pursuing the chimera? British Journal of Educational Studies, 59 (4), 459-
485. http://dx.doi.org/10.1080/00071005.2011.620944
Tveit, S. (2014). Educational assessment in Norway, Assessment in Education: Principles,
Policy & Practice, 21(2), 221-237.
http://dx.doi.org/10.1080/0969594X.2013.830079
Wahlstrm, N., & Sundberg, D. (2015). Theory-based evaluation of the curriculum
Lgr 11. Uppsala: IFAU.
Watt Boolsen, M. (2007). Kvalitativa analyser, Malm: Gleerups.
Wikstrm, C. (2006). Education and assessment in Sweden. Assessment in Education:
Principles, Policy & Practice, 13(1): 113-128.
http://dx.doi.org/10.1080/09695940600563470
Wiliam, D., & Thompson, M. (2007). Integrating assessment with instruction: What will
it
take to make it work? In C. A. Dwyer (Ed.), The future of assessment: Shaping
teaching
and learning (pp. 5382). Mahwah, NJ: Lawrence Erlbaum.
Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation
37(1), 314. https://www.udir.no/globalassets/filer/vurdering/vfl/andre-
dokumenter/felles/what-is-assessment-for-learning1.pdf

161

Vol. 16, No. 1, pp. 161-175, January 2017
Review in Form of a Game:

Practical Remarks for a Language Course
Snejina Sonina
University of Toronto Scarborough
Toronto, Canada
Abstract. The article summarizes my own experience of conducting

reviews for exams in French language courses. Through more than ten
years of teaching French language and linguistics, I have developed pre-
exam review sessions based on game models to help my students repeat
term lessons in an engaging and memorable atmosphere. The article is
intended as a practical guide for creating and organising such a review
game. It is constituted of seven parts with subtitles, which should make
it easy to navigate. After a short introduction describing challenges
encountered in end-of-year reviews, the reader will find a general
description of the game in part 2 and examples of one complete round of
it in part 4, while the interceding part 3 will provide practical tips on
support materials, which themselves make up part 5. These supports
deal with ways of formulating questions and displaying answers using
animations in PowerPoint presentations. The final section, part 6, will
offer advice on using a course website and classroom aids to increase
student attendance and encourage more effective classroom
participation. The brief conclusion enumerates the beneficial effects of
the game, underlines the value of a well-prepared PowerPoint
presentation, and gives examples of students positive feedback on this
format of material review.
Keywords: language teaching; technology and PowerPoint; game-based

approach; final review
Introduction
Year after year, teachers and students face the same situation: as the end of the
term nears, inevitably bringing with it final exams, there comes a need to review
the entire terms material in one or two hours. Review sessions, meant to help a
nervous and busy student population in its preparations, turn into proto exams
that sometimes bore, never manage to address all questions and concerns, and
most often only heighten students anxiety. To counter this situation, I have
throughout the past fifteen years developed review sessions based on game
models to engage my students and help them repeat term lessons with a
lighthearted but effective touch.
Having noticed that games and game-like approaches are used rather
minimally in university teaching practices, I turned to the didactic literature

162
available on the subject in our Ontario Institute for Studies in Education (OISE).
Despite scarce sources on the matter, those few authors who do examine play
activities provide ample confirmation of the effectiveness of games not only
for teaching language or for instructing young people but for teaching in general
(though they do utter some words of caution too). So before introducing and
offering up my own game for reader review, let me provide a brief overview of
recent research on the didactic value of games in the following paragraphs.
I was pleased to see that contemporary teachers and researchers still
refer to Johan Huizinga and his Homo Ludens: The Playing Man or Man the Player,
a work that influenced my own approach to education years ago. Justin
Hodgson (2013, 46) uses Huizingas arguments to justify his design of a whole
course based on a video game, while Robert J. Blake (2013, 16364) insists on the
importance of gaming environments for learning when he follows Huizingas
argument that the deep-seated imperative to play is embedded in our human
nature. Since Huizinga outlined the crucial importance of game-like activities for
humans from ancient civilizations to modern-day cultures, I have always
considered playing an appropriate activity for the university classroom.
Nonetheless, I still relegated it to a secondary role even in language-practice
courses. This approach seems to reflect the general situation in our didactics:
most authors writing about play activities deal either with younger learners or
concentrate on computer games and computer-assisted language learning
(CALL) as a supplement to a wider-ranging education. The researchers all agree
that the games add to the pleasure of and motivation for learning. The authors
of one of the most recent volumes dedicated to engaging activities in school
language classes offer many variations of word games and brand-game formats
as a motivating way to contextualize language learning based on their
association with fun, relaxation, and pleasure (Watts and Phillips, 2014).
These same reasons fun and motivation are often put forth in
favour of educational video games. In this domain, James Paul Gees book What
Video Games Have to Teach Us about Learning and Literacy, which proffers thirty-six
learning principles used in digital games that, according to Gee, should exist in
any good learning environment, became as influential as Huizingas Homo
Ludens. Although Gees tome deals mainly with concepts specific to digital
games, such as role models, false identities, or command obedience, all
researchers promoting game use in teaching point to one characteristic of games
that Gee deems to be of paramount importance: A game is always engaging,
and thus it can engage in situated and embodied learning,1 therefore
transform[ing] language learning experiences2 and contributing to deep,
sustained, and transferable learning.3
It should not be left unsaid that a few voices do warn against too much
play. For example, Jackson, Dempsey, and McNamara question the balance
between the efforts necessary for mastering game skills and the learning
outcomes,4 while Robert J. Blake urges educators implementing games into the
1 Hodgson (2013, 48)

2 Sykes and Reinhardt (2013, 3)
3 Filsecker and Bndgens-Kosten (2012, 50)
4 Jackson, Dempsey and McNamara (2012, 115)

163
curriculum to ask themselves a question: Are students learning to play or playing

to learn?5 This question uses the title of an online article by H. C. Arnseth and
follows the latter authors argument. Although these concerns appear more
relevant for digital games than for my in-class game-like activities, I, too, have to
respond to this question. As I hope to demonstrate in the description of the
game below, I give my answer with certainty in alignment with another
influential book on games. Much like Prensky6, who, with his Games2train,
teaches serious business content with the help of game-like speed and graphics, I
place much greater importance on the content to be learned through the games
than on the pleasure they provide. A report from the Summit on Educational
Games by the Federation of American Scientists, quoted by Robert J. Blake, lends
further evidence to the value and importance of engaging learning activities:
Students only remember 10 percent of what they read; 20 percent of what they
hear; 30 percent if they see visuals; 50 percent if they watch someone model
something while explaining it; and 90 percent, if they engage in the job
themselves, even if only as a simulation or game.7
In this article I will summarize my own experience of conducting reviews
for exams in French language courses. The main goal of this undertaking is to
encourage language teachers to use ludolinguistics8 at least in review sessions by
showing that a little extra work is worth the effort. My secondary objective is to
engage in the discussion about the use of PowerPoint in the classroom by
forwarding a few arguments in its favour.
Since the article is intended as a kind of practical guide for organising a
review game, it seems sensitive to introduce the contents of its seven parts in
form of an outline with keywords in bold to make it easier to navigate. The
reader will find a general description of the game in part 2 and examples of one
complete round of it in part 4, while the following questions will be addressed in
each part in the following order:
1. What kind of challenges do we face in the reviews for final exams?
2. How can a game transform a tiresome review into an engaging activity?
3. How do different support materials influence the effectiveness of the
game?
4. What is the best way to formulate game questions?
5. What is the best way to display answers and how can PowerPoint help?
6. What kind of practical tips can I offer based on my experience?
7. Why is the game an excellent way to conduct a review for a final exam?
1. Challenges of reviews for final exams

In final review classes, we often encounter two main challenges: the immense
quantity of material to cover and an overall nervousness about the future exam.
To review the material for the final exam in a helpful manner usually means to
5 Robert J. Blake (2013, 175)

6 Prensky (2007)
7 Robert J. Blake (2013, 164)
8 This term describing play activities in language teaching was coined by Anthony
Mollica. See Anthony Mollica, Ludolinguistica e Glottodidattica (Guerra-Soleil, 2010).

164
discuss the format of the exam, to give samples of questions, to offer strategies
for dealing with them, and to remind students of possible mistakes or traps. It is
difficult to fit all this into a one-hour class, especially at the very end of the
semester, when we cannot count on students to practise on exams from previous
years or to formulate pertinent questions based on their studiesmost of them
are too busy with end-of-term tests and assignments.
The task of providing useful strategies for exam preparation is not an
easy one either: all the shoulds and musts we feel we should offer (as in
you should cope with limited time, you must decide how many hours you need
to study, you should never give up, etc.9) clearly only add to students anxiety
and, given in the form of a list, do nothing but overwhelm. Even if phrased as
friendly advice or personal experience, these warnings act mainly on the surface
of the mind and do not sink deeper without guided experience: students simply
do not know how to fill these abstract recommendations with content. When,
after a few conventional review classes, I recognized the difficulty of this task if I
truly wanted to contribute to student success, I decided that a practical activity
with some positive emotional value would be a logical solution for the problem.
2. Review as a game
One possible way to provide real-life experience of time management, which can
be an issue for students both during the exam and in preparation, is to create a
kind of competition. To further create a positive atmosphere and increase
students motivation in a competition, treats like chocolates offer themselves as a
simple but useful solution. So when my conventional review sessions failed to
produce the desired results, I began trying out a few different review games
from a simple question lottery to a team competitionall encouraged with
various Belgian chocolates, which the class infallibly welcomed with a loud
Wow!
Yet these initial activities did not work for all students: the games did not
address well either the quantity of the material or the students varying levels of
proficiency in French. Most often, while the best students would answer and
win chocolates for themselves or their teams, the rest of the class could not
follow. I could see it in their eyes but could not afford to dwell longer on each
question. So while these games definitely added some fun to the reviews, they
worked mainly for those who would get good marks anyway, leaving the
students most in need of review with even more anxiety and despondency,
expressed in sighs and desperate looks.
A few years ago I finally developed a format for a final-review game that
truly works for everybody. The format closely resembles that of Who Wants to Be
a Millionaire?, whose French title, Qui veut gagner des millions?, can be easily
transformed into Qui veut gagner des chocolats?.
Just like the well-known television game, the review game requires a
main player, who must answer a series of questions, and an audience, from
which the player is selected and which watches the answer progress closely and
9Cf. James L. Clare and G. D. Morison, Hints on Study and Exam Techniques,
http://www.casact.org/admissions/syllabus/2016-new/index.cfm?fa=hints, December 07, 2016.

165
is ready to participate in questions through the ask-the-audience lifeline. The

games particular purpose in school of course also warrants some differences
from the TV show. For instance, a rather frequent change of main players and
activities keeps all students on their toes and interested. Let me briefly describe
the individual steps of the review game:
1. The Rules. I explain the games purpose and its rules, reminding
students that the questions are formulated as on the exam; I also show
the chocolate prizes.
2. The Fastest-Finger Question. We run through one question to make sure
that everybody understands how the game works, and then we play the
first question for real and select the first main player: the one who
answered it correctly first.
3. The Rounds. The main player has a chance to answer four questions,
after which the round is over: the main player receives a chocolate prize,
and everyone else in class gets ready to participate in the next round of
the fastest-finger question that will provide the next main player. Within
an hour it is usually possible to go through about six rounds of four
questions each.
4. The End. I reward all students with chocolates earned for participation in
the game and in the term, remind them that the game will be posted on
the course website, and encourage them to practise one round of the
game at a time before the exam.
One further change to the rules of the original television game became necessary
while playing the game in class for the first time. I had tried to be strict about
correct and incorrect answers and to withdraw a first chocolate won in case of an
incorrect answer to a following question, as it is done with money in the
television game. This did not go well: the whole class almost sang Oh, no!
when I announced the rule for the first time. I realized immediately that since
the point was to review grammar in an enjoyable environment it was better not
only to leave all chocolates in the hands of the players but also to help them win
more. So now I sometimes even suggest to call on a friend or to use one more
ask-the-audience lifeline under the pretext that the question is too tricky. The
latest game rules thus look like this:10
10The text translates as follows: The rules of the game: 1. Write your answers to the
fastest-finger question on your yellow paper 2. Raise you hand AFTER having
written them 3. If you are the first, youll need to answer four questions 4. If you
answer two questions correctly, youll get a small chocolate 5. If you answer all four
questions correctly, youll get a big chocolate 6. You can ask the class to help you. NB:
I will not translate question samples: they do not work for English.

166
Le r
rglement du jeu : 4. Si vous r
rpondez bien deux
questions, vous gagnerez un petit
1. crivez vos r
rponses la question chocolat.
qui est le premier (= fastest-
fastest-finger)
finger)
sur votre papier jaune. 5. Si vous r
rpondez toutes les
questions, vous gagnerez un grand
chocolat.
2. Levez la main APR
APRS les avoir crites.
6. Vous pouvez demander la classe

3. Si vous tes LE PREMIER, vous
de vous aider.
devrez r
rpondre quatre questions.
7 8
3. Different support materials: from paper to PowerPoint presentations

It is possible to use different support materials to organize this game. Even in the
years when not all classrooms were equipped with computers and projectors,
the game worked relatively well. In this case, the rules were written on a poster
and the questions printed out and distributed to each student. In the paper
version, it is necessary to fit all questions for one round on a single page, because
the main player should not see the questions in advance: once the main player
for the round is selected, all the students can turn the page and work with the
questions, but the main player gets the questions one by one from the instructor.
It is also important to write the correct answer on the blackboard afterward,
because there are always students who miss oral instructions.
One of the advantages of paper is that it leaves a hard copy with
examples in students hands, which is helpful if we can trust that they marked
the answers correctly and added appropriate comments about incorrect answers
and possible traps. The use of transparencies adds two more advantages: no
time is wasted with writing on the blackboard, and it is also easier to comment
on the wrong answers, because the difficulties of the question can be pointed at
on the screen in front of everyone. Plus, if an erasable marker is used to indicate
correct answers, the transparencies become recyclable.
In classrooms equipped with computers and projectors, one can start the
review session by playing a short video with the French opening of Qui veut
gagner des millions? to set the mood and to facilitate the explanation of the rules.11
Once the students have recognized the prototype of the proposed game, one can
proceed to a PowerPoint presentation that greatly eases the flow of the game in
comparison with the use of paper and transparencies. Moreover, the use of
PowerPoint facilitates the recycling of questions and the changing of content,
which helps to constantly improve the quality of the review. Well-prepared
PowerPoint slides can also enhance the quality of the presentation with colours
and animations. I will give more examples of and explanations for such quality
11See Qui veut gagner des millions gnrique TV France 2010,

http://www.youtube.com/watch?v=K8LFAHkaO30&feature=related, December 07,
2016.

167
improvements in part 5. In the following part, I will provide examples of

questions that tolerate any support: paper, transparencies, or PowerPoint slides.
4. Examples of questions
The game questions provide the means to discuss the format of the exam and to
review the material. There are two main types of questions: fastest-finger
questions, used to choose the main player for each round, and actual game
questions (most often concerning grammar), answered by the main player or by
the class as a lifeline.
In terms of preparation for the final exam, fastest-finger questions serve
to review vocabulary learned during the semester. In a language course they can
be of two main types: for verbs, reorganizing actions in a logical order and, for
nouns, adjectives, and adverbs, rearranging a list of given words according to
their meaningsize, value, position, or the like, starting from the biggest, the
cheapest, the lowest, etc. Here are two examples, one for each type, for a game in
a first-level French course (FREA01).
Rarrangez ces actions selon leur ordre logique!
a) se laver
b) sen aller
c) se rveiller
d) se lever12
Rarrangez ces adverbes de quantit selon la quantit exprime en
commenant par la plus petite!
a) beaucoup
b) peu
c) trop
d) assez 13
When the students are writing their answers to the fastest-finger question, I
usually wait until I see at least three hands, remember in what order they rose,
and then collect the pieces of paper with answers from the three fastest students.
This is necessary in case the first and the second students get it wrong. In the
rare case that all three answers are incorrect, I offer a collective game to the class
and make students vote for each question, that is, raise their hands in support of
the answer they deem correct. In such a case, I distribute the small chocolates
after this round, rather than at the end of class, as they represent a reward for
everybodys participation.
The grammar questions are used as a tool to discuss the way in which
questions on the final exam will be phrased, to review the main grammar points,
and to warn about possible traps. Once the main player has been chosen, I
explain the grammar task and the part of the exam to which it corresponds. For
example, in the round for the review of pronouns, the player will be managing
12 The text translates as follows: Rearrange these actions according to their logical order!
a) get up; b) go out; c) wake up; d) get washed/take a shower
13 The text translates as follows: Rearrange these adverbs of quantity according to the
quantities that they represent starting from the smallest! a) much/many; b) little/few; c)
too much/many; d) enough

168
two types of tasks that correspond to the parts D and E of the final exam. First, I
show and read the task in the exact wording of the exam:
D. Les pronoms personnels. Remplacez les mots souligns par y ,
en ou un pronom personnel. Mettez le pronom la place approprie.
Faites tous les changements ou les accords ncessaires.14 (10 points)
Then I display the first question with multiple-choice answers:
1. Est-ce que le singe a bouff les bananes?15
A) Il les a bouffs
B) Il en a bouff
C) Il les a bouffes
Once the answer is given by the main player, I ask the class why the two other
answers are incorrect and remind them of the dangers of forgetting about
agreement (A and C) or not paying attention to the article (B and C). Then comes
the second question:
2. Est-ce que je parle mon chat de mes problmes?16
A) Jen parle
B) Je lui parle des miens
C) Je lui en parle
Once the correct answer is given, the class is again invited to explain why in this
case the answer A is insufficient and B does not fit this exercise.
After the last question that corresponds to one of the exam exercises, I
ask whether students have any particular difficulties with this material. I usually
look at the students who did not fare well on grammar tests and invite questions
from them; then, depending on the feedback, I briefly review the most difficult
points or the general guidelines. For example, in this instance, I could go back to
the previous question or to the formulation of the task to remind students that
direct object pronouns are identical to definite articles; that indirect pronouns
are lui and leur; that en is used to replace nouns with the preposition de, the
indefinite article, and quantities; and that y stands for places and objects with the
preposition . After that, we continue with the next task, phrased exactly as on
the future exam:
E. Les autres pronoms. Remplissez les blancs. Utilisez les pronoms
possessifs, dmonstratifs, interrogatifs. 17(10 points)
Then follow the question and its possible answers:
3. Je ne prends pas dautres mdicaments que ________ quon ma
prescrits.18
14 The text translates as follows: D. Personal pronouns. Replace the underlined words
by y, en or a personal pronoun. Put the pronoun into a proper place. Make all
necessary changes and agreements.
15 The question translates as follows: Did the monkey devour the bananas? the
correct answer translates He devoured them

16 The question translates as follows: Do I talk to my cat about my problems? the
correct answer translates I talk to him about them

17 The text translates as follows: E. Other pronouns. Fill in the blanks. Use possessive,
demonstrative, and interrogative pronouns.
The correct answer translates as follows: I do not take any other pills than those that
18
was prescribed

169
A) ce
B) ceux-ci
C) ceux
This, too, is followed by an in-class discussion and a warning about the possible
confusion of ce and ceux (A and C) and a brief reminder about the use of ci and
l (B) with demonstratives. Finally comes the last question for this player:
4. Est-ce que cest ta valise?
Oui, cest ____________.19
A) celle-l
B) quelle
C) la mienne
Once the main player gets the chocolate prize and applause, I go back to the
original task phrase and talk with the class about particular difficulties they
might have with these pronouns. If there are no burning questions, I will simply
remind them of the agreement of gender and number with those of the
antecedent using question 4 and replacing ta valise with ton sac and tes affaires.
At the end I always remind my students that despite the multiple-choice
format of the review game, the final exam does not offer various answer options,
and that I am offering these choices here only to explain common errors, often
using incorrect answers from students in previous years.
5. Examples of answer displays20: Custom animation

The most important aspect of the game is to make sure that all students get the
correct answer to each question and understand why the other answers are
unacceptable. Unlike paper and transparencies, PowerPoint offers many
opportunities for the effective presentation of answers. For example, for
comments and warnings about incorrect answers I use call-outs that gradually
appear on the PowerPoint slide21. In the case of the first question of the round
described above, animations would appear in this order:
19 The correct answer translates as follows: Is it your suitcase? Yes, it is mine
20The pictures below illustrate various techniques for displaying answers and other
useful information, the translation of the French text, which contains mostly examples
understandable for an English speaker, is therefore unnecessary.
21In terms used by Sophie Dufour and Chantal Parpette in their article on the use of
PowerPoint presentations in university courses (2014, 25), the slide itself represents the
object of the explanations while the call-outs correspond to the extracts of the
explanations themselves. I believe that such use of the call-outs solves the problem
pointed out by Dufour and Parpette because they do not distract students attention
from oral explanations, as slides filled with longer explanatory extracts would do (2014,
28), but on the contrary draw the attention to the main points.

170
Pronoms 1. smiley indicating the

1. Le singe a bouff
bouff les bananes!
bananes! correct answer
= m.pl.
A) Il les a bouff
2. call-out for les in C
bouffs
B) Il en a bouff
bouff Remplace des 3. call-out for en in B
C) Il les a bouff
bouffes bananes
4. call-out for -es in C
= f.pl.
Remplace les
5. call-out for -s in A
bananes 11
In the fastest-finger questions, the screen may not only display the correct order
of the words but also some images that augment the visual salience22 of the
material and thus reinforce the memorising of vocabulary. The pictures below
show the same slide on which the answers appear in the correct order and with
illustrations, right after the main player is selected.
Qui est le premier? Qui est le premier?
Rarrangez
arrangez ces animaux selon leur Rarrangez
arrangez ces animaux selon leur
taille en commen
commenant par le plus taille en commen
commenant par le plus
petit!
petit! petit!
petit!
a) une vache a) une vache
b) un lapin b) un lapin
c) un cochon c) un cochon
b c a
18 19
In actual game questions, several explanations can be added to the answers; they
can explain the quantity of points lost or gained or even provide brief definitions
of complex concepts, as in the following slides taken from a course on
commercial French (FREB18):
Vocabulaire : Synonyme Qui est le premier? (transports)
Une valeur mobilire Rarrangez ces formules selon les frais

du fournisseur en commenant par la
plus coteuse !
1 Un titre 1 point A
a) port pay Le fournisseur paie tout
2 Une action 0,5 point
C b) port d Le client (acheteur) paie tout
3 Une scurit 0 Franco bord (free
(free on board)
board)
B c) FOB
23 20
22The importance of the salience of the material to be learned is discussed by Maria

Dakowska in In Search of Processes of Language Use in Foreign Language Didactics (2014,
293)

171
If the explanations are too bulky to fit the 6 x 6 PowerPoint rule,23 one can use
another slide to illustrate the point. I usually indicate the correct answer with a
smiley and change the background colour for the explanatory slide to show that
it is not part of the game:
Question de comprhension p.109 + tymologie

Quelle rduction est accorde
1 Escompte = quand on paie au comptant
au client qui paie au comptant ? = ex (=out) + (argent) comptant
2 Ristourne = un remboursement
1 Escompte retourn au client ( la fin de lanne)
3 Remise = le prix est remis (reconsidr)
2 Ristourne
pour quantit, service etc.
3 Remise
4 Rabais = le prix est abaiss cause
4 Rabais dun dfaut (ou hors-saison)
26 27
The most important part, however, is to choose the multiple-choice answers

wisely. Each one should serve a particular purpose: to illustrate a range of
grammatical or lexical phenomena,24 to point to major difficulties, or to reveal
possible confusions. For this reason, I never offer simply wrong or impossible
answersthey would not have any explanatory force and might even lead to an
unconscious memorisation of mistakes.
6. Practical advice plus templates, clickers, and natural pauses

To run the game smoothly, it is important to do a few things in advance:
a) assign students vocabulary review as homeworkit is a smaller task
than having to review the entire course material, so there is more
probability they will do it, especially if one mentions that it will be
important to win chocolates;
b) post the format of the final exam on the course website and announce
that the final and VERY important review will take place in the last class;
c) buy some Belgian or Swiss chocolates of different sizesthe smaller ones
as encouragement prizes and the bigger ones as main prizes: so about six
big chocolates and enough small chocolates for each student in the class,
plus about ten more for intermediate prizes and other unexpected
encouragement needs;
d) cut enough pieces of colour paper, 2 x 2, to distribute to all students
they will be necessary to jot down the answers to the fastest-finger
questions and to draw attention to the rules. They should be small to be
23A well-designed and readable slide contains no more than six lines, with no more than
six words in each line.
24 For example, the pronouns celle/quelle/mienne, offered as answers for the fourth
question in 4, give examples of all types of pronouns
demonstrative/interrogative/possessiverequired for the reviewed exercise; the lexical
items in the slide above offer the whole range of synonyms for the term discount.

172
easy to collect and coloured to be easy to spot and to be referred to as

the yellow paper, for example.
The Internet offers a lot of help for those who want to create classroom activities,
including sites that provide templates for all sorts of games. Yet in my
experience readymade templates have rarely helped. For example, the template
for Who Wants to Be a Millionaire would force me to offer four possible answers
for each question. This is undesirable in terms of time management, both in slide
preparation and in classroom timing, and also unjustified in terms of possible
mistakes worthy of discussion; usually two are enough to exemplify reasonable
errors. My own simple PowerPoint slides offer more flexibility and thus suit the
needs of the game better.
After a few games, I also realised that there is no need to economise each
minute wasted on the technical needs of the game. At first, I had students help
me collect pieces of paper with answers and bring chocolates to the winners to
save a few seconds and to be able to do more rounds of the game: the idea was
ill-advised. In fact, the natural pauses that occur when I walk toward the winner
to present the chocolate, for example, actually allow students to breathe: to write
down the correct answer still displayed on the screen, to clap, or to simply
recover briefly from the quite intense tempo of the game. It appears necessary to
give students some room for just breathing in any class, but during a final
review, this is even more the case because of the quantity of the material
covered: it is better to combine two exam exercises in one round to be able to go
through all of them than to rush through more rounds without having enough
time to dwell on the main points and to give students a few seconds to digest the
information.
Finally, to improve students participation, I always encourage as many
ask-the-audience situations as possible, because they invite the whole class to
become actively involved in the game. For this purpose, clickers, which are
becoming more and more popular in university classrooms, work very well.
Vos rponses (clickers)
Each student presses the letter
a)
corresponding to the answer chosen as
b)
correct, and we can see the distribution
c)
of choices on the screen.

30
If a class is equipped with clickers and it is essential with a particular question

that the majority of the class understands it well, this same game can be
conducted without the main player. In the same way, slides inspired by the
game can prove very useful in everyday teaching practice, especially for classes
that use clickers.
7. Conclusion
The described game certainly offers a solution to the review challenges outlined
in 1. It allows the discussion of all main grammar points in the very format of
the future exam and shows many possible traps in practice. Thus the game
solves the problem of an enormous quantity of material through the careful
choice of questions and answers, which include typical errors from previous
years. Moreover, this type of review allows including game activities into

173
university courses, which usually do not have any room for such activities
because of a tight schedule. I believe that the time deficit is the main reason for
the lack of attention to ludolinguistics in the higher education: for example, the
most recent volumes regrouping publications concerning didactics in higher
education contain only a few articles on game-like activities25 and even though
there are many publications that prompt innovative teaching and approaches
to student learning26, very few of them suggest games, and even fewer
professors ever have a chance to try games in the classroom27.
However, my review game has always been time efficient. Since each
student tries to choose the right answer while the main player is taking the time
to decide, all students become more aware of grammatical dangers after seeing
themselves falling into well-designed traps during the game. The game also
helps students maintain their highest level of attention during the review, not
only because it offers a diversity of activities but also because everybody can
keep up with the tempo thanks to natural pauses and well-displayed
information.
The game also solves the problem of typical exam anxiety by creating a
positive atmosphere of mild competition, by offering challenging questions that
provoke a desire to try and succeed, and by offering a sweet reward for
participation. It also offers an encouragement for everybody when the chocolates
are distributed at the end. It is a great pleasure to see the students positively
reacting in classYes!, I knew it!, Bien sr!and leaving class inspired
and willing to try the game again at home. The data from surveys that I conduct
during the final test in the commercial French course confirm the effectiveness of
the review game: no one has ever answered no to the question whether the
final game helped remember the material on the test.
From this point of view, the value of a well-prepared PowerPoint
presentation is priceless. Of course, it also contributes to the smooth flow of the
game and to better timing, as well as to the quantity and quality of the reviewed
material. Most of all, however, it helps by encouraging every student to revisit
the game posted on the course website. My parting words usually are: See, if
you review only one round a day, and one round takes only five minutesnest-
ce pas?and then spend a little more time to review the corresponding
25The collection of articles on Emerging Issues in Smart Learning has only two articles on
game-based learning activities one describing their benefits for listening and speaking,
the other for working memory capacity. The Proceedings of the 11th European
Conference on Technology Enhanced Learning in Lyon (Adaptive and Adaptable Learning,
2016) contain only three presentations dealing with games but only one of them deals
with the game activities in the classroom, while two other treat game-based exercises.
26Deborah Cartmell and Imelda Whelehan A Short History of Adaptation Studies in the
Classroom in Teaching Adaptations (edited by Deborah Cartmell) New York : Palgrave
Macmillan, 2014, 10.
27Most of the articles in the Emerging technologies for the classroom: a learning sciences
perspective (2013) actually describe educational games in virtual worlds and not in the
classroom.

174
grammar in the textbook, youll do great! Oui, a va! Merci! Cest promis!,
comes the answer.
References
Adaptive and Adaptable Learning : 11th European Conference on Technology Enhanced
Learning, EC-TEL 2016, Lyon, France, September 13-16, 2016, Proceedings.
(edited by Katrien Verbert, Mike Sharples, Toma Klobuar)Cham: Springer
International Publishing, 2016.
Blake, Robert J. Brave new digital classroom: technology and foreign language learning.
Washington, D.C. : Georgetown University Press, 2008.
Cartmell, Deborah and Imelda Whelehan A Short History of Adaptation Studies in the
Classroom in Teaching Adaptations (edited by Deborah Cartmell) New York:
Palgrave Macmillan, 2014, 1-10.
Dakowska, Maria. In Search of Processes of Language Use in Foreign Language Didactics.

Frankfurt am Main: Peter Lang GmbH, Internationaler Verlag der
Wissenschaften, 2015.
Dufour, Sophie and Chantal Parpette. Les powerpoints dans les cours universitaires :
une mthodologie stabiliser in Langues, cultures et pratiques en contexte :
interrogations didactiques. Paris : Riveneuve ditions, 2014. 23-45.
Emerging Issues in Smart Learning (edited by Guang Chen, Vive Kumar, Kinshuk,
Ronghuai Huang, Siu Cheung Kong) Berlin, Heidelberg : Springer Berlin
Heidelberg: Springer, 2015.
Emerging technologies for the classroom: a learning sciences perspective. (edited by Chrystalla
Mouza & Nancy Lavigne) New York; London: Springer, 2013.
Filsecker, Michael and Judith Bndgens-Kosten. Behaviorism, Constructivism, and

Communities of Practice: How Pedagogic Theories Help Us Understand Game-
Based Language Learning. In Digital Games in Language leaning and Teaching
(edited by Hayo Reinders). New York: Palgrave Macmillan, 2012, 50-69.
Gee, James Paul. What Video Games Have to Teach us About Learning and Literacy. New
York: Palgrave Macmillan, 2007.
Jackson, Tanner G., Kyle B. Dempsey and Danielle S. McNamara. Game-Based Practice
in a Reading Strategy Tutoring System: Showdown in iSTART-ME. In Digital
Games in Language leaning and Teaching (edited by Hayo Reinders). New York:
Palgrave Macmillan, 2012, 115-138.
Hodgson, Justin. Developing and Extending Gaming Pedagogy: Designing a Course as

Game. In Rhetoric/Composition/Play through Video Games: Reshaping Theory and
Practice of Writing. (edited by Richard Colby, Matthew S.S. Johnson, and
Rebekah Shultz Colby) New York : Palgrave Macmillan, 2013, 45-60.
Huizinga, Johan. Homo Ludens: a Study of the Play-Element in Culture. Boston: Beacon
Press, [1972], 1950.
Mollica, Anthony. Mollica, Anthony. Ludolinguistica e Glottodidattica. Perugia: Guerra-

Soleil, 2010.

175
Prensky, Marc. Digital Game-Based Learning. St. Paul, MN: Paragon House, 2007.
Sykes, Julie M. and Jonathon Reinhardt. Language at Play: Digital Games in Second and
Foreign Language Teaching and Learning. Boston: Pearson Education, 2013
Watts, Catherine and Hilary Phillips. Jumpstart! French and German: Engaging Activities for
Ages 7-12. New York: Routledge, 2014.

Vol 16 No 1 - Janaury 2017

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 16 No 1 - Janaury 2017

Uploaded by

Copyright:

Available Formats

IJLTER.

Using Reflective Journaling to Promote Achievement in Graduate Statistics Coursework...................................... 120

International Journal of Learning, Teaching and Educational Research

Item Consistency Index: An Item-Fit Index for

Hollis Lai,1 Mark J. Gierl,2 Ying Cui,2 Oksana Babenko3

School of Dentistry, Faculty of Medicine & Dentistry

University of Alberta, Canada

Abstract. An item-fit index is a measure of how accurately a set of

2017 The authors and IJLTER.ORG. All rights reserved.

Item discrimination helps describe how well an item can differentiate

Cognitive Diagnostic Assessment and Model Fit

2017 The authors and IJLTER.ORG. All rights reserved.

An essential part of CDA development relies on the definition of a Q-

To ensure that CDA results provide the most accurate information to

Evaluating Model-Fit for CDA

Sinharay and Almond (2007) also developed an approach for evaluating

2017 The authors and IJLTER.ORG. All rights reserved.

where is the number of examinees with skill pattern k, is the

2017 The authors and IJLTER.ORG. All rights reserved.

where Xj is the examinees scored response for item j, sj is an index set

Item Consistency Index (ICI)

Alternatively, to evaluate the misfit for item j, the number of misfitting

where is student s score (1 or 0) to item , and is student s score

2017 The authors and IJLTER.ORG. All rights reserved.

subset of skills measured by item j. In this manner, for a correct response

The set of alternative comparisons combined with comparisons from

where is student s score (1 or 0) to item , is an index set that

2017 The authors and IJLTER.ORG. All rights reserved.

item across all students.

To illustrate the calculation of the ICI, consider a hypothetical

Suppose this CDA of four skills was administered to an examinee who

To demonstrate the performance of this item-model fit index across a

2017 The authors and IJLTER.ORG. All rights reserved.

Methods and Results

Study 1: Simulation Study

The simulation process is similar to the actual steps used in developing

To simulate the effectiveness of ICI under different testing conditions,

2017 The authors and IJLTER.ORG. All rights reserved.

Second, unlike the related person-fit HCI which is independent of sample

Third, an important feature for an item-model fit index is to detect items

To evaluate the effectiveness of the ICI for detecting poor-fitting items, a

2017 The authors and IJLTER.ORG. All rights reserved.

item fit was needed because no point of comparison currently exists in

To ensure the classification results were consistently produced, each of

Proportion of Mean ICI

2017 The authors and IJLTER.ORG. All rights reserved.

1600 5% 15 0.21 0.41

2017 The authors and IJLTER.ORG. All rights reserved.

Table 3. Power of ICI for identifying poor-fitting items

Test Sample Proportion of Poor-Fitting Items

Table 4 summarizes the likelihood of a well-fitting item being

Taken together, the simulation study results highlight important trends

2017 The authors and IJLTER.ORG. All rights reserved.

Table 4. Misclassification rate of ICI in identifying well-fitting items

Test Sample Proportion of Well-Fitting Items

When the proportion of poor-fitting items was manipulated, the power

Study 2: Use Case Application

The CDA we used was designed to evaluate student mastery for

2017 The authors and IJLTER.ORG. All rights reserved.

to 9 with regrouping). The CDA was developed as follows. First, a

2017 The authors and IJLTER.ORG. All rights reserved.

correlated with either the difficulty or discrimination values. This result

Table 5. Summary of the results from the subtraction CDA

Attribute Item Number P-Value Discrimination ICI