LANGUAGE TESTING AND

ASSESSMENT
WRITING ITEMS AND TASK
Evidence-centered design (ECD)
Describing items and tasks
Tasks and teaching
PROTOTYPES, PROTOTYPING, AND
FIELD TEST

HERBERT MOUREN
SIANIPAR
LANGUAGE TESTING AND EVALUATION

EVIDENCE-CENTERED DESIGN
(ECD)
ECD is a methodology for designing assessments
that underscores the central role of evidentiary
reasoning in assessment design.
ECD treats knowledge as scientific because it is
primarily a method that leads to the test
designer understanding more about relations
between variables for a particular assessment
context.

ECD is based on three premises:

(1)An assessment must build around the important
knowledge is acquired and put to use;
(2)The chain of reasoning from what participants
say and do in assessments to inferences about what
they know, can do, or should do next, must be based
on the principles of evidentiary reasoning;
(3)Purpose must be the driving force behind design
decisions, which reflect constraints, resources and
conditions of use.
(Mislevy et al., 2003: 20)

THE STRUCTURE OF ECD

ECD is considered to be a framework in the
sense that it is structured and formal and thus
enables the actual work of designing and
implementing assessments in a way that makes
a validity argument more explicit.
There are six models within EDC-style test
specification, they are:
- Student model
Presentation model
- Evidence models
Assembly model
- Task models
Delivery model

MODELS IN THE CONCEPTUAL

ASSESSMENT FRAMEWORK OF ECD

DESCRIBING ITEMS AND TASKS

There are three steps in task design:
- Identify the knowledge, skills or abilities
(constructs) that we wish to test.
- Identify the relationship between the constructs
and the behaviours in situations that call for
their use.
- Describe the features of situations that provide
the evidence to draw inferences from behaviour to
constructs.

TASK ACTIVITIES AND GOALS

Interactional activity-with relation to

information
1. Each participant holds different information
and needs both to give and to receive
information in order to complete the task.
2. One participant holds all the information
and supplies it.
3. Each participant holds all the information.

 Interaction

requirement for activity

1. Each participant is expected both to request
and to supply information.
2. One participant supplies all the information.
3. Each participant is expected to request and
supply information but is not obliged to do so.

 Communication

goal
With respect to goal orientation
1. Participants have convergent goals.
2. Participants have divergent goals.
With respect to outcome options
1. Only one acceptable outcome is possible.
2. More than one outcome is possible.

TASKS AND TEACHING

It is essential that we are critical of tests and the

claims that surround them. Without the detailed
information about constructs, all teachers can do
is copy the test tasks for use in the classroom and
hope for the best. With the constructs and
evidentiary reasoning upon which a test is based,
they are empowered to be creative in the
development of communicative competence in the
classroom. This is how testing can best help the
teacher, in the classroom.

PROTOTYPES,
PROTOTYPING AND FIELD
TESTS
Prototypes
Prototyping
Field testing
The iterative nature of the process

PROTOTYPES
Prototype comes from the Greek word prototypo
( ), which is sometimes translated as
archetype, made up as it is of the two elements
proto ( ) or first, and typos () or
type. The meaning of type in this sense is of a
mould or stamp, from which other examples could
be created.
From the explanation above, we can conclude
that, in design task, we have to focus of the first
type of task its self, to have a contrast or basic
view developing next task for the learners.

PROTOTYPING
In this part we have to focus how we create a new
test. This is very different from writing new
items or tasks for a test that already exists, and
which is the topic of most other texts on language
testing. We are concerned about the creation
of the first form of a test. Other forms are then
generated from the test specifications.
These are used in rapid prototyping to discover
whether the parts will work well within the
larger product that is being developed.

FIELD TESTING
In the integrated academic writing-reading
task, there are some clues that we can concern
by those purposes ;
Investigating construct-irrelevant influences on
score meaning
Studying internal test structure
Is it necessary to have a minimum reading ability
before it is possible to perform on writing tasks?
Do the writing tasks relate to one another in
theoretically anticipated ways?
Do integrated tasks relate in expected ways to
independent tasks?

Studying external relations

What is the relationship of the new TOEFL to its
predecessors?
What is the relationship between new writing
measures and qualities in writing that
experienced markers and teachers value?
Group comparisons, or the differences
between performance of
graduate versus undergraduate students
native versus non-native writers
skilled versus less skilled writers
students before and after instruction.

THE ITERATIVE NATURE

OF THE PROCESS

The iterative nature of the process means that have to do

the tasks more and more, it will be a good behavioral
activity for the learners. As long as the tasks encourage the
skills of the learners, the processes will be a continue
habit. All of the tasks will be a better one in case of how
the process in delivering clues or information clearly done
to do an assessment. In language testing there is no point
in manufacturing many forms of a test that is unreliable or
not valid for its intended purpose. However, this is not to
say that there are no test designers who produce tests of
poor quality, sell them, but fail to tell their customers of
the potential faults of the product.