Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission

Principles of Assessment and Evaluation

Validity & Validity Types Reliability & Reliability Types

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

characteristics of a good assessment

validity reliability

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

Lets have a look at the definitions

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

Eh wait. Didnt I ask you to look up for the definition yourself?

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

reliability

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

Hmm arent you supposed to have the definition with you?

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

Are these definitions clear enough to guide you in general understanding? Can you find something off about these definitions?

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session three

validity and reliability

refreshed definitions

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity
Validity is the interpretation of the information that the test generates. It refers to the accuracy of the interpretation.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity

For example, a driving test must include a competency-based assessment which requires the candidates to drive a vehicle in a real setting.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

QUESTION

Can you think of examples of valid or invalid measures for test?

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity of assessment
Validity of any assessment is dependent upon the purpose of the assessment and the way in which evidence is interpreted and used by the key stakeholders.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity

Our interpretation on validity often involves the process of making inference.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

inference
Refers to the conclusions drawn on the synthesis and interpretation of evidence and subsequently, requires judgment on the basis of the evidence gathered and prior beliefs/expectations.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity types
face validity

content validity
construct validity consequential validity

criterion validity

Principles of Assessment and Evaluation

CHARACTERISTICS OF GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

face validity
At surface level, the assessment translates sufficiently the content and construct being assessed.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

content validity
How well the content of the test samples the class situations or subject matter about which conclusions are to be drawn.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

face vs. content validity

content validity is more important than face validity.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

construct validity
It refers to the degree to which the inferences can legitimately be made. Applicable when there is no obvious body of content or when there are no existing criteria.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

construct validity

The extent a variable is abstract rather than concrete, we speak of it as being a construct. Such a variable is literally a construct in that it is something put together from their own imaginations.
Numally, J.C. (1978) Psychometric Theory

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

consequential validity
Refers to the consequences of the assessment.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

criterion validity
two types of criterion validity:
predictive validity concurrent validity

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

criterion validity

predictive
The extent to which the assessment is able to predict something that it should theoretically be able to predict.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

criterion validity

concurrent
Occurs when the criterion measures are obtained at the same time as the test scores.
The ability of a measure to indicate an individuals present standing on the criterion variable.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

reliability

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

reliability
Reliability is the ability to control background noise.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

reliability types
inter-rater (across assessors)

intra-rater (within assessor)

parallel forms (across tasks) internal consistency (within task)

split halves (within task)

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

inter-rater reliability
Consistency of judgment across different assessors using the same assessment task and procedure.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

intra-rater reliability
Consistency of assessment outcomes across time and location, and using the same assessment task administered by the same assessor.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

parallel forms reliability

Assessing two different assessments with the purpose of obtaining consistent results.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

internal consistency reliability

Concerned with how well the items or tasks act together to elicit a consistent type of response.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

The method used to collect the evidence will impact on reliability.

The way in which the assessor uses and interprets the evidence collected will impact on validity.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity or reliability?

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity or reliability?

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

validity or reliability?

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

Practicality

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

In addition to validity and reliability, considerations must also be given to the aspect of practicality.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

practicality
Considerations given to the operationalization of an assessment (i.e. cost, stake, worth) which may also affect decisions on validity and reliability.

Principles of Assessment and Evaluation

CHARACTERISTICS OF A GOOD ASSESSMENT

Created bt Zuraimi Zakaria Not to be distributed or reproduced without permission session five

end of session five