1

Item Analysis in the Construction of Test Instrument (Dr See Kin Hai)
Item Analysis
The item analysis is an important phase in the development of an exam program. In this phase statistical methods are used to identify any test items that are not working well. If an item is too easy, too difficult, failing to show a difference between skilled and unskilled examinees, or even scored incorrectly, an item analysis will reveal it. The two most common statistics reported in an item analysis are the item difficulty, which is a measure of the proportion of examinees who responded to an item correctly, and the item discrimination, which is a measure of how well the item discriminates between examinees who are knowledgeable in the content area and those who are not. After you create your objective assessment items and give your test, how can you be sure that the items are appropriate -- not too difficult and not too easy? How will you know if the test effectively differentiates between students who do well on the overall test and those who do not? An item analysis is a valuable, yet relatively easy, procedure that teachers can use to answer both of these questions. To determine the difficulty level of test items, a measure called the Difficulty Index is used. This measure asks teachers to calculate the proportion of students who answered the test item accurately. By looking at each alternative (for multiple choice), we can also find out if there are answer choices that should be replaced. 1. Difficulty Index: to show the test items if they are: i) too difficult, ii) average, or iii) too easy I= Total No of students who answer correctly (C) Total no of students doing the test (T) 2. If the sample size is large and to cut down the time in computation of the Index, I Difficulty Index = I = C H + CL TH + L

Where CH = Total no. of students in the High Achievement group that answer correctly CL = no. of students in the Low Achievement group that answer correctly TH + L = Total no. of students in the High and Low achievement groups responding Difficulty Index I < 0.3 0.3 I 0.8 I > 0.8 Evaluation of item too difficult average too easy Conclusion modify the item accept the item modify the item

2 The Item Discrimination Index (D) Discrimination Index, refers to how well an assessment differentiates between high and low scorers. In other words, you should be able to expect that the high-performing students would select the correct answer for each question more often than the low-performing students. If this is true, then the assessment is said to have a positive discrimination index (between 0 and 1) -- indicating that students who received a high total score chose the correct answer for a specific item more often than the students who had a lower overall score. If, however, you find that more of the low-performing students got a specific item correct, then the item has a negative discrimination index (between -1 and 0). Another consideration for an item analysis is the cognitive level that is being assessed. For example, you might categorize the questions based on Bloom's taxonomy (perhaps grouping questions that address Level I and those that address Level II). In this manner, you would be able to determine if the difficulty index and discrimination index of those groups of questions are appropriate. For example, you might note that the majority of the questions that demand higher levels of thinking skills are too difficult or do not discriminate well. You could then concentrate on improving those questions and focus your instructional strategies on higher-level skills. 1. D reports the difference between the proportion of High and Low aacxhievers in answering aa test item correctly 2. D = high shows the item is good, D = low shows the item is badly constructed 3. The purpose of D is used to distinguish between the high achievers from the low achievers D = Discrimination Index = C H CL 1 TH + L 2

Discrimination Index (D) D > 0.4 0.2 D 0.4 0 D < 0.2 D<0

Item Evaluation high positive value average positive value low positive value negative value

Conclusion acceptable modify the item reject item and rewrite reject

(Low achievers perform better then high achievers)

3 COURSEWORK 1 (Test Construction) You have constructed a 10 items (questions) test and 20 students have responded. The results are shown in the Table below. 1. Fill in all the blanks. 2. Conclude if you accept the test item ( ) or reject the test item ( X ) at the Conclusion row.

4 COURSEWORK 2

Item Analysis Worksheet

Ten students have taken an objective assessment. The quiz contained 10 questions. In the table below, the students scores have been listed from high to low (Joe, Dave, Sujie, Darrell, and Eliza are in the upper half). There are five students in the upper half and five students in the lower half. The number1 indicates a correct answer on the question; a 0 indicates an incorrect answer. Student Name Joe Dave Sujie Darrell Eliza Zoe Grace Hannah Ricky Anita Total Score (%) 100 90 80 70 70 60 60 50 40 30 Questions 1 1 1 1 0 1 1 0 0 1 0 2 1 1 1 1 1 1 1 1 1 1 3 1 1 0 1 1 1 1 1 1 0 4 1 1 1 1 0 0 0 1 0 0 5 1 1 1 1 1 1 1 0 1 0 6 1 1 1 1 1 1 1 0 0 1 7 1 1 1 0 1 0 0 1 0 0 8 1 1 1 1 0 1 1 0 0 0 9 1 0 0 0 0 0 0 1 0 1 10 1 1 0 1 1 0 1 0 1 0

Calculate the Difficulty Index (p) and the Discrimination Index (D) for each question. # Correct (Upper group) # Correct (Lower group) Difficulty (p)

Item Question 1 Question 2 Question 3 Question 4 Question 5 Question 6 Question 7 Question 8 Question 9 Question 10

Discrimination (D)

5 Answer the following questions: 1.Which question was the easiest? 2. Which question was the most difficult? 3. Which item has the poorest discrimination? 4. Which questions would you eliminate first (if any) why?