Скачать презентацию Psychometrics: test development and adaptation Lecture 3 Dr. Скачать презентацию Psychometrics: test development and adaptation Lecture 3 Dr.

Psychometrics_lecture_3.1.ppt

  • Количество слайдов: 16

Psychometrics: test development and adaptation Lecture 3 Dr. psych. Tatjana Kanonire 22. 01. 2014 Psychometrics: test development and adaptation Lecture 3 Dr. psych. Tatjana Kanonire 22. 01. 2014

Lecture plan p p Items’ reaction/difficulty indices Items’ discrimination indices Decisions about items Items Lecture plan p p Items’ reaction/difficulty indices Items’ discrimination indices Decisions about items Items analysis: practical work

Item’s difficulty/reaction index In classical test theory, a common item's statistical measure is the Item’s difficulty/reaction index In classical test theory, a common item's statistical measure is the item’s difficulty index, or “P value. ” It shows the proportion of participants who got the item correct. The difficulty index for personality tests shows how many participants gave the “yes”/ “agree” answer for item. This index is called the reaction index. For dichotomous items we can compute the difficulty index by formula: Pj – P value for j-item Nj – participants who got the item correct N – the total number of participants who gave the answer

Item’s difficulty/reaction index (2) Item’s difficulty index for dichotomous scale vary from 0 till Item’s difficulty/reaction index (2) Item’s difficulty index for dichotomous scale vary from 0 till 1 The lower is P index, the more difficult is the item. Item 1 . 45 Item 2 . 00 Item 3 1. 00 Item 4 . 75

Item’s difficulty/reaction index (3) If items are evaluated in ordinal scale, we compute difficulty/reaction Item’s difficulty/reaction index (3) If items are evaluated in ordinal scale, we compute difficulty/reaction index as arithmetic mean. But how can we know that item is difficult enough to stay in the test?

Item’s difficulty/reaction index (4) From psychometrics point of view the best items are those Item’s difficulty/reaction index (4) From psychometrics point of view the best items are those whose difficulty index varies from 20% till 80%. For dichotomous items -. 20 -. 80 For items with more than two possible answers we calculate the appropriate interval by formula: for upper bound: xmin + 0, 2 • (xmax – xmin) for lower bound: xmin + 0, 8 • (xmax – xmin)

Item’s difficulty/reaction index (4) For example, we want to calculate the bounds for 5 Item’s difficulty/reaction index (4) For example, we want to calculate the bounds for 5 -point scale. for upper bound: xmin + 0, 2 • (xmax – xmin) for lower bound: xmin + 0, 8 • (xmax – xmin) for upper bound: 1 + 0, 2 • (5 – 1)=1. 8 for lower bound: 1 + 0, 8 • (5 – 1)= 4. 2

Item’s difficulty/reaction index (5) Item 1 1. 85 Item 2 3. 45 Item 3 Item’s difficulty/reaction index (5) Item 1 1. 85 Item 2 3. 45 Item 3 2. 01 Item 4 4. 25

Item’s discrimination index p p It shows how well the item serves to discriminate Item’s discrimination index p p It shows how well the item serves to discriminate between participants with higher and lower values in the test (scale). => we can exclude the items that don’t work.

Item’s discrimination index (2) p We compute it as point-biserial correlation for dichotomous scales. Item’s discrimination index (2) p We compute it as point-biserial correlation for dichotomous scales. M 1 – scale mean for participants who got the item correct (1 point answer) M 0 - scale mean for participants who got the item incorrect (0 -point answer) SD – standard deviation of the scale for all sample P – item’s difficulty index Q = 1 -P

Item’s discrimination index (3) p The point-biserial correlation could be in interval from -1 Item’s discrimination index (3) p The point-biserial correlation could be in interval from -1 till +1. The point-biserial will be positive if participants with higher test score answered the item correctly more frequently than participants with lower score did, and negative if the opposite occurred. Acceptable discrimination index . 20 -. 80 p For ordinal scales we calculate Pearson correlation. p It is not possible to calculate the point-biserial correlation in SPSS, but Pearson correlation give the same value. p p

Item’s discrimination index (4) p p Another way to calculate discrimination index is item-total Item’s discrimination index (4) p p Another way to calculate discrimination index is item-total correlation, when we correlate item with test score minus item’s value. It is more strict criteria. In SPSS the item-total correlation is computed through “Scale” option.

Item’s discrimination index (5) There also other possibilities to evaluate item’s discrimination – “direct Item’s discrimination index (5) There also other possibilities to evaluate item’s discrimination – “direct method” p From sample select 10 -33% of participants with low and high test score; p Compute how many participants with high scores gave positive answer to the item, and how many participants with low answer gave positive answer to the item. Determine the proportion between them (PH - the proportion of participants with high score; PL - the proportion of participants with low score). p Compute discrimination index dj by formula: dj = PH – PL

Item’s discrimination index (6) We make a decision about items (what item will be Item’s discrimination index (6) We make a decision about items (what item will be left in the test) based on item’s difficulty/reaction and discrimination indices. 5 -point scale: p Items Reaction index Discrimination index 1. 3. 25 . 71 2. 4. 60 . 21 3. 2. 20 . 34 4. 1. 60 . 19 5. 4. 10 . 25 6. 2. 76 . 56

Item Response Theory (IRT) IRT or latent trait theory is the alternative way of Item Response Theory (IRT) IRT or latent trait theory is the alternative way of items analysis (“modern theory”). Based on the analysis of item’s regression curve, which characterizes the probability of correct answer (difficulty index) for respondents with diffirent level of test’score.