Comparison of Item Response Theory Scaling Methods with ROC Analysis

Creative Commons License

Yurtcu M., GÜZELLER C. O.

JOURNAL OF MEASUREMENT AND EVALUATION IN EDUCATION AND PSYCHOLOGY-EPOD, cilt.13, sa.1, ss.15-22, 2022 (ESCI) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 13 Sayı: 1
  • Basım Tarihi: 2022
  • Doi Numarası: 10.21031/epod.892079
  • Dergi Adı: JOURNAL OF MEASUREMENT AND EVALUATION IN EDUCATION AND PSYCHOLOGY-EPOD
  • Derginin Tarandığı İndeksler: Emerging Sources Citation Index (ESCI), Scopus, TR DİZİN (ULAKBİM)
  • Sayfa Sayıları: ss.15-22
  • Anahtar Kelimeler: test equating, ROC analysis, scaling methods, AUC, CURVES, PACKAGE, AREA
  • İnönü Üniversitesi Adresli: Evet

Özet

In this study, one-dimensional item response theory models were evaluated using different scaling methods. In this context, the equating errors and the area under the curve of four scaling methods (Stocking-Lord, Heabara, Mean-Sigma, Mean-Mean), and one, two, and three parameters logistic models (1PL, 2PL, and 3PL) in nonequivalent groups with anchor test (NEAT) design were examined. Additionally, the equating errors of the scaling methods and the results obtained from ROC analysis were compared. Qatar's and Australia's PISA 2012 mathematical literacy test data were used in the study. The minimum error was obtained from the Mean-Mean method with the 1PL model, and the maximum error was obtained from the Mean-Mean method with the 3PL model. Similar results were observed in all comparisons and supported each other. It is concluded that ROC analysis can be used to compare different conditions, methods and models.