Classical And Rasch Analyses Of Dichotomously Scored Reading Comprehension Test Items

Abstract

This study demonstrates the use of both the classical and the Rasch Model item analyses in an attempt to evaluate the quality of reading test items used in an English Language placement test context. Thirty-five dichotomously scored items from the reading comprehension section of the placement test administered to the Matriculation Centre students of the International Islamic University Malaysia (IIUM) were subjected to both the Classical and Rasch Model analyses using statistical packages appropriate for the two analyses (SPSS and BIGSTEPS). The discriminant and difficulty indices of the items and the classification of the items according to their item characteristics based on Classical Test Theory are reported along with the results of the Rasch analysis of the same set of items. The characteristics of the items based on the two analyses are compared in an attempt to demonstrate the amount of information gained and the usefulness of the more powerful Rasch Model analysis in investigating item quality and test reliability. Given the type of decisions made on the basis of test scores in high-stakes tests, this study demonstrates the need to apply not just the classical approach, but also the need to incorporate modern measurement theory in the evaluation of high- stakes tests.