Investigating Item Bias on the PISA 2009 Reading Assessment: A Case of Macau with Chinese and English Versions.

Abstract

In recent years, there has been significant increase of regions and countries participating in international large-scale assessments, and this increase is largely due to the extensive information and analysis of the results that are available to the schools, parents, researchers, and educators. Nonetheless, it is questionable whether the results of these international large-scale assessments are reliable, valid, and comparable for different countries. Since most school authorities and educators are interested in using the results of these assessments to enhance the existing school curriculum, educational policy, and program development, the fairness of the assessments and equivalency need to be examined carefully. In this study, a selected sample of the Macau dataset obtained from the Programme for International Student Assessment (PISA) 2009 reading literacy assessment were analyzed to detect potentially biased items using the Mantel-Haenzsel (MH) and item response theory (IRT) methods. Findings indicated that both of the methods commonly identified 5 items that were not working equally across the groups. Limitations and implications were discussed.