Please use this identifier to cite or link to this item:

Multiple-choice discourse completion tasks in Japanese English language assessment

File SizeFormat 
Setoguchi (2008)_27(1).pdf342.98 kBAdobe PDFView/Open

Item Summary

Title: Multiple-choice discourse completion tasks in Japanese English language assessment
Authors: Setoguchi, Eric
Advisor: Brown, James D.
Issue Date: 2008
Abstract: A new class of multiple-choice discourse completion tasks (MDCTs) is beginning to gain popularity in the Japan English as a Foreign Language (JEFL) assessment context. In this study, an experimental MDCT test was administered to a sample of Japanese university students. An item format analysis was conducted focusing on the construct validity and discrimination of MDCTs in measuring the English listening proficiency of JEFL speakers. Using a combination of classical test theory and Rasch analysis, test performance was analyzed in regard to two research questions: (a) whether a pragmatic proficiency construct is related to item difficulty, and (b) whether the use of different distractor types has an effect on item discrimination characteristics. The results suggest that a pragmatic proficiency construct plays a role in determining item difficulty on MDCTs, bringing into question the construct validity of MDCTs as a listening language proficiency measurement. Additionally, MDCT item discrimination might be affected by the type of distractors being used, hinting at possible ways to optimize discrimination of MDCTs in norm-referenced testing (NRT). Given the high probability of continued use of MDCTs in the JEFL context and the need for more investigation into these items, this study is hopefully an early step towards better and more informed MDCT test design and use.
Pages/Duration: 61 pages
Appears in Collections:SLS Papers

Items in ScholarSpace are protected by copyright, with all rights reserved, unless otherwise indicated.