Please use this identifier to cite or link to this item: http://hdl.handle.net/10125/51512

A Validation Argument for Cloze Test Item Function in Second Language Assessment

File Description SizeFormat 
2016-08-phd-trace_r.pdfVersion for non-UH users. Copying/Printing is not permitted2.92 MBAdobe PDFView/Open
2016-08-phd-trace_uh.pdfFor UH users only3.01 MBAdobe PDFView/Open

Item Summary

Title: A Validation Argument for Cloze Test Item Function in Second Language Assessment
Authors: Trace, Jonathan
Keywords: Validation
Cloze
Many-faceted Rasch measurement
Structural equation modeling
Mechanical Turk
Issue Date: Aug 2016
Publisher: [Honolulu] : [University of Hawaii at Manoa], [August 2016]
Abstract: This study presents a validation argument for cloze item function. While cloze tests have been said to measure reading ability in L1 contexts, most L2 research has linked their function to measures of general language proficiency. This study explores this issue more closely through examining evidence to construct a warranted interpretation/use argument (Kane, 2013) for cloze item function.
Performance on fifteen 30-item cloze passages was examined for both native users of English and second language learners from Japan and Russia. Passages were examined in relation to their composition, syntactic structure, and cohesive features based on a number of text analysis methods. Item-level features for each of the 450 cloze items were also classified according to their composition and their interaction with surrounding context. For the latter, native user data were collected to gather information about the kinds of contextual information utilized by the individual items through a series of sentence completion tasks. These tasks, as well as L1 cloze test data, were gathered using Amazon Mechanical Turk. Variables for each passage composition, syntactic structure, cohesion, and item characteristics were explored in relation to Rasch logit measures of item difficulty for both the L1 and L2 cloze datasets, and used to create a structural equation model explaining the influence of each on item function.
The results indicated that cloze items performed very similarly for both L1 and L2 users of English. Items were found to access context on multiple levels, and different classes of items functioned well for both groups of examinees. Other similarities were also observed through test analyses, item analyses, and correlations between item difficulty and passage-level variables. In addition, a single structural model was able to explain item function for both groups. Differences were found for L1 and L2 examinees in how factors for passage composition, structure, cohesion, and items related to item function, however, even these results seemed more in line with differences in the ability of the examinees rather than differences in the construct being measured for each. Overall, the evidence pointed to cloze items measuring the same construct for L1 and L2 test takers.
Description: Ph.D. University of Hawaii at Manoa 2016.
Includes bibliographical references.
URI/DOI: http://hdl.handle.net/10125/51512
Appears in Collections:Ph.D. - Second Language Studies


Please contact sspace@hawaii.edu if you need this content in an alternative format.

Items in ScholarSpace are protected by copyright, with all rights reserved, unless otherwise indicated.