A Validation Argument for Cloze Test Item Function in Second Language Assessment

dc.contributor.author Trace, Jonathan
dc.date.accessioned 2017-12-18T22:12:32Z
dc.date.available 2017-12-18T22:12:32Z
dc.date.issued 2016-08
dc.description Ph.D. University of Hawaii at Manoa 2016.
dc.description Includes bibliographical references.
dc.description.abstract This study presents a validation argument for cloze item function. While cloze tests have been said to measure reading ability in L1 contexts, most L2 research has linked their function to measures of general language proficiency. This study explores this issue more closely through examining evidence to construct a warranted interpretation/use argument (Kane, 2013) for cloze item function. Performance on fifteen 30-item cloze passages was examined for both native users of English and second language learners from Japan and Russia. Passages were examined in relation to their composition, syntactic structure, and cohesive features based on a number of text analysis methods. Item-level features for each of the 450 cloze items were also classified according to their composition and their interaction with surrounding context. For the latter, native user data were collected to gather information about the kinds of contextual information utilized by the individual items through a series of sentence completion tasks. These tasks, as well as L1 cloze test data, were gathered using Amazon Mechanical Turk. Variables for each passage composition, syntactic structure, cohesion, and item characteristics were explored in relation to Rasch logit measures of item difficulty for both the L1 and L2 cloze datasets, and used to create a structural equation model explaining the influence of each on item function. The results indicated that cloze items performed very similarly for both L1 and L2 users of English. Items were found to access context on multiple levels, and different classes of items functioned well for both groups of examinees. Other similarities were also observed through test analyses, item analyses, and correlations between item difficulty and passage-level variables. In addition, a single structural model was able to explain item function for both groups. Differences were found for L1 and L2 examinees in how factors for passage composition, structure, cohesion, and items related to item function, however, even these results seemed more in line with differences in the ability of the examinees rather than differences in the construct being measured for each. Overall, the evidence pointed to cloze items measuring the same construct for L1 and L2 test takers.
dc.identifier.uri http://hdl.handle.net/10125/51512
dc.language.iso eng
dc.publisher [Honolulu] : [University of Hawaii at Manoa], [August 2016]
dc.relation Theses for the degree of Doctor of Philosophy (University of Hawaii at Manoa). Second Language Studies
dc.subject Validation
dc.subject Cloze
dc.subject Many-faceted Rasch measurement
dc.subject Structural equation modeling
dc.subject Mechanical Turk
dc.title A Validation Argument for Cloze Test Item Function in Second Language Assessment
dc.type Thesis
dc.type.dcmi Text
Original bundle
Now showing 1 - 2 of 2
No Thumbnail Available
2.85 MB
Adobe Portable Document Format
Version for non-UH users. Copying/Printing is not permitted
No Thumbnail Available
2.94 MB
Adobe Portable Document Format
For UH users only