Please use this identifier to cite or link to this item:

Evaluating Rating Variations in Holisting Writing Placement Assessment

File SizeFormat 
Koyama2007.pdf586.1 kBAdobe PDFView/Open

Item Summary

Title: Evaluating Rating Variations in Holisting Writing Placement Assessment
Authors: Koyama, Dennis
Keywords: placement test
writing assessment
rating variation
academic writing
Issue Date: 2007
Abstract: This paper reports on an investigation of a writing assessment, the English Language Institute (ELI) writing placement test for international graduate students. The purpose of the study was four-fold: (a) to determine how many placement essays need further adjudication (i.e., additional readers); (b) to investigate the use of rating agreement ratios for estimating reliability; (c) to explore the potential use of collapsing the current seven-point rating scale into a threepoint rating scale; and (d) to determine how borderline placement ratings contribute in determining placement. The findings showed that 35% of essays needed further adjudications. The results of this study also indicated that there are no apparent losses in collapsing the seven-point rating scale into a three-point scale. The findings also suggested that the use of rating agreement ratios is superior to the Pearson product moment correlation coefficient in illuminating the similarities of ratings, rather than a similarity in rating patterns, which supports the conclusions of other researchers working with language performance assessments (e.g., Halleck, 1995, 1996; Kenyon & Tschirner, 2000; Norris, 2001; Thomson, 1995, 1996). The implications of these findings are discussed in terms of recommendations to the ELI for future writing placement practice.
Pages/Duration: 60 pages
Appears in Collections:Student Work - ELI

Please email if you need this content in an ADA-compliant format.

Items in ScholarSpace are protected by copyright, with all rights reserved, unless otherwise indicated.