Please use this identifier to cite or link to this item:

Abridged Estimate of Inter-rater Reliability in ELI Writing Assessment

File SizeFormat 
Koyama and Silver2006.pdf346.36 kBAdobe PDFView/Open

Item Summary

Title: Abridged Estimate of Inter-rater Reliability in ELI Writing Assessment
Authors: Koyama, Dennis
Silver, Dan
Keywords: placement test
writing assessment
inter-rater reliability
test reliability
L2 writing
Issue Date: 2006
Abstract: The current study investigates the reliability estimates of the newly adopted writing placement test at ELI. To examine whether three raters’ judgment on learner level were consistent enough to justify the placement, statistical reliability measures such as Cronbach’s alpha, Standard Error of Measurement, and Correlation coefficients were adopted. The analysis of the consistencies among raters revealed that although generally the placement results were consistent among raters, several limitations were found. For example, frequent rater inconsistencies were found and oral report seemed to reveal high variability in their first decision and their final report. Further suggestions to improve inter-rater reliability are presented.
Pages/Duration: 15 pages
Appears in Collections:Student Work - ELI

Please contact if you need this content in an alternative format.

Items in ScholarSpace are protected by copyright, with all rights reserved, unless otherwise indicated.