Abridged Estimate of Inter-rater Reliability in ELI Writing Assessment

Date
2006
Authors
Koyama, Dennis
Silver, Dan
Journal Title
Journal ISSN
Volume Title
Publisher
Volume
Number/Issue
Starting Page
Ending Page
Alternative Title
Abstract
The current study investigates the reliability estimates of the newly adopted writing placement test at ELI. To examine whether three raters’ judgment on learner level were consistent enough to justify the placement, statistical reliability measures such as Cronbach’s alpha, Standard Error of Measurement, and Correlation coefficients were adopted. The analysis of the consistencies among raters revealed that although generally the placement results were consistent among raters, several limitations were found. For example, frequent rater inconsistencies were found and oral report seemed to reveal high variability in their first decision and their final report. Further suggestions to improve inter-rater reliability are presented.
Description
Keywords
placement test, writing assessment, inter-rater reliability, test reliability, L2 writing
Citation
Extent
15 pages
Format
Geographic Location
Time Period
Related To
Rights
Rights Holder
Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.