A Comprehensive Criterion-Referenced Language Testing Project

Abstract

The English Language Institute (ELl) at the University of Hawai'i regularly offers seven courses in academic listenin~ readin~ and writing. The curriculum for each course has been extensively revised including thorough needs analysis, development of objectives, criterion-referenced tests, and materials, as well as improvements in teaching practices and regularly conducted formative evaluation procedures. This paper reports on the criterion-referenced test development portion of the curriculum. Each of the seven ELI courses has two forms of a criterion-referenced test designed expressly to measure the objectives of that course. The two forms are administered at the beginning and end of instruction in a counterbalanced design. Hence this testing project is large in scale including 14 different tests administered before and after instruction for about 500-600 students per year. While the objectives and resulting tests differ in organization and form across the seven courses, the processes involved in putting the tests in place are quite similar. The initial item development, piloting and revision processes are described in general terms. Details are provided about the results of the administrations of these CRTs during fall 1989. Descriptive and item statistics are presented (including the difference index, item cjl, B-index, and item agreement index) for each test. Dependability estimates [phi and phi(lambda)] are given, and evidence for the content and construct validity of the tests is also provided. The discussion centers on the problems encountered in developing such a comprehensive testing program, then turns to the benefits which CRTs can provide for overall curriculum development.