An exploration of examinee abilities, rater performance, and task differences using diverse analytic techniques

Abstract

The results of the current study indicated that: (a) the two group oral tasks and three semi-direct speaking tasks are not comparable in difficulty, (b) the method effect of the group oral tasks is so strong as to attenuate trait effect, (c) the three tasks demonstrate a strength in eliciting different aspects of oral proficiency such as accuracy, complexity, and lexical diversity, and (d) there is no statistically meaningful association between how the examinees perceived difficulty of the tasks and how they performed on them. In concluding this study by summarizing the findings, pedagogical implications, limitations, and future research avenues are also discussed.