Visual saliency in captioned digital videos and learning of English collocations: An eye-tracking study

Abstract

This study explored how visual input enhancement impacts caption-reading behaviors, the acquisition of English collocations, and the recall of onscreen captions. The participants comprised 53 Korean undergraduate students at a high-intermediate level of English proficiency. They were assigned to either a baseline or an enhancement group. The baseline group viewed a digital video with unenhanced captions, whereas the enhancement group watched the same video with enhanced captions (i.e., captions including yellow-colored collocations). The eye movements of the participants were measured using an eye tracker. Thereafter, they completed a collocation test and a caption recall test. The results showed that the baseline and enhancement groups did not vary in their caption-reading behaviors. Conversely, the enhancement group significantly outperformed the baseline group on the collocation test. In the caption recall test, the enhancement group recalled significantly more target collocations than the baseline group, whereas the two groups did not differ in recalling unenhanced captions. Finally, correlational analyses revealed nonsignificant correlations between attention to target collocations and collocation test scores in both groups. This evidence suggests that enhanced video captions may be an effective means of stimulating collocational competence that is not at the expense of second language learners’ ability to learn video content.