How do Pedagogical Conversational Agents affect Learning Outcomes among High School Pupils: Insights from a Field Experiment

Abstract

Pedagogical conversational agents (CA) support formal and informal learning to help students achieve better learning outcomes by providing information, guidance or fostering reflections. Even though the extant literature suggests that pedagogical CAs can improve learning outcomes, there exists little empirical evidence of what design features drive this effect. This study reports on an exploratory field experiment involving 31 pupils in commercial high schools and finds that students achieved better learning outcomes when preparing for their tests with a pedagogical CA than without. However, the drivers of this effect remain unclear. Neither the use frequency of the design features nor the pupils’ expectations towards the CA could explain the improvement in marks. However, for the subjective perception of learning achievement, pupils’ expectations was a significant predictor. These findings provide support for the use of pedagogical CAs in teaching but also highlight that the drivers of better learning outcomes still remain unknown.