Moving from off-the-shelf chatbots to a user-designed bespoke L2 chatbot

Abstract

This study investigates the application of Large Language Model (LLM)-based chatbots for second language (L2) learning, focusing on the three chatbot-building platforms such as ChatGPT, Poe AI, and Pi. Engaging 96 pre-service teachers in South Korea, it examined their perceptions of chatbots built via these platforms concerning human-likeness, pedagogical usefulness, and specific strengths and weaknesses. Participants were asked to create task-oriented chatbots using these platforms and to converse with them. The findings reveal varied perceptions of human-likeness among the chatbots, with Pi rated the highest. Regarding usefulness for L2 learning, the chatbots built via all three platforms were deemed beneficial, especially for engaging in realistic scenarios and providing authentic, context-appropriate expressions. Each platform demonstrated unique strengths but also showed some limitations, based on which we provide the pedagogical implications. Overall, the present study contributes to the evolving field of chatbot-assisted language learning, demonstrating the utility of LLM-based platforms in creating customized L2 learning chatbots.