Exploring AI-Generated text in student writing: How does AI help?

Abstract

English as a foreign language (EFL) students’ use of artificial intelligence (AI) tools that generate human-like text may enhance students’ written work. However, the extent to which students use AI-generated text to complete a written composition and how AI-generated text influences the overall writing quality remain uncertain. 23 Hong Kong secondary school students wrote stories with AI-writing tools, integrating their own words and AI-generated text into the stories. We analyzed the basic structure, organization, and syntactic complexity of each story and its AI-generated text. Experts scored the quality of each story’s content, language, and organization. By employing multiple linear regression and cluster analyses, we found that both the number of human words and the number of AI-generated words significantly contributed to writing scores. Furthermore, students could be classified into competent and less competent writers based on the variations of students’ usage of AI-generated text compared to their peers. Cluster analyses revealed some benefit of AI-generated text in improving the scores of both high-scoring students’ and low-scoring students’ writing. We suggest differentiated, pedagogical strategies for EFL students to effectively use AI-writing tools and AI-generated text to complete writing tasks.