Can ChatGPT make reading comprehension testing items on par with human experts?

dc.contributor.author Shin, Dongkwang
dc.contributor.author Lee, Jang Ho
dc.contributor.editor Mimi Li
dc.date.accessioned 2023-09-29T22:41:35Z
dc.date.available 2023-09-29T22:41:35Z
dc.date.issued 2023-10-02
dc.description.abstract Given the recent increased interest in ChatGPT in the L2 teaching and learning community, the present study sought to examine ChatGPT’s potential as a resource for generating L2 assessment materials on par with those created by human experts. To this end, we extracted five reading passages and testing items in the format of multiple-choice questions from the English section of the College Scholastic Ability Test (CSAT) in South Korea. Additionally, we used ChatGPT to generate another set of readings and testing items in the same format. Next, we developed a survey made up of Likert-scale questions and open-ended response questions that asked about participants’ perceptions of the diverse aspects of the target readings and testing elements. The study’s participants were comprised of 50 pre- and in-service teachers, and they were not informed of the target materials’ source or the study’s purpose. The survey’s results revealed that the CSAT and ChatGPT-developed readings were perceived as similar in terms of naturalness of the target passages’ flow and expressions. However, the former was judged as having included more attractive multiple-choice options, as well as having a higher completion level regarding testing items. Based on such outcomes, we then present implications for L2 teaching and future research.
dc.identifier.citation Shin, D., & Lee, J. H. (2023). Can ChatGPT make reading comprehension testing items on par with human experts? Language Learning & Technology, 27(3), 27–40. https://hdl.handle.net/10125/73530
dc.identifier.issn 1094-3501
dc.identifier.uri https://hdl.handle.net/10125/73530
dc.publisher University of Hawaii National Foreign Language Resource Center
dc.publisher Center for Language & Technology
dc.rights Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
dc.rights.uri https://creativecommons.org/licenses/by-nc-nd/4.0/
dc.subject Artificial Intelligence
dc.subject Automated Item Generation
dc.subject ChatGPT
dc.subject Content Generation
dc.title Can ChatGPT make reading comprehension testing items on par with human experts?
dc.type Column
dspace.entity.type
local.llt.topic Language Teacher Education and Technology Forum
prism.endingpage 40
prism.number 3
prism.publicationname Language Learning & Technology
prism.startingpage 27
prism.volume 27
Files
Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
27_01_10125-73530.pdf
Size:
379.66 KB
Format:
Adobe Portable Document Format
Description: