Effective Utilization of Generative AI Tool in Education: A Study Exploring Question Pattern of Varied Student Performance

Abstract

While generative AI (GAI) holds promise in education, students and instructors often struggle to use it effectively. To help address this challenge, a Data Science tool (DST) was developed for use in an introduction to data science course project. DST integrates a task-tracking Kanban board with a Q&A GAI bot. Our findings revealed that higher-performing students exhibited a pattern of systematic daily DST usage. More importantly, their questions were generally Informational questions, exploring concepts in depth, and seeking a fundamental understanding of complex topics. In contrast, low-performing students (LPS) asked Informational and Procedural questions equally, relying on the DST for complete solutions, potentially indicating struggles with core concepts. Thus, tools like DST can effectively support student learning when utilized for informational questions to extend their conceptual understanding. Furthermore, these findings suggest that instructors can use these tools to proactively identify weak students based on the frequency and quality of their questions.