Generative and Conversational AI in Information Systems Research and Education: Opportunities and Challenges

Permanent URI for this collection

https://hdl.handle.net/10125/110012

Browse

Recent Submissions

Now showing 1 - 4 of 4
  • Thumbnail Image
    Item
    Solving Coding Challenges Jointly with a Large Language Model: Understanding Student Journeys Through Bloom’s Taxonomy
    (2025-01-07) Memmert, Lucas; Borchers, Marten; Plückhahn, Juliane; Bittner, Eva
    With the advancement of large language models (LLMs) such as ChatGPT, students increasingly employ LLMs during university courses to improve their coding skills. However, it is still unclear what using these systems means for student learning. In our study, we explore how students employ a LLM when solving a realistic coding challenge over the course of a semester in small groups. We analyze students’ LLM chat log data through the framework of Bloom’s taxonomy, a well-established education framework, to understand their problem-solving and learning behavior. We enrich our analysis with students’ responses to weekly survey questions, contextualizing our findings with subjective experiences. We find that students primarily use LLMs for lower-order thinking skills like remembering, understanding, and applying procedural knowledge. With our study, we contribute to the growing literature on understanding the effects of working and learning with LLMs and offer practical suggestions for teachers and students.
  • Thumbnail Image
    Item
    Generative Artificial Intelligence in Information Systems Research: Insights from a Scoping Review and Bibliometric Analysis
    (2025-01-07) Chouikh, Arbi; Khechine, Hager; Ammari, Mohamed Lassaad; Mellouli , Sehl
    This study aims to review and analyze the latest scientific breakthroughs in Generative artificial intelligence (Gen-AI) published in information system (IS) journals. The goal is to consolidate and organize the collective knowledge of Gen-AI in IS. We adopted a scoping review approach and complemented it with bibliometric analyses. We selected 55 articles from a collection of scholarly publications from several databases covering 2018 to 2024. Results reveal significant trends regarding the scientific production, the influential publications and articles, the study areas and their interconnections, the bibliographic coupling, the methodologies, and the theoretical foundations employed. This comprehensive analysis foresees a productive area of study on Gen-AI in IS as it established the basis for identifying key contributions and gaps and generating novel ideas for further exploration.
  • Thumbnail Image
    Item
    Integrating Generative AI into Information Systems Research: A Framework for Synthetic Data Evaluation
    (2025-01-07) Bono Rossello, Nicolas; Simonofski, Anthony; Bono Rossello, Lluc; Castiaux, Annick
    Generative AI is paving its way into the research process. Among the plethora of available generative AI solutions, the generation of synthetic data is one of the most controversial. The current division of opinion and the lack of formal approach to AI use in research create a situation of conflicting bad practices and under-used potential. This work aims to add nuance and structure to this research practice by providing a general framework to evaluate the use of synthetic data in different stages of the research process, based on the objective and methods of generation. Relying on a breakout literature review, we explore the fields of Data quality management and Control theory to transfer method theories from these fields to help us build the framework. The resulting conceptual framework provides an iterative scheme where, based on the desired properties of the data and its comparison to the synthetic result, the researcher can improve the outcome of the generation process and, equivalently, formally present the properties that make this data suitable for research.
  • Thumbnail Image
    Item