Generative AI for Systems Thinking: Can a GPT Question-Answering System Turn Text into the Causal Maps Produced by Human Readers?

Simple item page
dc.contributor.authorGiabbanelli, Philippe
dc.contributor.authorWitkowicz, Nathan
dc.date.accessioned2023-12-26T18:54:28Z
dc.date.available2023-12-26T18:54:28Z
dc.date.issued2024-01-03
dc.identifier.doi10.24251/HICSS.2024.905
dc.identifier.isbn978-0-9981331-7-1
dc.identifier.otherb52f1cee-9416-401b-ac63-5aaed9f098df
dc.identifier.urihttps://hdl.handle.net/10125/107291
dc.language.isoeng
dc.relation.ispartofProceedings of the 57th Hawaii International Conference on System Sciences
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0/
dc.subjectGenerative and Conversational AI in Information Systems Research and Education: Opportunities and Challenges
dc.subjectcausal map
dc.subjectgenerative ai
dc.subjectgpt
dc.subjectnetwork analysis
dc.subjectsystems science
dc.titleGenerative AI for Systems Thinking: Can a GPT Question-Answering System Turn Text into the Causal Maps Produced by Human Readers?
dc.typeConference Paper
dc.type.dcmiText
dcterms.abstractRepresenting a system as a network is critical to support systems thinking, hence several tools have been developed to derive networks from text in educational technology, modeling and simulation, or forecasting. Large-Scale Pre-Trained Language Models (PLMs) have recently come to the forefront to create question-answering systems (Q&A) that can extract networks from text. In this paper, we design and implement a Q&A system that uses GPT-3.5 together with 12 filters to extract causal maps text. Our evaluation on two topics via several policy documents finds that GPT accurately extracts relevant concept nodes but occasionally reverses causal directions and struggles with the type of causality as it lacks an understanding of event sequence. We also show that automatically extracted maps can only partially resemble human-made maps collected on the same topics. By making our Q&A system open-source on a permanent repository, researchers can evaluate it with newer PLMs as technology improves.
dcterms.extent10 pages
prism.startingpage7540

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
0736.pdf
Size:
986.75 KB
Format:
Adobe Portable Document Format
Download

Collections

Generative and Conversational AI in Information Systems Research and Education: Opportunities and Challenges