Transforming Generative Large Language Models' Limitations into Strengths using Gestalt: A Synergetic Approach to Mathematical Problem-Solving with Computational Engines
Transforming Generative Large Language Models' Limitations into Strengths using Gestalt: A Synergetic Approach to Mathematical Problem-Solving with Computational Engines
Files
Date
2024-01-03
Authors
Dunn, Cayden
Hashemi Tonekaboni, Navid
Contributor
Advisor
Department
Instructor
Depositor
Speaker
Researcher
Consultant
Interviewer
Annotator
Journal Title
Journal ISSN
Volume Title
Publisher
Volume
Number/Issue
Starting Page
5185
Ending Page
Alternative Title
Abstract
This paper presents an innovative approach, known as Gestalt, to enhance the mathematical problem-solving capabilities of Generative Large Language Models (GLLMs) while addressing their inherent limitations. Recognizing the inherent structure and discerning strength of GLLMs, the core of our approach strategically offloads computations, deterministic questions, and knowledge retrieval to external tools such as Wolfram Alpha and Python REPL. This critical augmentation not only mitigates GLLMs' variable reliability in these areas but also fortifies their innate strength - understanding the underlying structure of the problems at hand. With this novel implementation, GLLMs can harness the potential of external systems through well-structured queries, enabling them to make significant strides in problem-solving. In a preliminary evaluation, the Gestalt system demonstrates exceptional performance on a portion of the MATH benchmark dataset, achieving a state-of-the-art accuracy of 59.00%. In comparison, GPT-4 achieves an accuracy of 53.9% on the identical dataset. Through our augmentation approach, we aim to transform the limitations of GLLMs into their strengths, opening up exciting new possibilities not only in advanced mathematical problem-solving but also in various deterministic tasks such as medical diagnosis.
Description
Keywords
Design and Appropriation of Knowledge, Chatbot and Other AI Systems,
artificial intelligence (ai),
code interpreter,
generative large language models (gllms),
mathematical problem-solving,
natural language processing (nlp)
Citation
Extent
10 pages
Format
Geographic Location
Time Period
Related To
Proceedings of the 57th Hawaii International Conference on System Sciences
Table of Contents
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International
Rights Holder
Local Contexts
Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.