Synthetic APIs: Enabling Language Models to Act as Interlocutors Between Natural Language and Code

Simple item page
dc.contributor.authorMullins, Ryan
dc.contributor.authorTerry, Michael
dc.date.accessioned2022-12-27T18:54:25Z
dc.date.available2022-12-27T18:54:25Z
dc.date.issued2023-01-03
dc.description.abstractLarge language models (LLMs) can synthesize code from natural language descriptions or by completing code in-context. In this paper, we consider the ability of LLMs to synthesize code, at inference time, for a novel API not in its training data, and specifically examine the impact of different API designs on this ability. We find that: 1) code examples in model training data seem to facilitate API use at inference time; 2) hallucination is the most common failure mode; and 3) the designs of both the novel API and the prompt affect performance. In light of these findings, we introduce the concept of a Synthetic API: an API designed to be used by LLMs instead of by humans. Synthetic APIs for LLMs offer the potential to further accelerate development of natural language interfaces to arbitrary tools and services.
dc.format.extent10
dc.identifier.doi10.24251/HICSS.2023.073
dc.identifier.isbn978-0-9981331-6-4
dc.identifier.other9c8063f3-03c9-4057-b68e-19cecaad4428
dc.identifier.urihttps://hdl.handle.net/10125/102701
dc.language.isoeng
dc.relation.ispartofProceedings of the 56th Hawaii International Conference on System Sciences
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0/
dc.subjectText Mining and Analytics
dc.subjectapi design
dc.subjectcode synthesis
dc.subjecthuman-ai interaction
dc.subjectlarge language models
dc.titleSynthetic APIs: Enabling Language Models to Act as Interlocutors Between Natural Language and Code
dc.type.dcmitext
prism.startingpage565

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
0055.pdf
Size:
221.58 KB
Format:
Adobe Portable Document Format
Download

Collections

Text Mining and Analytics