Is the Larger the Better? An Exploratory Study into Human-Large Language Model Collaboration

Abstract

Large language models (LLMs) have garnered considerable attention in both academics and industry. Given an array of LLMs available, one of the primary challenges lies in their selection and adaptation strategies. Although LLMs are generally large, they still vary significantly in size and larger LLMs consume significantly more computing resources. This prompts the inquiry into whether larger models perform better. In addition, there is a widespread recognition of the power of LLMs in performing open-ended or generative tasks. However, how to use LLMs to address a closed-ended problem remains under explored. The exploration of human-LLM collaboration on close-ended problem has been even more sparse. This research aims to address the above limitations by comparing different types of state-of-the-art adaptation strategies for LLMs, including in-context learning and fine-tuning. Moreover, it employs multi-class multi-label classification - a close-ended problem to empirically evaluate those adaptation strategies. The research findings provide valuable insights and recommendations for human users considering deploying LLMs for close-ended problems.