HALO: A Hierarchical and Adaptive Large Language Model Framework with Centralized Feedback

Abstract

The rapid growth of Large Language Models (LLMs) has driven advances across many NLP tasks, but their size and computational demands hinder deployment on resource-constrained client devices, from laptops to small-GPU desktops. At the same time, expanding edge applications and privacy concerns demand on-device intelligence that is efficient, adaptive, and personalized. Existing techniques such as pruning and quantization address parts of this challenge but fall short of balancing accuracy, hardware diversity, user customization, and long-term adaptability. In this paper, we present HALO (Hierarchical and Adaptive LLM Framework with Centralized Feedback), a system for lightweight and continuously improving edge deployment. HALO profiles device hardware, selects and fine-tunes suitable LLM variants with LoRA and quantization, and leverages user feedback to flag underperforming outputs for centralized correction and retraining. Updated adapters are then periodically redistributed, enabling sustained accuracy, efficiency, and privacy across heterogeneous edge devices.