Interpretability and Control in Forecasting Support Systems

Abstract

Forecasting Support Systems (FSS) exemplify the explainable artificial intelligence (XAI) challenge: their black-box algorithms frequently invite mistrust and harmful overrides. We experimentally compared three FSS designs to evaluate how algorithmic interpretability and control—two commonly proposed remedies—affect human–AI collaboration. We juxtaposed an Opaque baseline against an Interpretable variant that visualizes time-series decomposition and a Control variant that lets users re-parameterize components. In a controlled experiment (n=197) using real-world retail data, plain interpretability reduced the frequency and volume of judgmental adjustments and yielded a small but significant accuracy gain over the baseline. Adding component-level control, however, increased adjustment variance, did not improve average accuracy, and produced heavier error tails; self-reports indicated lower intuitiveness and satisfaction, consistent with higher perceived cognitive load. We conclude that interpretability helps calibrate users’ adjustments, whereas powerful control options introduce considerable risk of overconfident tinkering—insights directly relevant for FSS interface design and organizational governance of AI-assisted forecasting.