Why do Large Language Models Judge Differently than Humans? An Examination of Sentiment Analysis of Movie Reviews

Abstract

This research investigates the root causes of divergence between Large Language Model (LLM)-based and human sentiment judgments. Using an inductive approach, we qualitatively analyzed a movie review dataset and identify two main causes: (i) contextual statements, where sentiment depends on situational factors (e.g., describing a film as “childish” may be positive for younger audiences but negative for adults); and (ii) linguistic statements, where sentiment shifts due to complex constructions such as sarcasm or double negation. Our study thus highlights the importance of both context (where, when, and for whom a statement is made) and linguistic form (how it is phrased) in sentiment interpretation. We contribute to the literature by identifying justificatory mechanisms behind differences in sentiment judgments between humans and LLMs. This may initiate a broader discourse on whether machine-generated sentiment can serve as a valid proxy for human interpretation. Even more, human-in-the-middle approaches may still outperform solely LLM-based sentiment interpretations.