PosEmo – An automated system for measuring user interest and attitude in real time

Abstract

Given the increasing prevalence of digital services across various aspects of life, it has become crucial to understand and recognize the mental states of individuals interacting with artificial systems. To address this concern, we aimed to develop the PosEmo – an automated application that can assess individuals' affective states using a video web camera. While studying affective states, we focused on two kinds of emotional behavior: approach/avoidance behavior and behavioral freezing/activation. To measure these behaviors, we use computer vision techniques to track the movement of the participant's head in video recordings, as well as in real-time video streaming. This method offered the seated research participant convenience, replicability, and non-intrusiveness. Drawing from established theoretical frameworks and supported by initial empirical findings, we developed the software and validated it in the online experiment. We found that PosEmo recognized whether people watched negative, neutral, or positive videos. Thus, our innovative approach enables us to accurately estimate people's affective states. In sum, by adopting a human-centered approach, we combined artificial intelligence methodologies to create an innovative system supporting human-computer interaction. Our system's potential research applications span various domains, such as psychology, cognitive science, usability studies, psychotherapy sessions, content quality assessment, and education.