Detecting Persuasion Techniques in Online Propaganda: An AI-annotation Intervention

Abstract

In today's digital world, online propaganda poses a growing threat to society. To address this challenge, we explored the potential of a real-time training intervention. This intervention exposes people to manipulative text identified by AI and provides cognitive guidance to help them recognize and critically analyze similar cases. To examine whether people trust AI technology enough to read the annotations and follow them, we conducted a one-factor experiment. Our findings show no difference in training performance whether the annotations are claimed to be provided by domain experts or by AI. The intervention is effective in improving people’s ability to identify propagandistic texts and persuasion techniques, and performance varies by technique. We also found that people judge the texts based on one or more of three aspects: the content of the material, the author’s intention inferred from the material, and their personal ideology related to the topic of the material.