Multisensory Effects of Causal Perception

Abstract

Previous research has shown that visual perception is influenced by Newtonian constraints. Kominsky et al. (2017) showed that humans can more efficiently detect unnatural collision events, where objects break Newtonian motion constraints by moving at a faster speed after colliding with another object, compared to collisions that do not violate Newtonian constraints. Their results provide evidence that the perceptual system distinguishes between realistic and unrealistic causal events. However, collisions between two objects are rarely silent in the real world. The present study extends this research by adding a sound at the collision point between two objects to evaluate how multisensory integration influences the perception of colliding events that either follow or violate Newtonian constraints. To accomplish this, participants viewed an array of three simultaneous videos, each depicting two moving objects. Two of the videos showed discs that moved at the same speed in a horizontal back and forth motion, and an oddball video of discs that either moved faster before the collision and slower after (natural) or slower before the collision and faster after (unnatural), violating Newtonian motion constraints in the latter. On each trial, a brief click was presented at the collision point of either the target video, one of the distractors, or none of the videos. Participants were asked to indicate the oddball video via keypress. Replicating Kominsky et al. (2017), participants were faster when identifying unnatural motion events compared to natural motion events both with and without sound. However, results also showed a lower accuracy rate for unnatural events compared to natural events, especially when a sound was added. These findings suggest that the addition of a sound could be distracting to participants, as a speed-accuracy trade-off is observed and exacerbated by sound. Furthermore, this provides evidence of the complexity of interactions that influence the human visual perceptual system and its ability to perceive causal events, as well as the limitations of attentional resources.