Comparing Performance Between Online and In-Person Research with an Audiovisual Experimental Paradigm

Abstract

Introduction: The COVID-19 pandemic has forced many researchers and educators to operate primarily from an online platform. While online information dissemination is fast, efficient, and inexpensive (Buhrmester, Talaifar, & Gosling, 2018), it is imperative that the data are reliable. One of the popular online platforms used is Zoom. Previously, psychometric studies found that web-based assessments disseminated via Zoom yield similar results to paper-based questionnaires (Riva, Teruzzi, & Anolli, 2003). However, the dissemination of information online may differ from in-person methods, as online platforms for presenting information have shown less precision and more variability in reaction times compared to in-person methods (Bridges et al., 2020). This could raise questions for the potential of using experimental methods online, as the utility of experimental studies via Zoom has yet to be evaluated.

Objectives: The present study aims to investigate the reliability of using Zoom with an audiovisual motion perception paradigm and compare performance between in-person and online presentations.

Methods: Participants met with a researcher either in-person or via Zoom to complete the experiment on Labvanced. Screenshare and remote control functions were used to share the paradigm and complete participation. The paradigm required participants to select an oddball video out of three videos. Reaction time, accuracy, and lag (ITI) was analyzed and compared across in-person versus Zoom to determine if these types of psychophysical experiments can be replicated online.

Results: Participant ages ranged from 18 to 52 years of age (M = 21.3, SD = 5.2). 35 participants were female, and 11 were male. In-person (n = 24) and online (n = 22) results were very similar in terms of reaction time, however, sound was shown to improve accuracy when detecting the oddball video for online participants. This differed from in-person results, as participants were less accurate when a sound was paired with the oddball video.

Discussion: The current study showed very little variation in reaction time for Zoom versus in-person performance, however, accuracy results with audiovisual stimuli differed. Given that the current study utilized Labvanced, a browser based platform, other studies have found variability in the precision of stimulus presentation via browser-based experiments (Bridges et al., 2020). Therefore, results from the current study fall in line existing literature, although studies investigating online presentation of stimuli suggest that online methods can be suitable for a wide range of applications in research and education as long as sources of variability are considered. However, a limitation of the current study is the small sample size and limited power due to the lack of participants. Future extensions of this study will include recruiting more participants to increase the sample size, as in-person recruitment for this study was interrupted by the COVID-19 pandemic.