Online-Computer-Mediated Interviews and Observations: Overcoming Challenges and Establishing Best Practices in a Human-AI Teaming Context

Abstract

Prior research has established the feasibility of conducting online interviews and observations, yet there is limited guidance in how to interact with participants when conducting fully mediated research with screen-sharing and video. This study, conducted during early phases of COVID-19, included 15 volunteer tweet annotators working with an emergency response organization. This method contribution uses cues-related and surveillance theories to reveal challenges and best practices when asking research participants to share their screen, be on video, and participate in a multiple-interview study. The findings suggest that researchers conducting online-mediated research should be prepared to provide technical support for the devices and interfaces participants use during the study, find ways to “see” beyond what is on the mediated screen, and consider ethical issues not often discussed. In addition to these findings, an output of this research is two brief training videos useful for other researchers embarking on conducting fully mediated research.