PokerFace Mask: Exploring Augmenting Masks with Captions through an Interactive, Mixed-Reality Prototype

Abstract

The COVID-19 pandemic in early 2020 made masks a daily wearable for personal protective equipment as a public health precaution. Traditional mask designs obscure communication by obstructing the face and muffling the voice which can make communication especially difficult for users who are deaf or hard of hearing (DHH). PokerFace uses a commodity smartphone and recycled materials to display a live-stream of a user’s mouth and nose on the mask surface. This maintains the safety precautions afforded by the mask, while mitigating the obfuscation of traditional mask designs. To compare PokerFace’s ability to facilitate communication with traditional masks, we conducted a user study with 18 participants, who played a collaborative communication game similar to charades. Participants performed better at this collaborative communication task with our prototype than with traditional masks, and even non-DHH users became aware of the importance of lip-reading and facial cues in communication due to study participation.