Crowdsourcing Privacy Design Critique: An Empirical Evaluation of Framing Effects

Abstract

When designed incorrectly, information systems can thwart people’s expectations of privacy. An emerging technique for evaluating systems during the development stage is the crowdsourcing design critique, in which design evaluations are sourced using crowdsourcing platforms. However, we know that information framing has a serious effect on decision-making and can steer design critiques in one way or another. We investigate how the framing of design cases can influence the outcomes of privacy design critiques. Specifically, we test whether -˜Personas’, a central User-Centered Design tool for describing users, can inspire empathy in users while criticizing privacy designs. In an experiment on Amazon Mechanical Turk workers (n=456), we show that describing design cases by using personas causes intrusive designs to be criticized more harshly. We discuss how our results can be used to enhance privacy-by-design processes and encourage user-centered privacy engineering.