Does It Make a Difference? Data Vizualizations and the Use of Research and Evaluation Reports.

Abstract

Evaluation use has transfixed the evaluation community since its inception: How do we ensure that the good work we do as evaluators results in something more than a report? Drawing from research on cognition, a new crop of evaluators argued that data visualizations promote the use of evaluations, following the theory that the visuals engage and aid stakeholders in making sense of information (Evergreen, 2011a). This study builds on that theory to explore if the use and quality of data visualizations in research and evaluation reports increases the likelihood that reports will be used. Use, in this case, is an individual referencing a research or evaluation report in legislative testimony on teacher quality. Because use is multifaceted and slippery, I also looked at alternative predictors of use including the length of reports, if they were more like “advocacy” research or “traditional” research, and if the user was affiliated with a university. Using a Poisson regression with frequency of use as the dependent variable, I did not find a relationship between the use or quality of data visualizations and use of reports. However, I did find predictive relationships between the type and length of reports and frequency of use, where longer reports with data visualizations were less likely to be used and reports that were characteristic of advocacy research, (i.e. based on anecdotal evidence, lacking an objective tone, etc.) were more likely to be used.