A MIXED METHODS STUDY OF PRINCIPALS’ EXPERIENCE USING DATA ANALYTIC TOOLS IN HAWAI‘I

Abstract

The potential benefits of the growing amount of available educational data have not been fully realized, especially for principals, who play a key role in deriving insight from data to inform school improvement. The purpose of this mixed methods study was to describe experiences of principals using data analysis tools in a K‒12 system. With a two-phased explanatory sequential design anchored in activity theory, the researcher investigated the types of data and tools used by principals, how they were used, and what factors influenced such use. Data were collected through an online survey administered to all public school principals in Hawai‘i as well as a think-aloud protocol and semistructured interviews with a smaller group. Responses showed that principals used school process and perceptual data on a more frequent basis than student learning or demographic data despite the greater investment in tools for the latter. Factors that influenced use included: (a) data characteristics, such as quality and quantity; (b) school organizational characteristics, such as professional learning and time; (c) computer system characteristics, such as integration, accessibility, user interface, and functionality; and (d) user characteristics, such as disposition and skills. Trends emerged around the ways the tools were not as useful or usable because of a lack of the following: data visualization, interactive dashboards, customization, and technological advancements. Thus, school district administrators are urged to increase investment in advanced analytic tools that can support a greater variety of data types and ways to integrate data systems. Findings may be used by tool designers and developers to improve tools. Structured time and training are also needed to support users. Additional research is suggested for improved understanding of the mediating role of evolving digital tools in analysis activity in order to help educators leverage greater insights.