Harnessing Complexity in High Performance Computing Ecosystems: A Complex Adaptive Systems Framework

Abstract

The use of high performance computing (HPC) has been generating influential scientific breakthroughs since the twentieth century. Yet there have been few studies of the complex socio-technical systems formed by these supercomputers and the humans who operate and use them. In this paper, we describe the first complex adaptive systems (CAS) analysis of the dynamics of HPC ecosystems. We conducted an 18-month ethnographic study that included scientific collaborations that use an HPC research center and examined the processes in HPC socio-technical systems via CAS theory to devise organizational designs and strategies that take advantage of system complexity. We uncovered several significant mismatches in the variation and adaptation processes within subsystems and conclude with three potential design directions for management and organization of HPC socio-technical ecosystems.