Rigorous Validation of Systems Security Engineering Analytics

Llanso, Thomas
Mcneil, Martha
Jamieson, Jessie
Journal Title
Journal ISSN
Volume Title
In response to the asymmetric advantage that attackers enjoy over defenders in cyber systems, the cyber community has generated a steady stream of cybersecurity-related frameworks, methodologies, analytics, and “best practices” lists. However, these artifacts almost never under-go rigorous validation of their efficacy but instead tend to be accepted on faith, to, we suggest, our collective detriment based on evidence of continued attacker success. But what would rigorous validation look like, and can we afford it? This paper describes the design and estimates the cost of a controlled experiment whose goal is to deter-mine the effectiveness of an exemplar systems security analytic. Given the significant footprint that humans play in cyber systems (e.g., their design, use, attack, and defense), any such experiment must necessarily take into account and control for variable human behavior. Thus, the paper reinforces the argument that cybersecurity can be understood as a hybrid discipline with strong technical and human dimensions.
Cybersecurity and Software Assurance, validation cybersecurity analytics experiment
Access Rights
Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.