AI Agency Risks and Their Mitigation Through Business Process Management: A Conceptual Framework

Abstract

After over 60 years of research and development, AI has made its way into mainstream business operations. Continuous advances in the fields of machine learning, knowledge representation, and logical reasoning are expected to result in higher autonomy of AI-enabled systems such as Distributed AI (DAI) agents that can think and act. The increased agency of the AI systems is expected to result in agency risks and the need for mitigating such risks through AI governance. In this paper, we build on agency theory and identify factors that increase the risk of an agency problem between a principal (a human or an organization) and an AI agent and propose a framework for AI agency problem analysis. The framework is illustrated through AI use cases and industry examples. Implications for AI governance research and practice are discussed.