Optimization of protocols

Abstract

The development in capabilities of artificial intelligence brings the increased participation of intelligent machines in the protocols of computer networks and society, playing some roles earmarked for machines and others ripe for deception. This exacerbates existing concerns, and it introduces a new dimension to the problems of privacy \& security. Whereas a cryptographic protocol can be analyzed with formal methods in terms of the properties of traces it produces, the probabilistic protocols involving potentially deceitful AI participants are analyzed in terms of probability distributions over its traces. In contrast with formal specifications of explicit requirements, the requirements for such AI protocols are specified informally and implicitly by reward models trained on data. A mathematical model of protocol post-training is proposed in terms of an objective function defined by such rewards and regularized by statistical distances from the pre-trained behaviors. It is shown that any instance of such a protocol post-training problem admits solutions at a level of generality that does not depend on particular details of algorithms or computational paradigms, thus showing the existence of optimal behaviors that learning algorithms aim to represent in a way that applies to reinforcement learning algorithms and algorithms in any other paradigm of learning. This establishes the proposed model of protocol post-training as a general setting for reasoning about the opportunities and limitations of protocols involving AI actors.