Reinforcement Learning for Adversarial Systems Using Relational Observations

Abstract

This paper investigates the integration of relational observations with the reinforcement learning (RL) framework for improved generalization capability. A hide-and-seek simulation environment is designed in Unity for proof-of-concept demonstration. Two observation representations—relational (analogical) and standard positional—are designed to evaluate agent learning and generalization capabilities. Agents are trained using the Proximal Policy Optimization (PPO) algorithm in a random-room environment and tested in both the random-room environment and a novel environment with greater spatial complexity and path obstructions. Comparative studies indicate that relational representation of objects in the adversarial environment could potentially improve the generalization capability of RL agents to novel and complex environments. Cross-testing results also suggest that relational observations may enhance agents’ effectiveness in pursuit and evasion tasks in adversarial environments.