Model-agnostic Trajectory Abstraction and Visualization Method for Explainability in Reinforcement Learning
Date
2024
Authors
Contributor
Advisor
Department
Instructor
Depositor
Speaker
Researcher
Consultant
Interviewer
Narrator
Transcriber
Annotator
Journal Title
Journal ISSN
Volume Title
Publisher
Volume
Number/Issue
Starting Page
Ending Page
Alternative Title
Abstract
Reinforcement learning (RL) has evolved rapidly in the past decade and is now capable of achieving human capabilities, such as self-driving cars. Moreover, in the last few years, the performance of deep RL, which applies deep neural networks to RL, has surpassed that of skilled human players in areas of video games, chess, and Go. However, as deep RL models become more complex, understanding and interpreting these models poses significant challenges.
Explainable AI (XAI) research has shown the potential to close the gap between humans and a deep RL agent by providing explanations that help users to understand how the agent works. XAI approaches have been tailored for both RL experts and non-experts. For RL experts, visualizations of internal agent parameters reveal the learning mechanisms of deep RL agents, offering precise insights into agent behavior. However, this approach is less accessible to users who do not have RL expertise (non-RL experts). The communication gap between RL experts and non-experts thus remains a critical issue. For example, in discussions about the decision boundaries of autonomous Unmanned Aerial Vehicles (UAVs) between RL practitioners and pilots, the following issues arise: Pilots, who are non-RL experts, have domain knowledge, but they cannot use XAI interfaces designed for RL experts in the assessment of the RL model; In order to obtain feedback from pilots, RL experts need to explain the behavior of the RL model while minimizing the use of RL terminology; Pilots may use domain specific terminology during the assessment and the RL expert needs to interpret the pilot's statements and apply them to the model;
Therefore, the central questions are: How can both RL experts and non-RL experts understand the behavior of an agent? In other words, how can humans naturally build a mental model of an agent? A promising approach is the 'familiarization effect' from cognitive psychology, where exposure to an agent's behavior in various scenarios helps users intuitively understand the agent, which is later applied to Human Robot Interaction. For instance, one research group observed that watching a robot’s trajectory in videos enables users to predict the robot's future trajectory. Another study pointed out that short video clips of an agents' game-play can effectively build mental models of the agents' performance. However, this strategy may be less effective with multiple agents or in complex, extended tasks due to human limitations in short-term visual memory.
Therefore, this dissertation addresses this problem by proposing a trajectory visualization that gives a high-level view of agents' behaviors through an abstraction of agents' behavior. This research will open up new directions, such as that domain experts who are not familiar with RL can get more involved in the development of RL which can lead to identifying important agent's behavior patterns that cannot be recognized by RL experts alone, and that the possibility of allowing general users to assess the capabilities and limitations of agents in the task of monitoring self-driving agents as a driver.
Description
Keywords
Computer science, Explainable AI, Human-Computer Interaction, Reinforcement Learning, Storyline Visualization, Trajectory Abstraction
Citation
Extent
104 pages
Format
Geographic Location
Time Period
Related To
Related To (URI)
Table of Contents
Rights
All UHM dissertations and theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission from the copyright owner.
Rights Holder
Local Contexts
Collections
Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.