Model-agnostic Trajectory Abstraction and Visualization Method for Explainability in Reinforcement Learning

Date

2024

Contributor

Instructor

Depositor

Speaker

Researcher

Consultant

Interviewer

Narrator

Transcriber

Annotator

Journal Title

Journal ISSN

Volume Title

Publisher

Volume

Number/Issue

Starting Page

Ending Page

Alternative Title

Abstract

Reinforcement learning (RL) has evolved rapidly in the past decade and is now capable of achieving human capabilities, such as self-driving cars. Moreover, in the last few years, the performance of deep RL, which applies deep neural networks to RL, has surpassed that of skilled human players in areas of video games, chess, and Go. However, as deep RL models become more complex, understanding and interpreting these models poses significant challenges. Explainable AI (XAI) research has shown the potential to close the gap between humans and a deep RL agent by providing explanations that help users to understand how the agent works. XAI approaches have been tailored for both RL experts and non-experts. For RL experts, visualizations of internal agent parameters reveal the learning mechanisms of deep RL agents, offering precise insights into agent behavior. However, this approach is less accessible to users who do not have RL expertise (non-RL experts). The communication gap between RL experts and non-experts thus remains a critical issue. For example, in discussions about the decision boundaries of autonomous Unmanned Aerial Vehicles (UAVs) between RL practitioners and pilots, the following issues arise: Pilots, who are non-RL experts, have domain knowledge, but they cannot use XAI interfaces designed for RL experts in the assessment of the RL model; In order to obtain feedback from pilots, RL experts need to explain the behavior of the RL model while minimizing the use of RL terminology; Pilots may use domain specific terminology during the assessment and the RL expert needs to interpret the pilot's statements and apply them to the model; Therefore, the central questions are: How can both RL experts and non-RL experts understand the behavior of an agent? In other words, how can humans naturally build a mental model of an agent? A promising approach is the 'familiarization effect' from cognitive psychology, where exposure to an agent's behavior in various scenarios helps users intuitively understand the agent, which is later applied to Human Robot Interaction. For instance, one research group observed that watching a robot’s trajectory in videos enables users to predict the robot's future trajectory. Another study pointed out that short video clips of an agents' game-play can effectively build mental models of the agents' performance. However, this strategy may be less effective with multiple agents or in complex, extended tasks due to human limitations in short-term visual memory. Therefore, this dissertation addresses this problem by proposing a trajectory visualization that gives a high-level view of agents' behaviors through an abstraction of agents' behavior. This research will open up new directions, such as that domain experts who are not familiar with RL can get more involved in the development of RL which can lead to identifying important agent's behavior patterns that cannot be recognized by RL experts alone, and that the possibility of allowing general users to assess the capabilities and limitations of agents in the task of monitoring self-driving agents as a driver.

Description

Keywords

Computer science, Explainable AI, Human-Computer Interaction, Reinforcement Learning, Storyline Visualization, Trajectory Abstraction

Citation

Extent

104 pages

Format

Geographic Location

Time Period

Related To

Related To (URI)

Table of Contents

Rights

All UHM dissertations and theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission from the copyright owner.

Rights Holder

Local Contexts

Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.