https://github.com/iamhatesz/rld

A development tool for evaluation and interpretability of reinforcement learning agents.
https://github.com/iamhatesz/rld

debugging-tools development-tools interpretability reinforcement-learning

Last synced: 6 months ago
JSON representation

A development tool for evaluation and interpretability of reinforcement learning agents.

• Host: GitHub
• URL: https://github.com/iamhatesz/rld
• Owner: iamhatesz
• License: mit
• Created: 2020-03-26T14:14:39.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2023-05-09T06:29:56.000Z (over 2 years ago)
• Last Synced: 2024-12-30T11:44:26.051Z (about 1 year ago)
• Topics: debugging-tools, development-tools, interpretability, reinforcement-learning
• Language: Python
• Homepage:
• Size: 1.45 MB
• Stars: 2
• Watchers: 2
• Forks: 1
• Open Issues: 17
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# ![rld logo](https://imgur.com/XlDt5Hi.png)

![Build and test](https://github.com/iamhatesz/rld/workflows/Build%20and%20test/badge.svg)

A development tool for evaluation and interpretability of reinforcement learning agents.

![rld demo gif](https://imgur.com/hodTIcj.gif)

## Installation

```bash

pip install rld

```

## Usage

Firstly, calculate attributations for your rollout using:

```bash

rld attribute [--rllib] [--out ] config.py 

```

This will take `INPUT_ROLLOUT` (possibly in the Ray RLlib format, if `--rllib` is set)

and calculate attributations for each timestep in each trajectory,

using the configuration stored in `config.py`.

The output file will be stored as `ROLLOUT`.

See the `Config` class for possible configuration.

Once the attributations are calculated, you can visualize them using:

```bash

rld start --viewer  

```

See the [examples](./examples) for reference.

## Description

rld provides a set of tools to evaluate and understand behaviors of reinforcement

learning agents. Under the hood, rld uses [Captum](https://captum.ai/) to calculate

attributations of observation components. rld is also integrated with

[Ray RLlib](https://ray.io/) library and allows to load agents trained in RLlib.

### Current limitations

rld is currently in its early development stage, thus the functionality is very limited.

#### RL algorithms

rld is algorithm-agnostic, but currently it is more suitable for policy-based methods.

This is due to the fact that the `Model` is now expected to output logits for a given

observation. This, however, will change in the future, and rld will support more

algorithms.

#### Viewers

This is the list of viewers, which ship with rld:

* `none`

* `cartpole`

* `atari`

You can easily create your own viewer, for your own environment, but to make it visible

for rld, you have to rebuild the project. This will be improved in the future.

#### Observation and action spaces

The table below presents currently supported observation and action spaces.

    

        

        

        Action space

    

    

        

        

        Discrete

        MultiDiscrete

    

    

        Obs space

        Box

        :heavy_check_mark:

        :heavy_check_mark:

    

    

        Dict

        :heavy_check_mark:

        :heavy_check_mark:

    

## Roadmap

See the [issues](https://github.com/iamhatesz/rld/issues) page to see the list of

features planned for the future releases. If you have your own ideas,

you are encouraged to post them there.