{"id":20119057,"url":"https://github.com/iamhatesz/rld","last_synced_at":"2025-07-18T10:03:46.127Z","repository":{"id":50232302,"uuid":"250277493","full_name":"iamhatesz/rld","owner":"iamhatesz","description":"A development tool for evaluation and interpretability of reinforcement learning agents.","archived":false,"fork":false,"pushed_at":"2023-05-09T06:29:56.000Z","size":1521,"stargazers_count":2,"open_issues_count":17,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-30T11:44:26.051Z","etag":null,"topics":["debugging-tools","development-tools","interpretability","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iamhatesz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-03-26T14:14:39.000Z","updated_at":"2024-11-06T14:50:16.000Z","dependencies_parsed_at":"2023-12-15T22:00:17.539Z","dependency_job_id":null,"html_url":"https://github.com/iamhatesz/rld","commit_stats":{"total_commits":137,"total_committers":1,"mean_commits":137.0,"dds":0.0,"last_synced_commit":"e966d663fc419032b7502f9359a034e66007c31c"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamhatesz%2Frld","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamhatesz%2Frld/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamhatesz%2Frld/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamhatesz%2Frld/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iamhatesz","download_url":"https://codeload.github.com/iamhatesz/rld/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233704837,"owners_count":18717027,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["debugging-tools","development-tools","interpretability","reinforcement-learning"],"created_at":"2024-11-13T19:14:09.030Z","updated_at":"2025-01-13T07:09:32.202Z","avatar_url":"https://github.com/iamhatesz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ![rld logo](https://imgur.com/XlDt5Hi.png)\n\n![Build and test](https://github.com/iamhatesz/rld/workflows/Build%20and%20test/badge.svg)\n\nA development tool for evaluation and interpretability of reinforcement learning agents.\n\n![rld demo gif](https://imgur.com/hodTIcj.gif)\n\n## Installation\n\n```bash\npip install rld\n```\n\n## Usage\n\nFirstly, calculate attributations for your rollout using:\n\n```bash\nrld attribute [--rllib] [--out \u003cROLLOUT\u003e] config.py \u003cINPUT_ROLLOUT\u003e\n```\n\nThis will take `INPUT_ROLLOUT` (possibly in the Ray RLlib format, if `--rllib` is set)\nand calculate attributations for each timestep in each trajectory,\nusing the configuration stored in `config.py`.\nThe output file will be stored as `ROLLOUT`.\nSee the `Config` class for possible configuration.\n\nOnce the attributations are calculated, you can visualize them using:\n\n```bash\nrld start --viewer \u003cVIEWER_ID\u003e \u003cROLLOUT\u003e\n```\n\nSee the [examples](./examples) for reference.\n\n## Description\n\nrld provides a set of tools to evaluate and understand behaviors of reinforcement\nlearning agents. Under the hood, rld uses [Captum](https://captum.ai/) to calculate\nattributations of observation components. rld is also integrated with\n[Ray RLlib](https://ray.io/) library and allows to load agents trained in RLlib.\n\n### Current limitations\n\nrld is currently in its early development stage, thus the functionality is very limited.\n\n#### RL algorithms\n\nrld is algorithm-agnostic, but currently it is more suitable for policy-based methods.\nThis is due to the fact that the `Model` is now expected to output logits for a given\nobservation. This, however, will change in the future, and rld will support more\nalgorithms.\n\n#### Viewers\n\nThis is the list of viewers, which ship with rld:\n* `none`\n* `cartpole`\n* `atari`\n\nYou can easily create your own viewer, for your own environment, but to make it visible\nfor rld, you have to rebuild the project. This will be improved in the future.\n\n#### Observation and action spaces\n\nThe table below presents currently supported observation and action spaces.\n\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003c/td\u003e\n        \u003ctd\u003e\u003c/td\u003e\n        \u003ctd colspan=\"2\"\u003e\u003cstrong\u003eAction space\u003c/strong\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003c/td\u003e\n        \u003ctd\u003e\u003c/td\u003e\n        \u003ctd\u003eDiscrete\u003c/td\u003e\n        \u003ctd\u003eMultiDiscrete\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd rowspan=\"3\"\u003e\u003cstrong\u003eObs space\u003c/strong\u003e\u003c/td\u003e\n        \u003ctd\u003eBox\u003c/td\u003e\n        \u003ctd\u003e:heavy_check_mark:\u003c/td\u003e\n        \u003ctd\u003e:heavy_check_mark:\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003eDict\u003c/td\u003e\n        \u003ctd\u003e:heavy_check_mark:\u003c/td\u003e\n        \u003ctd\u003e:heavy_check_mark:\u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n## Roadmap\n\nSee the [issues](https://github.com/iamhatesz/rld/issues) page to see the list of\nfeatures planned for the future releases. If you have your own ideas,\nyou are encouraged to post them there.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamhatesz%2Frld","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiamhatesz%2Frld","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamhatesz%2Frld/lists"}