{"id":16271669,"url":"https://github.com/jianzhnie/deep-rl-toolkit","last_synced_at":"2025-09-06T13:32:14.596Z","repository":{"id":223592389,"uuid":"760304174","full_name":"jianzhnie/deep-rl-toolkit","owner":"jianzhnie","description":"RLToolkit is a flexible and high-efficient reinforcement learning framework. Include implementation of DQN, AC,A2C, A3C, PG, DDPG, TRPO, PPO, SAC, TD3 and ....","archived":false,"fork":false,"pushed_at":"2024-10-30T07:09:26.000Z","size":549,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-29T13:44:24.479Z","etag":null,"topics":["actor-critic","atari","ddpg","deep-reinforcement-learning","dqn","gym","mujoco","ppo","sac","td3","trpo"],"latest_commit_sha":null,"homepage":"https://jianzhnie.github.io/llmtech/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jianzhnie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-20T06:59:26.000Z","updated_at":"2024-11-18T06:08:35.000Z","dependencies_parsed_at":"2024-03-18T03:28:47.141Z","dependency_job_id":"ee4ebb52-e415-48d5-84c0-a534c38acd2f","html_url":"https://github.com/jianzhnie/deep-rl-toolkit","commit_stats":null,"previous_names":["jianzhnie/deep-rl-toolkit"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fdeep-rl-toolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fdeep-rl-toolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fdeep-rl-toolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fdeep-rl-toolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jianzhnie","download_url":"https://codeload.github.com/jianzhnie/deep-rl-toolkit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232126168,"owners_count":18476190,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actor-critic","atari","ddpg","deep-reinforcement-learning","dqn","gym","mujoco","ppo","sac","td3","trpo"],"created_at":"2024-10-10T18:14:21.880Z","updated_at":"2025-01-01T20:59:40.217Z","avatar_url":"https://github.com/jianzhnie.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep-RL-Toolkit\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"docs/images/logo.png\" alt=\"logo\" width=\"1000\"/\u003e\n\u003c/p\u003e\n\n## Overview\n\nDeep RL Toolkit is a flexible and high-efficient reinforcement learning framework. RLToolkit is developed for practitioners with the following advantages:\n\n- **Reproducible**. We provide algorithms that stably reproduce the result of many influential reinforcement learning algorithms.\n\n- **Extensible**. Build new algorithms quickly by inheriting the abstract class in the framework.\n\n- **Reusable**.  Algorithms provided in the repository could be directly adapted to a new task by defining a forward network and training mechanism will be built automatically.\n\n- **Elastic**: allows to elastically and automatically allocate computing resources on the cloud.\n\n- **Lightweight**: the core codes \\\u003c1,000 lines (check [Demo](examples/cleanrl/cleanrl_runner.py)).\n\n- **Stable**: much more stable than [Stable Baselines 3](https://github.com/DLR-RM/stable-baselines3) by utilizing various ensemble methods.\n\n## Table of Content\n\n- [Deep-RL-Toolkit](#deep-rl-toolkit)\n  - [Overview](#overview)\n  - [Table of Content](#table-of-content)\n  - [Supported Algorithms](#supported-algorithms)\n  - [Supported Envs](#supported-envs)\n  - [Examples](#examples)\n    - [Quick Start](#quick-start)\n  - [References](#references)\n    - [Reference Papers](#reference-papers)\n    - [References code](#references-code)\n\n## Supported Algorithms\n\nRLToolkit implements the following model-free deep reinforcement learning (DRL) algorithms:\n\n![../_images/rl_algorithms_9_15.svg](https://spinningup.openai.com/en/latest/_images/rl_algorithms_9_15.svg)\n\n## Supported Envs\n\n- **OpenAI Gym**\n- **Atari**\n- **MuJoCo**\n- **PyBullet**\n\nFor the details of DRL algorithms, please check out the educational webpage [OpenAI Spinning Up](https://spinningup.openai.com/en/latest/).\n\n## Examples\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/jianzhnie/RLToolkit/blob/main/docs/images/trained.gif\" alt=\"logo\" width=\"810\"/\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/jianzhnie/RLToolkit/blob/main/examples/tutorials/assets/img/breakout.gif\" width = \"200\" height =\"200\"/\u003e\n\u003cimg src=\"https://github.com/jianzhnie/RLToolkit/blob/main/examples/tutorials/assets/img/spaceinvaders.gif\" width = \"200\" height =\"200\"/\u003e\n\u003cimg src=\"https://github.com/jianzhnie/RLToolkit/blob/main/examples/tutorials/assets/img/seaquest.gif\" width = \"200\" height =\"200\"/\u003e\n\u003cimg src=\"https://github.com/jianzhnie/RLToolkit/blob/main/docs/images/Breakout.gif\" width = \"200\" height =\"200\" alt=\"Breakout\"/\u003e\n\u003cbr\u003e\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/jianzhnie/RLToolkit/blob/main/docs/images/performance.gif\" width = \"265\" height =\"200\" alt=\"NeurlIPS2018\"/\u003e\n\u003cimg src=\"https://github.com/jianzhnie/RLToolkit/blob/main/docs/images/Half-Cheetah.gif\" width = \"265\" height =\"200\" alt=\"Half-Cheetah\"/\u003e\n\u003cimg src=\"https://github.com/jianzhnie/RLToolkit/blob/main/examples/tutorials/assets/img/snowballfight.gif\" width = \"265\" height =\"200\"/\u003e\n\u003cbr\u003e\n\nIf you want to learn more about deep reinforcemnet learning, please read the [deep-rl-class](https://jianzhnie.github.io/llmtech/) and run the [examples](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples).\n\n- [Classic Control](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/discrete)\n- [Atari Benchmark](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/atari)\n- [Box2d Benchmark](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/box2d)\n- [Mujuco Benchmark](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/mujoco)\n- [Petting Zoo](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/pettingzoo)\n\n### Quick Start\n\n```bash\ngit clone https://github.com/jianzhnie/deep-rl-toolkit.git\n\n# Run the DQN algorithm on the CartPole-v0 environment\npython examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo dqn\npython examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo ddqn\npython examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo dueling_dqn\npython examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo dueling_ddqn\n\n# Run the C51 algorithm on the CartPole-v0 environment\npython examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo c51\n\n# Run the DDPG algorithm on the Pendulum-v1 environment\npython examples/cleanrl/cleanrl_runner.py --env Pendulum-v0 --algo ddpg\n\n# Run the PPO algorithm on the CartPole-v0 environment\npython examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo ppo\n```\n\n## References\n\n### Reference Papers\n\n01. Deep Q-Network (DQN) \u003csub\u003e\u003csup\u003e ([V. Mnih et al. 2015](https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf)) \u003c/sup\u003e\u003c/sub\u003e\n02. Double DQN (DDQN) \u003csub\u003e\u003csup\u003e ([H. Van Hasselt et al. 2015](https://arxiv.org/abs/1509.06461)) \u003c/sup\u003e\u003c/sub\u003e\n03. Advantage Actor Critic (A2C)\n04. Vanilla Policy Gradient (VPG)\n05. Natural Policy Gradient (NPG) \u003csub\u003e\u003csup\u003e ([S. Kakade et al. 2002](http://papers.nips.cc/paper/2073-a-natural-policy-gradient.pdf)) \u003c/sup\u003e\u003c/sub\u003e\n06. Trust Region Policy Optimization (TRPO) \u003csub\u003e\u003csup\u003e ([J. Schulman et al. 2015](https://arxiv.org/abs/1502.05477)) \u003c/sup\u003e\u003c/sub\u003e\n07. Proximal Policy Optimization (PPO) \u003csub\u003e\u003csup\u003e ([J. Schulman et al. 2017](https://arxiv.org/abs/1707.06347)) \u003c/sup\u003e\u003c/sub\u003e\n08. Deep Deterministic Policy Gradient (DDPG) \u003csub\u003e\u003csup\u003e ([T. Lillicrap et al. 2015](https://arxiv.org/abs/1509.02971)) \u003c/sup\u003e\u003c/sub\u003e\n09. Twin Delayed DDPG (TD3) \u003csub\u003e\u003csup\u003e ([S. Fujimoto et al. 2018](https://arxiv.org/abs/1802.09477)) \u003c/sup\u003e\u003c/sub\u003e\n10. Soft Actor-Critic (SAC) \u003csub\u003e\u003csup\u003e ([T. Haarnoja et al. 2018](https://arxiv.org/abs/1801.01290)) \u003c/sup\u003e\u003c/sub\u003e\n11. SAC with automatic entropy adjustment (SAC-AEA) \u003csub\u003e\u003csup\u003e ([T. Haarnoja et al. 2018](https://arxiv.org/abs/1812.05905)) \u003c/sup\u003e\u003c/sub\u003e\n\n### References code\n\n- rllib\n\n  - https://github.com/ray-project/ray\n  - https://docs.ray.io/en/latest/rllib/index.html\n\n- coach\n\n  - https://github.com/IntelLabs/coach\n  - https://intellabs.github.io/coach\n\n- Pearl\n\n  - https://github.com/facebookresearch/Pearl\n  - https://pearlagent.github.io/\n\n- tianshou\n\n  - https://github.com/thu-ml/tianshou\n  - https://tianshou.org/en/stable/\n\n- stable-baselines3\n\n  - https://github.com/DLR-RM/stable-baselines3\n  - https://stable-baselines3.readthedocs.io/en/master/\n\n- PARL\n\n  - https://github.com/PaddlePaddle/PARL\n  - https://parl.readthedocs.io/zh-cn/latest/\n\n- openrl\n\n  - https://github.com/OpenRL-Lab/openrl/\n  - https://openrl-docs.readthedocs.io/zh/latest/\n\n- cleanrl\n\n  - https://github.com/vwxyzjn/cleanrl\n  - https://docs.cleanrl.dev/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjianzhnie%2Fdeep-rl-toolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjianzhnie%2Fdeep-rl-toolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjianzhnie%2Fdeep-rl-toolkit/lists"}