{"id":27944888,"url":"https://github.com/devslem/aine-drl","last_synced_at":"2025-05-07T12:55:02.014Z","repository":{"id":54566323,"uuid":"522193538","full_name":"DevSlem/AINE-DRL","owner":"DevSlem","description":"AINE-DRL is a deep reinforcement learning (DRL) baseline framework. AINE means \"Agent IN Environment\".","archived":false,"fork":false,"pushed_at":"2023-04-04T14:13:02.000Z","size":41239,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-07T12:55:01.158Z","etag":null,"topics":["ai","deep-learning","deep-reinforcement-learning","drl","gym","machine-learning","ml-agents","pomdp","ppo","python","pytorch","reinforcement-learning","rl","unity"],"latest_commit_sha":null,"homepage":"https://devslem.github.io/AINE-DRL/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DevSlem.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-07T11:26:22.000Z","updated_at":"2024-12-15T02:25:09.000Z","dependencies_parsed_at":"2023-02-19T10:01:24.886Z","dependency_job_id":"0c3f2334-a25e-4e10-a1f5-cb3f747c212c","html_url":"https://github.com/DevSlem/AINE-DRL","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevSlem%2FAINE-DRL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevSlem%2FAINE-DRL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevSlem%2FAINE-DRL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevSlem%2FAINE-DRL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DevSlem","download_url":"https://codeload.github.com/DevSlem/AINE-DRL/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252882789,"owners_count":21819154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deep-learning","deep-reinforcement-learning","drl","gym","machine-learning","ml-agents","pomdp","ppo","python","pytorch","reinforcement-learning","rl","unity"],"created_at":"2025-05-07T12:54:59.373Z","updated_at":"2025-05-07T12:55:02.007Z","avatar_url":"https://github.com/DevSlem.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AINE-DRL\n\n[AINE-DRL](https://github.com/DevSlem/AINE-DRL) is a deep reinforcement learning (DRL) baseline framework. **AINE** means \"Agent IN Environment\". If you want to know how to use, see [AINE-DRL Documentation](https://devslem.github.io/AINE-DRL/).\n\n| [Implementation](#implementation) | [Experiments](#experiments) | [Setup](#setup) |\n\n\u003e We always welcome your contributions! Please feel free to open an issue or pull request.\n\n## Implementation\n\nAINE-DRL provides below things:\n\n* deep reinforcement learning agents\n* compatible with [OpenAI Gym](https://github.com/openai/gym)\n* compatible with [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents)\n* inference (rendering, gif, picture)\n* model save/load\n* YAML configuration\n\nIf you're using AINE-DRL for the first time, please read [Getting Started](https://devslem.github.io/AINE-DRL/getting-started/).\n\n### Agent\n\nAINE-DRL provides deep reinforcement learning (DRL) agents. If you want to use them, it's helpful to read [Agent](https://devslem.github.io/AINE-DRL/agent) docs. \n\n|Agent|Source Code|\n|:---:|:---:|\n|[REINFORCE](https://devslem.github.io/AINE-DRL/agent/reinforce)|[reinforce](aine_drl/agent/reinforce/)|\n|[A2C](https://devslem.github.io/AINE-DRL/agent/a2c)|[a2c](aine_drl/agent/a2c/)|\n|[Double DQN](https://devslem.github.io/AINE-DRL/agent/double-dqn)|[dqn](aine_drl/agent/dqn/)|\n|[PPO](https://devslem.github.io/AINE-DRL/agent/ppo)|[ppo](aine_drl/agent/ppo)|\n|[Recurrent PPO](https://devslem.github.io/AINE-DRL/recurrent-ppo)|[ppo](aine_drl/agent/ppo/)|\n|[PPO RND](https://devslem.github.io/AINE-DRL/agent/ppo-rnd)|[ppo](aine_drl/agent/ppo/)|\n|[Recurrent PPO RND](https://devslem.github.io/AINE-DRL/agent/recurrent-ppo-rnd)|[ppo](aine_drl/agent/ppo/)|\n\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003ch3\u003eTODO\u003c/h3\u003e\u003c/summary\u003e\n\n- [ ] DDPG\n- [ ] Prioritized Experience Replay \n- [ ] SAC\n- [ ] Intrinsic Curiosity Module (ICM)\n- [ ] Random Network Distillation (RND)\n\n\u003c/details\u003e\n\n## Experiments\n\nYou can see our experiments (source code and result) in [experiments](experiments/). We show some recent experiments.\n\n### BipedalWalker-v3 with PPO\n\nTrain agents in OpenAI Gym [BipedalWalker-v3](https://github.com/openai/gym/wiki/BipedalWalker-v2) which is continuous action space task.\n\nFig 1. BipedalWalker-v3 inference (cumulative reward - PPO: 248):\n\n![](images/BipedalWalker-v3_PPO.gif)\n\n* [experiment](experiments/bipedal_walker_v3/)\n* [PPO configuration](config/experiments/bipedal_walker_v3_ppo.yaml)\n\nTo train the agent, enter the following command:\n\n```bash\npython experiments/bipedal_walker_v3/run.py\n```\n\nDetail options:\n\n```\nUsage:\n    experiments/bipedal_walker_v3/run.py [options]\n\nOptions:\n    -i --inference                Wheter to inference [default: False].\n```\n\nIf paging file error happens, see [Paging File Error](#paging-file-error).\n\n### CartPole-v1 with No Velocity\n\nCompare [Recurrent PPO](https://github.com/DevSlem/AINE-DRL/wiki/Recurrent-PPO) (using LSTM) and [Naive PPO](https://github.com/DevSlem/AINE-DRL/wiki/PPO) in OpenAI Gym [CartPole-v1](https://github.com/openai/gym/wiki/CartPole-v0) with No Velocity, which is [Partially Observable Markov Decision Process (POMDP)](https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process) setting. Specifically, we remove **\"cart velocity\"** and **\"pole velocity at tip\"** from the observation space. This experiment shows to require memory ability in POMDP setting.\n\nFig 2. [CartPole-v1 with No Velocity](https://github.com/openai/gym/wiki/CartPole-v0) inference (cumulative reward - Recurrent PPO: 500, Naive PPO: 41):\n\n|Recurrent PPO|Naive PPO|\n|:---:|:---:|\n|![](images/CartPole-v1-NoVelocity_RecurrentPPO.gif)|![](images/CartPole-v1-NoVelocity_NaivePPO.gif)|\n\nFig 3. [CartPole-v1 with No Velocity](https://github.com/openai/gym/wiki/CartPole-v0) cumulative reward (black: Recurrent PPO, cyan: Naive PPO):\n\n![](images/cartpole-v1-no-velocity-cumulative-reward.png)\n\n* [experiment](experiments/cartpole_v1_no_velocity/)\n* [Recurrent PPO configuration](config/experiments/cartpole_v1_no_velocity_recurrent_ppo.yaml)\n* [Naive PPO configuration](config/experiments/cartpole_v1_no_velocity_naive_ppo.yaml)\n\nTo train the Recurrent PPO agent, enter the following command:\n\n```bash\npython experiments/cartpole_v1_no_velocity/run.py\n```\n\nDetail options:\n\n```\nUsage:\n    experiments/cartpole_v1_no_velocity/run.py [options]\n\nOptions:\n    -a --agent=\u003cAGENT_NAME\u003e       Agent name (recurrent_ppo, naive_ppo) [default: recurrent_ppo].\n    -i --inference                Wheter to inference [default: False].\n```\n\n## Setup\n\nFollow the instructions.\n\n### Installation\n\nThis installation guide is simple. If you have a problem or want to see details, refer to [Installation](https://devslem.github.io/AINE-DRL/installation.html) docs.\n\nFirst, install [Python 3.9](https://www.python.org/) version.\n\nIf you want to use NVIDIA CUDA, install [PyTorch](https://pytorch.org/) with CDUA manually:\n\n```bash\npip install torch==1.11.0+cu113 --extra-index-url https://download.pytorch.org/whl/cu113\n```\n\nNow, install AINE-DRL package by entering the command below:\n\n```bash\npip install aine-drl\n```\n\n### Run\n\nRun a sample script in [samples](samples/) directory. Enter the following command:\n\n```bash\npython samples/\u003cFILE_NAME\u003e\n```\n\nExample:\n\n```bash\npython samples/cartpole_v1_ppo.py\n```\n\nSee details in [Getting Started](https://devslem.github.io/AINE-DRL/getting-started/) docs.\n\n### Paging File Error\n\nWhen you use too many workers (e.g., greater than 8), because of too many multi parallel environments in multi threads, **\"The paging file is too small for this operation to complete.\"** error may happen. If it happens, you can mitigate it using the command (Windows):\n\n```bash\npip install pefile\npython fixNvPe.py --input=C:\\\u003cAnaconda3 Path\u003e\\envs\\aine-drl\\Lib\\site-packages\\torch\\lib\\*.dll\n```\n\n`\u003cAnaconda3 Path\u003e` is one in which your Anaconda3 is installed.\n\nReference: [cobryan05/fixNvPe.py (Github)](https://gist.github.com/cobryan05/7d1fe28dd370e110a372c4d268dcb2e5)  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevslem%2Faine-drl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevslem%2Faine-drl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevslem%2Faine-drl/lists"}