{"id":20226969,"url":"https://github.com/ahmedheakl/multi-level-rl-for-robotics","last_synced_at":"2025-07-26T06:14:39.814Z","repository":{"id":55404582,"uuid":"523053431","full_name":"ahmedheakl/multi-level-rl-for-robotics","owner":"ahmedheakl","description":"A library for training robots using RL under the scheme of multi-level RL.","archived":false,"fork":false,"pushed_at":"2023-04-20T00:20:01.000Z","size":6951,"stargazers_count":10,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-14T22:04:53.046Z","etag":null,"topics":["continuous-integration","deep-reinforcement-learning","gym-environment","pyglet","python","reinforcement-learning","robotics","unittest"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ahmedheakl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-08-09T17:49:12.000Z","updated_at":"2024-11-20T02:25:36.000Z","dependencies_parsed_at":"2025-04-10T17:12:25.642Z","dependency_job_id":"73062c7c-e576-4b06-9ffa-4bf0363f8eb2","html_url":"https://github.com/ahmedheakl/multi-level-rl-for-robotics","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/ahmedheakl/multi-level-rl-for-robotics","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedheakl%2Fmulti-level-rl-for-robotics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedheakl%2Fmulti-level-rl-for-robotics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedheakl%2Fmulti-level-rl-for-robotics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedheakl%2Fmulti-level-rl-for-robotics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ahmedheakl","download_url":"https://codeload.github.com/ahmedheakl/multi-level-rl-for-robotics/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedheakl%2Fmulti-level-rl-for-robotics/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267127946,"owners_count":24040154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-26T02:00:08.937Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["continuous-integration","deep-reinforcement-learning","gym-environment","pyglet","python","reinforcement-learning","robotics","unittest"],"created_at":"2024-11-14T07:21:50.078Z","updated_at":"2025-07-26T06:14:39.793Z","avatar_url":"https://github.com/ahmedheakl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Main Tests](https://github.com/ahmedheakl/Mutli_Level_RL_Robotics/workflows/test/badge.svg)](https://github.com/ahmedheakl/Mutli_Level_RL_Robotics/actions)\n\n# Highrl: Multi-level Reinforcement Learning for Robotics Navigation\n\nHighrl is a library for training robots using RL under the scheme of multi-level RL. The library has numerous features from generating random environment, training agents to generate curriculum learning schemes, or train robots in pre-defined environments.\n\nThe robot can explore a synthetic [gym](https://www.gymlibrary.dev/) environment using lidar vision (either in flat or rings format). The robot is trained to reach the goal with obstacles in between to hinder its movement and simulate the real-world environment. A teacher (an RL agent) will be trained to synthesize the perfect curriculum learning for the robot, so that the robot will solve maps with certain difficulties in minimal time.  \n\nThe robot model is implemented with a [CNN](https://en.wikipedia.org/wiki/Convolutional_neural_network) for the feature extractor and an [MLP](https://en.wikipedia.org/wiki/Multilayer_perceptron) for both the value and policy networks. The teacher model is implemented with an LSTM network for the feature extractor and an MLP for value/policy network. The robot model is fed with the lidar data and outputs the velocity ```(vx, vy)``` of the robot. The teacher model is fed with data of the last session for the robot that the teacher is training, and outputs the configurations for the next environment to train the robot. At each step of the teacher, a new robot will be generated with probability of 10% and will be trained for a fixed number of steps. You can find the models in ```src/highrl/policy```. \n\n\n## Installation\n\nPlease note that the library is **only tested on Linux distributions**. If you want to install it on Windows, you can use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).\n\nUse the package manager [pip](https://pip.pypa.io/en/stable/) to install higrl library.\n\n```bash\npip install highrl\n```\n\n## Usage\n\n```bash\nhighrl -h # get available arguments\n```\n### Configurations\n**--robot-config**: path of configuration file of robot environment (relative path). \n\n**--teacher-config**: path of configuration file of teacher environment (relative path). \n\n**--mode**: choose train or test mode.\n\n**--env-mode**: choose whether to train the robot alone or in the presence of a teacher to generate curriculum learning. \n\n**--render-each**: the frequency of rendering for robot environment (integer).\n\n**--output-path**: relative path to output results for robot mode. \n\n**--lidar-mode**: mode to process lidar flat=1D, rings=2D.\n\n### Example\n```bash\nhighrl --render-each=50 --output-dir=~/Desktop\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg height=\"300\" width=\"500\" src=\"https://github.com/ahmedheakl/multi-level-rl-for-robotics/blob/main/imgs/highrl-img.png\"\u003e\n\u003c/p\u003e\n\n\n## Contributing\n\nPull requests are welcome. For major changes, please open an issue first\nto discuss what you would like to change.\n\nPlease make sure to update tests as appropriate.\n\n## License\n\n[MIT](https://choosealicense.com/licenses/mit/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedheakl%2Fmulti-level-rl-for-robotics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fahmedheakl%2Fmulti-level-rl-for-robotics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedheakl%2Fmulti-level-rl-for-robotics/lists"}