{"id":20827374,"url":"https://github.com/gokulp01/meta-qlearning-humanoid","last_synced_at":"2025-10-19T08:13:17.494Z","repository":{"id":186638235,"uuid":"675464651","full_name":"gokulp01/meta-qlearning-humanoid","owner":"gokulp01","description":"Meta QLearning experiments to optimize robot walking patterns ","archived":false,"fork":false,"pushed_at":"2024-08-21T05:50:25.000Z","size":34263,"stargazers_count":27,"open_issues_count":1,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-07T21:04:50.636Z","etag":null,"topics":["gym","gym-environment","humanoid","humanoid-robot","humanoid-walking","meta-learning","meta-qlearning","mujoco","mujoco-environments","pybullet","reinforcement-learning","robotics"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gokulp01.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-07T01:46:20.000Z","updated_at":"2025-02-21T15:57:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"b7ac4875-4e3d-4b0b-92ff-fec157437b34","html_url":"https://github.com/gokulp01/meta-qlearning-humanoid","commit_stats":{"total_commits":5,"total_committers":2,"mean_commits":2.5,"dds":"0.19999999999999996","last_synced_commit":"cfb93e43a94294f365045604e0840b3d46140b8f"},"previous_names":["gokulp01/meta-qlearning-humanoid"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gokulp01%2Fmeta-qlearning-humanoid","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gokulp01%2Fmeta-qlearning-humanoid/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gokulp01%2Fmeta-qlearning-humanoid/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gokulp01%2Fmeta-qlearning-humanoid/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gokulp01","download_url":"https://codeload.github.com/gokulp01/meta-qlearning-humanoid/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252954432,"owners_count":21830903,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gym","gym-environment","humanoid","humanoid-robot","humanoid-walking","meta-learning","meta-qlearning","mujoco","mujoco-environments","pybullet","reinforcement-learning","robotics"],"created_at":"2024-11-17T23:11:55.845Z","updated_at":"2025-10-05T04:45:16.266Z","avatar_url":"https://github.com/gokulp01.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# meta-qlearning-humanoid\nMeta QLearning experiments to optimize robot walking patterns \n![out](docs/learn_step.gif)\n\n# Overview:\nImplemented Meta-Q-Learning for optimizing humanoid walking patterns. We also demonstrate its effectiveness in improving stability, efficiency, and adaptability. Additionally, this work also explores the transferability of Meta-Q-Learning to new tasks with minimal tuning. \n\n## Conducted experiments:\n### Learn Stepping using MQL\nTest how adaptable the humanoid is by performing:\n- Side stepping\n- Ascending and Descending\n\n## Setting up the environment:\nThis repository contains everything needed to set up the environment and get the simulation up and running. \n\n### Clone the repository: \n```\ngit clone git@github.com:gokulp01/meta-qlearning-humanoid.git\n```\n\nMake sure the file structure is as follows:\n```\n\u003cYour folder\u003e\n├── algs\n│   └── MQL\n│       ├── buffer.py\n│       └── mql.py\n├── configs\n│   └── abl_envs.json\n├── Humanoid_environment\n│   ├── envs\n│   │   ├── common\n│   │   └── jvrc\n│   ├── models\n│   │   ├── cassie_mj_description\n│   │   └── jvrc_mj_description\n│   ├── scripts\n│   │   ├── debug_stepper.py\n│   │   └── plot_logs.py\n│   ├── tasks\n│   │   │   ├── rewards.cpython-37.pyc\n│   │   │   ├── stepping_task.cpython-37.pyc\n│   │   │   └── walking_task.cpython-37.pyc\n│   │   ├── rewards.py\n│   │   ├── stepping_task.py\n│   │   └── walking_task.py\n│   └── utils\n│       └── footstep_plans.txt\n├── misc\n│   ├── env_meta.py\n│   ├── logger.py\n│   ├── runner_meta_offpolicy.py\n│   ├── runner_multi_snapshot.py\n│   ├── torch_utility.py\n│   └── utils.py\n├── models\n│   ├── networks.py\n│   └── run.py\n├── README.md\n└── run_script.py\n```\n\n### Installing packages:\n```\npip3 install -r requirements.txt\n```\n\n### Training\n```\npython3 run_script.py\n```\n\n### Inference\nThis work was done as a fun project to learn RL and its applications, so I have not drawn a lot of theoretical inferences. That being said, here are some quantitative inferences from the work:\n![out](docs/graph1.png)\n![out](docs/graph2.png)\n\n\n\n## References:\nRasool Fakoor, Pratik Chaudhari, Stefano Soatto, \u0026 Alex Smola (2020). Meta-Q-Learning. In ICLR 2020, Microsoft Research Reinforcement Learning Day 2021\n\n\n### Some important notes:\n- Code is written to train using a GPU\n- Training time: ~55 hours on RTX 3080\n- **Feel free to contact the author for pre-trained model**\n- The code is not very well documented (PRs are more than welcome!)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgokulp01%2Fmeta-qlearning-humanoid","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgokulp01%2Fmeta-qlearning-humanoid","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgokulp01%2Fmeta-qlearning-humanoid/lists"}