{"id":13568554,"url":"https://github.com/Scitator/run-skeleton-run-in-3d","last_synced_at":"2025-04-04T04:31:24.632Z","repository":{"id":133293943,"uuid":"219129598","full_name":"Scitator/run-skeleton-run-in-3d","owner":"Scitator","description":"NeurIPS 2019: Learn to Move - Walk Around, 2nd place solution","archived":false,"fork":false,"pushed_at":"2023-11-09T22:23:51.000Z","size":4533,"stargazers_count":18,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-03T11:54:54.459Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2003.14210","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Scitator.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-02T09:09:00.000Z","updated_at":"2024-10-17T02:52:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"b817c5c2-0028-48c3-9c06-3ddf20a9a7af","html_url":"https://github.com/Scitator/run-skeleton-run-in-3d","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scitator%2Frun-skeleton-run-in-3d","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scitator%2Frun-skeleton-run-in-3d/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scitator%2Frun-skeleton-run-in-3d/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scitator%2Frun-skeleton-run-in-3d/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Scitator","download_url":"https://codeload.github.com/Scitator/run-skeleton-run-in-3d/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247123072,"owners_count":20887259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T14:00:28.084Z","updated_at":"2025-04-04T04:31:19.621Z","avatar_url":"https://github.com/Scitator.png","language":"Python","funding_links":[],"categories":["Repositories"],"sub_categories":[],"readme":"# NeurIPS 2019: Learn to Move – Walk Around\u003cbr/\u003e2nd place solution – powered by [Catalyst.RL](https://github.com/catalyst-team/catalyst)\n\n\u003cdiv align=\"center\"\u003e\n\n![alt text](./pics/skeleton.191102.v2.gif)\n\n[![Telegram](./pics/telegram.svg)](https://t.me/catalyst_team)\n[![Gitter](https://badges.gitter.im/catalyst-team/community.svg)](https://gitter.im/catalyst-team/community?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge)\n[![Slack](./pics/slack.svg)](https://opendatascience.slack.com/messages/CGK4KQBHD)\n\n\u003c/div\u003e\n\n\n## How2run\n\n### System requirements\n\nYou need to install [Anaconda](www.anaconda.com/download) and Redis:\n```\nsudo apt install redis-server\n```\n\n### Python requirements\n(Taken from the official [repo](https://github.com/stanfordnmbl/osim-rl)).\n\n**Anaconda** is required to run our simulations. Anaconda will create a virtual environment with all the necessary libraries, to avoid conflicts with libraries in your operating system. You can get anaconda from here https://docs.anaconda.com/anaconda/install/. In the following instructions we assume that Anaconda is successfully installed.\n\nFor the challenge we prepared [OpenSim](http://opensim.stanford.edu/) binaries as a conda environment to make the installation straightforward\n\nWe support Windows, Linux, and Mac OSX (all in 64-bit). To install our simulator, you first need to create a conda environment with the OpenSim package.\n\nOn **Windows**, open a command prompt and type:\n```bash\nconda create -n opensim-rl -c kidzik opensim python=3.6.1\nactivate opensim-rl\n```\n\nOn **Linux/OSX**, run:\n```bash\nconda create -n opensim-rl -c kidzik opensim python=3.6\nsource activate opensim-rl\nconda install python=3.6.1 -c conda-forge\n```\n\nThese commands will create a virtual environment on your computer with the necessary simulation libraries installed. Next, you need to install our python reinforcement learning environment. Type (on all platforms):\n```bash\nconda install -c conda-forge lapack git -y\npip install osim-rl -y\nconda remove nb_conda_kernels -y\nconda install -c conda-forge nb_conda_kernels -y\nconda install notebook jupyter nb_conda -y\nconda remove nbpresent -y\npip install -r ./requirements.txt\n```\n\n### Run ensemble training\n\n\u003cdiv align=\"center\"\u003e\n\n![alt text](./pics/achitecture.png)\n\n[Catalyst.RL](https://github.com/catalyst-team/catalyst) achitecture. Samplers interact with the environment and gather training data. Trainers retrieve collected data and update parameters of value function and policy approximators. All communication is conducted through a database.\n\n\u003c/div\u003e\n\n```\nexport LOGDIR=/path/to/logdir\nexport PORT=14001\nbash bin/prepare_configs.sh\n\nredis-server --port $PORT\n\nCUDA_VISIBLE_DEVICES=\"0\" \\\nPYTHONPATH=\".\" \\\nEXP_CONFIG=\"./configs/_exp_common.yml ./configs/env_l2m.yml ./configs/_dpg_common.yml ./configs/td3.yml\" \\\nDB_SPEC=\"null\" \\\nOMP_NUM_THREADS=1 MKL_NUM_THREADS=1 catalyst-rl-run \\\n    --db/prefix=td3-quan-01-04:str \\\n    --environment/history_len=1:int \\\n    --environment/frame_skip=4:int\n\nCUDA_VISIBLE_DEVICES=\"1\" \\\nPYTHONPATH=\".\" \\\nEXP_CONFIG=\"./configs/_exp_common.yml ./configs/env_l2m.yml ./configs/_dpg_common.yml ./configs/td3.yml\" \\\nDB_SPEC=\"null\" \\\nOMP_NUM_THREADS=1 MKL_NUM_THREADS=1 catalyst-rl-run \\\n    --db/prefix=td3-quan-04-04:str \\\n    --environment/history_len=4:int \\\n    --environment/frame_skip=4:int\n\nCUDA_VISIBLE_DEVICES=\"2\" \\\nPYTHONPATH=\".\" \\\nEXP_CONFIG=\"./configs/_exp_common.yml ./configs/env_l2m.yml ./configs/_dpg_common.yml ./configs/td3.yml\" \\\nDB_SPEC=\"null\" \\\nOMP_NUM_THREADS=1 MKL_NUM_THREADS=1 catalyst-rl-run \\\n    --db/prefix=td3-quan-08-04:str \\\n    --environment/history_len=8:int \\\n    --environment/frame_skip=4:int\n\nCUDA_VISIBLE_DEVICES=\"3\" \\\nPYTHONPATH=\".\" \\\nEXP_CONFIG=\"./configs/_exp_common.yml ./configs/env_l2m.yml ./configs/_dpg_common.yml ./configs/td3.yml\" \\\nDB_SPEC=\"null\" \\\nOMP_NUM_THREADS=1 MKL_NUM_THREADS=1 catalyst-rl-run \\\n    --db/prefix=td3-quan-12-04:str \\\n    --environment/history_len=12:int \\\n    --environment/frame_skip=4:int\n```\n\n### Check results in Tensorboard\n\n\u003cdiv align=\"center\"\u003e\n\n![alt text](./pics/rewards.png)\n\nAverage raw reward on validation seeds versus training time (in hours).\u003cbr/\u003eDifferent graphs correspond to different history lengths used for training.\n\n\u003c/div\u003e\n\n\n## Additional links [WIP]\n\n1. [Catalyst.RL](https://github.com/catalyst-team/catalyst)\n2. [Analysis of the solution - slides (in English)](https://docs.google.com/presentation/d/1g4g_Rxp9M3xAHwpp_hNzC87L9Gvum3H09g2DIQn1Taw/edit?usp=sharing)\n3. [Analysis of the solution - video (in Russian)](https://youtu.be/PprDcJHrFdg?t=4020)\n4. [Cool video with agent run](https://youtu.be/WuqNdNBVzzI)\n5. [NeurIPS 2019: Learn to Move - Walk Around – starter kit](https://github.com/Scitator/learning-to-move-starter-kit)\n\n\n## Citation\nPlease cite the following paper if you feel this repository useful.\n```\n@article{run_skeleton_in3d,\n  title={Sample Efficient Ensemble Learning with Catalyst.RL},\n  author = {Kolesnikov, Sergey and Khrulkov, Valentin},\n  journal={arXiv preprint arXiv:[WIP]},\n  year={2019}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FScitator%2Frun-skeleton-run-in-3d","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FScitator%2Frun-skeleton-run-in-3d","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FScitator%2Frun-skeleton-run-in-3d/lists"}