{"id":15640029,"url":"https://github.com/scitator/run-skeleton-run","last_synced_at":"2025-09-05T13:43:02.099Z","repository":{"id":133293919,"uuid":"110868459","full_name":"Scitator/Run-Skeleton-Run","owner":"Scitator","description":"Reason8.ai PyTorch solution for NIPS RL 2017 challenge","archived":false,"fork":false,"pushed_at":"2019-10-15T19:51:50.000Z","size":30008,"stargazers_count":84,"open_issues_count":1,"forks_count":14,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-12-26T14:12:14.731Z","etag":null,"topics":["actor-critic","ddpg","ddpg-agent","nips","nips-2017","physics-based","ppo","pytorch","pytorch-solution","reinforcement-learning","tensorflow","trpo"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Scitator.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-15T18:04:29.000Z","updated_at":"2024-09-05T11:34:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"7ae34954-c216-4c6c-b321-a98f3b8091c5","html_url":"https://github.com/Scitator/Run-Skeleton-Run","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scitator%2FRun-Skeleton-Run","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scitator%2FRun-Skeleton-Run/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scitator%2FRun-Skeleton-Run/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Scitator%2FRun-Skeleton-Run/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Scitator","download_url":"https://codeload.github.com/Scitator/Run-Skeleton-Run/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232812239,"owners_count":18580136,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actor-critic","ddpg","ddpg-agent","nips","nips-2017","physics-based","ppo","pytorch","pytorch-solution","reinforcement-learning","tensorflow","trpo"],"created_at":"2024-10-03T11:30:09.610Z","updated_at":"2025-01-07T02:26:54.384Z","avatar_url":"https://github.com/Scitator.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Run-Skeleton-Run\n\n[Reason8.ai](https://reason8.ai) PyTorch solution for 3rd place [NIPS RL 2017 challenge](https://www.crowdai.org/challenges/nips-2017-learning-to-run/leaderboards?challenge_round_id=12).\n\n[Theano version](https://github.com/fgvbrt/nips_rl)\n\nAdditional thanks to [Mikhail Pavlov](https://github.com/fgvbrt) for collaboration.\n\n## Agent policies\n\n### no-flip-state-action\n\n![alt text](https://github.com/Scitator/Run-Skeleton-Run/blob/master/gifs/noflip.gif)\n\n### flip-state-action\n\n![alt text](https://github.com/Scitator/Run-Skeleton-Run/blob/master/gifs/flip.gif)\n\n\n## How to setup environment?\n\n1. `sh setup_conda.sh`\n2. `source activate opensim-rl`\n\nWould like to test baselines? (Need MPI support)\n\n3. `sudo apt-get install openmpi-bin openmpi-doc libopenmpi-dev`\n3+. `sh setup_env_mpi.sh`\n\nOR like DDPG agents?\n3. `sh setup_env.sh`\n\n4. Congrats! Now you are ready to check our agents.\n\n\n## Run DDPG agent\n\n```\nCUDA_VISIBLE_DEVICES=\"\" PYTHONPATH=. python ddpg/train.py \\\n    --logdir ./logs_ddpg \\\n    --num-threads 4 \\\n    --ddpg-wrapper \\\n    --skip-frames 5 \\\n    --fail-reward -0.2 \\\n    --reward-scale 10 \\\n    --flip-state-action \\\n    --actor-layers 64-64 --actor-layer-norm --actor-parameters-noise \\\n    --actor-lr 0.001 --actor-lr-end 0.00001 \\\n    --critic-layers 64-32 --critic-layer-norm \\\n    --critic-lr 0.002 --critic-lr-end 0.00001 \\\n    --initial-epsilon 0.5 --final-epsilon 0.001 \\\n    --tau 0.0001\n```\n\n\n## Evaluate DDPG agent\n\n```\nCUDA_VISIBLE_DEVICES=\"\" PYTHONPATH=./ python ddpg/submit.py \\\n    --restore-actor-from ./logs_ddpg/actor_state_dict.pkl \\\n    --restore-critic-from ./logs_ddpg/critic_state_dict.pkl \\\n    --restore-args-from ./logs_ddpg/args.json \\\n    --num-episodes 10\n\n```\n\n\n## Run TRPO/PPO agent\n\n```\nCUDA_VISIBLE_DEVICES=\"\" PYTHONPATH=. python ddpg/train.py \\\n    --agent ppo \\\n    --logdir ./logs_baseline \\\n    --baseline-wrapper \\\n    --skip-frames 5 \\\n    --fail-reward -0.2 \\\n    --reward-scale 10\n```\n\n## Citation\nPlease cite the following paper if you feel this repository useful.\n```\n@article{run_skeleton,\n  title={Run, skeleton, run: skeletal model in a physics-based simulation},\n  author = {Mikhail Pavlov, Sergey Kolesnikov and Sergey M.~Plis},\n  journal={AAAI Spring Symposium Series},\n  year={2018}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscitator%2Frun-skeleton-run","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscitator%2Frun-skeleton-run","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscitator%2Frun-skeleton-run/lists"}