{"id":19185521,"url":"https://github.com/rmst/rtrl","last_synced_at":"2025-05-08T00:44:57.041Z","repository":{"id":123043432,"uuid":"196450952","full_name":"rmst/rtrl","owner":"rmst","description":"PyTorch implementation of our paper Real-Time Reinforcement Learning (NeurIPS 2019)","archived":false,"fork":false,"pushed_at":"2020-05-03T06:09:35.000Z","size":67108,"stargazers_count":73,"open_issues_count":1,"forks_count":17,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-08T00:44:50.132Z","etag":null,"topics":["deep-learning","deep-reinforcement-learning","machine-learning","pytorch","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rmst.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-11T19:05:00.000Z","updated_at":"2025-04-24T04:44:26.000Z","dependencies_parsed_at":null,"dependency_job_id":"32699283-ad92-4aea-bd85-8157b27d693d","html_url":"https://github.com/rmst/rtrl","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmst%2Frtrl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmst%2Frtrl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmst%2Frtrl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmst%2Frtrl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rmst","download_url":"https://codeload.github.com/rmst/rtrl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252978680,"owners_count":21834913,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-reinforcement-learning","machine-learning","pytorch","reinforcement-learning"],"created_at":"2024-11-09T11:10:48.840Z","updated_at":"2025-05-08T00:44:57.019Z","avatar_url":"https://github.com/rmst.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Real-Time Reinforcement Learning\n\nThis repo is accompanying our paper \"Real-Time Reinforcement Learning\" (https://arxiv.org/abs/1911.04448).\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"/resources/rl-rtrl.png\" width=70% /\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  Traditional Reinforcement Learning\n  \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n  Real-Time Reinforcement Learning\n\u003c/p\u003e\n\n\n### Getting Started\nThis repo can be pip-installed via\n```bash\npip install git+https://github.com/rmst/rtrl.git\n```\n\nTo train an RTAC agent on the basic `Pendulum-v0` task run\n```bash\npython -m rtrl run rtrl:RtacTraining Env.id=Pendulum-v0\n```\n\n\n\n### Mujoco Experiments\nTo install Mujoco you follow the instructions at [openai/gym](https://github.com/openai/gym) or have a look at [`our dockerfile`](github.com/rmst/rtrl/blob/master/docker/gym/Dockerfile). The following environments were used in the paper.\n\n![MuJoCo](resources/mujoco_horizontal.png)\n\n\nTo train an RTAC agent on `HalfCheetah-v2` run\n```bash\npython -m rtrl run rtrl:RtacTraining Env.id=HalfCheetah-v2\n```\n\nTo train a SAC agent on `Ant-v2` with a real-time wrapper (i.e. RTMDP in the paper) run\n```bash\npython -m rtrl run rtrl:SacTraining Env.id=Ant-v2 Env.real_time=True\n```\n\n### Avenue Experiments\nAvenue [(Ibrahim et al., 2019)](https://github.com/elementaI/avenue) can be pip-installed via\n```bash\npip install git+https://github.com/elementai/avenue.git\n```\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"resources/rtrl-avenue-white.gif\" width=100% /\u003e\u003c/p\u003e\n\n\u003c!-- \u003cp align=\"center\"\u003e\u003cimg src=\"/resources/avenue_collage.png\" width=95% /\u003e\u003c/p\u003e --\u003e\n\nTo train an RTAC agent to drive on a race track (right video) run\n```bash\npython -m rtrl run rtrl:RtacAvenueTraining Env.id=RaceSolo-v0\n```\nNote that this requires a lot of resources, especially memory (16GB+).\n\n\n### Storing Stats\n`python -m rtrl run` just prints stats to stdout. To save stats use the following instead.\n```bash\npython -m rtrl run-fs experiment-1 rtrl:RtacTraining Env.id=Pendulum-v0\n```\nStats are generated and printed every `round` but only saved to disk every `epoch`. The stats will be saved as pickled pandas dataframes in `experiment-1/stats`.\n\n### Checkpointing\nThis repo supports checkpointing. Every `epoch` the whole run object (e.g. instances of `rtrl.training:Training`) is pickled to disk and reloaded. This is to ensure reproducibilty.\n\nYou can manually load and inspect pickled run instances with the standard `pickle:load` or the more convenient `rtrl:load`. For example, to look at the first transition in a SAC agent's replay memory run\n```python\nimport rtrl\nrun = rtrl.load('experiment-1/state')\nprint(run.agent.memory[0])\n``` \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmst%2Frtrl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frmst%2Frtrl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmst%2Frtrl/lists"}