{"id":20207219,"url":"https://github.com/pathak22/exploration-by-disagreement","last_synced_at":"2025-06-16T00:09:12.944Z","repository":{"id":85023071,"uuid":"187551350","full_name":"pathak22/exploration-by-disagreement","owner":"pathak22","description":"[ICML 2019] TensorFlow Code for Self-Supervised Exploration via Disagreement","archived":false,"fork":false,"pushed_at":"2019-06-11T19:32:50.000Z","size":5051,"stargazers_count":123,"open_issues_count":3,"forks_count":23,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-10T12:55:34.734Z","etag":null,"topics":["artificial-curiosity","artificial-intelligence","curiosity","deep-learning","deep-reinforcement-learning","exploration","rl","self-supervised","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pathak22.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-05-20T02:00:21.000Z","updated_at":"2024-08-14T08:22:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"b9c07268-b5f6-4705-a8b1-dd586b7c3de2","html_url":"https://github.com/pathak22/exploration-by-disagreement","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pathak22/exploration-by-disagreement","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pathak22%2Fexploration-by-disagreement","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pathak22%2Fexploration-by-disagreement/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pathak22%2Fexploration-by-disagreement/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pathak22%2Fexploration-by-disagreement/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pathak22","download_url":"https://codeload.github.com/pathak22/exploration-by-disagreement/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pathak22%2Fexploration-by-disagreement/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260072759,"owners_count":22954922,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-curiosity","artificial-intelligence","curiosity","deep-learning","deep-reinforcement-learning","exploration","rl","self-supervised","tensorflow"],"created_at":"2024-11-14T05:28:02.359Z","updated_at":"2025-06-16T00:09:12.921Z","avatar_url":"https://github.com/pathak22.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"## Self-Supervised Exploration via Disagreement ##\n#### [[Project Website]](https://pathak22.github.io/exploration-by-disagreement/) [[Demo Video]](https://youtu.be/POlrWt32_ec)\n\n[Deepak Pathak*](https://people.eecs.berkeley.edu/~pathak/), [Dhiraj Gandhi*](http://www.cs.cmu.edu/~dgandhi/), [Abhinav Gupta](http://www.cs.cmu.edu/~abhinavg/)\u003cbr/\u003e\n(\u0026#42; equal contribution)\n\nUC Berkeley\u003cbr/\u003e\nCMU\u003cbr/\u003e\nFacebook AI Research\n\n\u003ca href=\"https://pathak22.github.io/exploration-by-disagreement/\"\u003e\n\u003cimg src=\"https://pathak22.github.io/exploration-by-disagreement/resources/method.jpg\" width=\"600\"\u003e\n\u003c/img\u003e\u003c/a\u003e\n\nThis is a TensorFlow based implementation for our [paper on self-supervised exploration via disagreement](https://pathak22.github.io/exploration-by-disagreement/). In this paper, we propose a formulation for exploration inspired by the work in active learning literature. Specifically, we train an ensemble of dynamics models and incentivize the agent to explore such that the disagreement of those ensembles is maximized. This allows the agent to learn skills by exploring in a self-supervised manner without any external reward. Notably, we further leverage the disagreement objective to optimize the agent's policy in a differentiable manner, without using reinforcement learning, which results in a sample-efficient exploration. We demonstrate the efficacy of this formulation across a variety of benchmark environments including stochastic-Atari, Mujoco, Unity and a real robotic arm. If you find this work useful in your research, please cite:\n\n    @inproceedings{pathak19disagreement,\n        Author = {Pathak, Deepak and\n                  Gandhi, Dhiraj and Gupta, Abhinav},\n        Title = {Self-Supervised Exploration via Disagreement},\n        Booktitle = {ICML},\n        Year = {2019}\n    }\n\n### Installation and Usage\nThe following command should train a pure exploration agent on Breakout with default experiment parameters.\n```bash\npython run.py\n```\nTo use more than one gpu/machine, use MPI (e.g. `mpiexec -n 8 python run.py` should use 1024 parallel environments to collect experience instead of the default 128 on an 8 gpu machine).\n\n### Other helpful pointers\n- [Paper](https://pathak22.github.io/exploration-by-disagreement/resources/icml19.pdf)\n- [Project Website](https://pathak22.github.io/exploration-by-disagreement/)\n- [Demo Video](https://youtu.be/POlrWt32_ec)\n\n### Acknowledgement\n\nThis repository is built off the publicly released code of [Large-Scale Study of Curiosity-driven Learning, ICLR 2019](https://github.com/openai/large-scale-curiosity).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpathak22%2Fexploration-by-disagreement","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpathak22%2Fexploration-by-disagreement","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpathak22%2Fexploration-by-disagreement/lists"}