{"id":13699606,"url":"https://github.com/hellochick/FICM","last_synced_at":"2025-05-04T16:35:33.576Z","repository":{"id":37604364,"uuid":"184388721","full_name":"hellochick/FICM","owner":"hellochick","description":":video_game: [IJCAI'20][ICLR'19 Workshop] Flow-based Intrinsic Curiosity Module. Playing SuperMario with RL agent and FICM!","archived":false,"fork":false,"pushed_at":"2022-12-08T05:10:39.000Z","size":16501,"stargazers_count":102,"open_issues_count":8,"forks_count":9,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-13T05:36:01.451Z","etag":null,"topics":["curiosity-driven","intrinsic-curiosity-module","intrinsic-motivation","optical-flow","supermario"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hellochick.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-01T08:27:48.000Z","updated_at":"2024-10-29T09:04:09.000Z","dependencies_parsed_at":"2023-01-24T12:00:14.785Z","dependency_job_id":null,"html_url":"https://github.com/hellochick/FICM","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hellochick%2FFICM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hellochick%2FFICM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hellochick%2FFICM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hellochick%2FFICM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hellochick","download_url":"https://codeload.github.com/hellochick/FICM/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252366708,"owners_count":21736624,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["curiosity-driven","intrinsic-curiosity-module","intrinsic-motivation","optical-flow","supermario"],"created_at":"2024-08-02T20:00:37.887Z","updated_at":"2025-05-04T16:35:28.490Z","avatar_url":"https://github.com/hellochick.png","language":"C++","funding_links":[],"categories":["C++"],"sub_categories":[],"readme":"#### Status: Archive. \n## Flow-based Intrinsic Curiosity Module (FICM) \n\u003ccenter\u003e\n\u003cimg src=\"./imgs/workflow.jpg\" width=\"800\"\u003e\u003c/img\u003e\n\u003c/center\u003e\n\n### [[Paper]](https://www.ijcai.org/Proceedings/2020/286) [[Demo Video]](https://www.youtube.com/watch?v=w-a6akKpWT0\u0026feature=youtu.be) [[Reddit Discuss]](https://www.reddit.com/r/MachineLearning/comments/bu05ua/p_playing_supermario_bros_without_knowing_any/) \n\n\nThis is a TensorFlow-based implementation for our paper \"Flow-based Intrinsic Curiosity Module\".\n\nFICM is used for evaluating the novelty of observations, it generates intrinsic rewards based on the prediction errors of optical flow estimation since the rapid change part in consecutive frames usually serve as important signals. \n\nWithout any external reward, FICM can help RL agent to play SuperMario successfully. \n\n\u003cimg src=\"./imgs/animate2.gif\" width=\"400\"\u003e\u003c/img\u003e\u003cimg src=\"./imgs/animate.gif\" width=\"400\"\u003e\u003c/img\u003e\n\n\nThis repo is largely inherited from [large-scale-curiosity](https://github.com/openai/large-scale-curiosity), and we also borrowed `correlation layer` from [flownet2_tf](https://github.com/sampepose/flownet2-tf).\n\n-----\n\n[Flow-based Intrinsic Curiosity Module](https://www.ijcai.org/Proceedings/2020/286)  \n[Hsuan-Kung Yang*](https://hellochick.github.io/), Po-Han Chiang*, Min-Fong Hong, and Chun-Yi Lee (* equal contributions)  \nElsaLab, National Tsing Hua University  \nIn [IJCAI'20 main track](https://ijcai20.org/) and [ICLR'19 TARL workshop](https://tarl2019.github.io/)\n\n\n\n### Dependencies\n* Python3.5\n\n### Installation\n```bash\npip install -r requirement.txt\npip install git+https://github.com/openai/baselines.git@3301089b48c42b87b396e246ea3f56fa4bfc9678\n```\n\n#### FICM-C (Optional)\nIf you want to use FICM-C architecture, you'll need to compile for correlation layer additionally.\n```bash\ncd correlation_layer\nmake all\n```\n\u003e Note: You might encounter some errors raised from the source codes of tensorflow, you can easily change the header file of\n'cuda_kernel_helper.h', 'cuda_device_functions.h', and 'cuda_launch_config.h'\n\n#### SuperMario (Optional)\nIf you want to train an agent to play SuperMario, you need to install `retro` and import `SuperMarioBros-Nes` game.\n\n```bash\npip install retro\n```\n\nRead the official guide [here](https://retro.readthedocs.io/en/latest/getting_started.html#importing-roms)\n\n### Example usage\n#### Mario\n```bash\npython run.py --feat_learning flowS --env_kind mario\n```\n#### Atari\n```bash\npython run.py --feat_learning flowS --env SeaquestNoFrameskip-v4 --seed 666\n```\n\n### Citation\nIf you find this paper or code implementation helpful, please consider to cite:\n\n```\n@inproceedings{flowbasedcuriosity2020,\n  title     = {Flow-based Intrinsic Curiosity Module},\n  author    = {Hsuan-Kung Yang, Po-Han Chiang, Min-Fong Hong and Chun-Yi Lee},\n  booktitle = {Proceedings of the Twenty-Ninth International Joint Conference on\n               Artificial Intelligence, {IJCAI-20}},\n  publisher = {International Joint Conferences on Artificial Intelligence Organization},             \n  editor    = {Christian Bessiere}\t\n  pages     = {2065--2072},\n  year      = {2020},\n  month     = {7},\n  note      = {Main track}\n  doi       = {10.24963/ijcai.2020/286},\n  url       = {https://doi.org/10.24963/ijcai.2020/286},\n}\n```\n\n### Reference\n    @inproceedings{largeScaleCuriosity2018,\n        Author = {Burda, Yuri and Edwards, Harri and\n                  Pathak, Deepak and Storkey, Amos and\n                  Darrell, Trevor and Efros, Alexei A.},\n        Title = {Large-Scale Study of Curiosity-Driven Learning},\n        Booktitle = {arXiv:1808.04355},\n        Year = {2018}\n    }\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhellochick%2FFICM","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhellochick%2FFICM","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhellochick%2FFICM/lists"}