{"id":15603742,"url":"https://github.com/achronus/rl_atari_games","last_synced_at":"2026-04-20T10:32:13.092Z","repository":{"id":37245979,"uuid":"498748704","full_name":"Achronus/rl_atari_games","owner":"Achronus","description":"An exploration of the effects of Intrinsic Motivation methods on RL algorithms using Atari games.","archived":false,"fork":false,"pushed_at":"2024-09-26T11:07:42.000Z","size":42817,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-22T22:41:10.722Z","etag":null,"topics":["curiosity","deep-reinforcement-learning","dqn-pytorch","empowerment","intrinsic-motivation","ppo-pytorch","pytorch","rainbow-dqn","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Achronus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-01T13:30:56.000Z","updated_at":"2024-09-26T11:07:47.000Z","dependencies_parsed_at":"2024-12-09T18:41:18.328Z","dependency_job_id":"5f9bf036-ae05-4b67-a81c-6f7c2a852416","html_url":"https://github.com/Achronus/rl_atari_games","commit_stats":{"total_commits":130,"total_committers":1,"mean_commits":130.0,"dds":0.0,"last_synced_commit":"a49f9db96c5d5c0d4dc86014d2331130fe91f4b6"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Achronus/rl_atari_games","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Achronus%2Frl_atari_games","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Achronus%2Frl_atari_games/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Achronus%2Frl_atari_games/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Achronus%2Frl_atari_games/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Achronus","download_url":"https://codeload.github.com/Achronus/rl_atari_games/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Achronus%2Frl_atari_games/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32043008,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-20T00:18:06.643Z","status":"online","status_checked_at":"2026-04-20T02:00:06.527Z","response_time":94,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["curiosity","deep-reinforcement-learning","dqn-pytorch","empowerment","intrinsic-motivation","ppo-pytorch","pytorch","rainbow-dqn","reinforcement-learning"],"created_at":"2024-10-03T03:20:49.793Z","updated_at":"2026-04-20T10:32:13.075Z","avatar_url":"https://github.com/Achronus.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Reinforcement Learning with Atari Games\n\nThis repository focuses on exploring how Intrinsic Motivation (IM) effects Reinforcement Learning (RL) policy-based models.\nWe use a Deep Q-Network (DQN) to simulate off-policy and Proximal Policy Optimization (PPO) as on-policy.\n\n_(Note)_ functionality for Rainbow Deep Q-Networks (RDQNs) is available but is not used during the study due to time constraints.\n\nWe focus on two IM methods: curiosity and empowerment, and evaluate their performance on three Atari games:\nSpace Invaders, Q*bert, and Montezuma's Revenge. Our research aims to provide intuition on how IM methods affect these popular on-policy and off-policy agents.\n\n![Atari Games](/imgs/atari-games.png)\n\n_Figure 1. Examples of each environment. From left to right: Space Invaders, Q*bert, and Montezuma's Revenge (Source: [Gym Docs](https://www.gymlibrary.dev/))._\n\n\n## File Structure\n\nThe file structure for the artefact is outlined below.\n\n``` ANSI\n+-- agents\n|   +-- _agent.py\n|   +-- dqn.py\n|   +-- ppo.py\n|   +-- rainbow.py\n+-- core\n|   +-- README.md\n|   +-- buffer.py\n|   +-- create.py\n|   +-- enums.py\n|   +-- env_details.py\n|   +-- exceptions.py\n|   +-- parameters.py\n|   +-- template.yaml\n+-- intrinsic\n|   +-- _im_base.py\n|   +-- controller.py\n|   +-- empower_models.py\n|   +-- model.py\n|   +-- module.py\n|   +-- parameters.py\n+-- models\n|   +-- _base.py\n|   +-- actor_critic.py\n|   +-- dueling.py\n|   +-- linear.py\n+-- tests\n|   +-- test_buffer.py\n|   +-- test_create.py\n|   +-- test_dqn.py\n|   +-- test_env_details.py\n|   +-- test_helper.py\n|   +-- test_intrisic.py\n|   +-- test_logger.py\n|   +-- test_models.py\n|   +-- test_ppo.py\n+-- utils\n|   +-- dataloader.py\n|   +-- helper.py\n|   +-- init_devices.py\n|   +-- logger.py\n|   +-- model_utils.py\n|   +-- plotter.py\n|   +-- render.py\n|   +-- validate.py\n+-- .coverage\n+-- .coveragerc\n+-- LICENSE\n+-- README.md\n+-- main.py\n+-- parameters.yaml\n+-- requirements.txt\n+-- demo.ipynb\n```\n\n- `\\agents` - contains the RL algorithm implementations\n- `\\core` - core functionality of the artefact\n- `\\intrinsic` - intrinsic motivation functionality\n- `\\models` - neural network models for the agents\n- `\\tests` - unit tests for the application\n- `\\utils` - utility classes and functions that provide extra functionality\n\n## Dependencies\n\nThis project requires a Python 3.10 environment, which can be created with the following instructions:\n\n1. Install [Visual Studio Build Tools](https://visualstudio.microsoft.com/downloads/?q=build+tools) (required for SuperSuit). \n\n2. Create (and activate) a new environment.\n\n   - Linux or Mac\n\n    ```bash\n    conda create --name rlatari python=3.10\n    source activate rlatari\n    ```\n\n   - Windows\n\n   ```bash\n   conda create --name rlatari python=3.10\n   conda activate rlatari\n   ```\n\n3. Clone the repository, navigate to the `rl_atari_games/` folder and install the required dependencies.\n\n    _(Note)_ a requirements.txt file is accessible within this folder detailing a list of the required dependencies.\n\n    ```bash\n    git clone https://github.com/Achronus/rl_atari_games.git\n    cd rl_atari_games\n    conda install -c conda-forge jupyterlab\n    conda install pytorch torchvision pytorch-cuda=11.7 -c pytorch -c nvidia\n    pip install gymnasium[atari,accept-rom-license,other]\n    pip install -r requirements.txt\n    ```\n\n4. Create an [IPython kernel](http://ipython.readthedocs.io/en/stable/install/kernel_install.html) for the `rlatari` environment.\n\n    ```bash\n    python -m ipykernel install --user --name rlatari --display-name \"rlatari\"\n    ```\n\n5. Run the `jupyter-lab` command to start JupyterLab and access the Jupyter Notebook named `rl_with_atari.ipynb`, or run the `main.py` file.\n\n    _(Note)_ running the `main.py` file will train the models. It is advised to examine this file before running it.\n\n\n## References\n\nThe algorithms created in this repository have been inspired and adapted from the following developers:\n\nRainbow DQN -\n- [Curt Park: Rainbow is all you need (GitHub)](https://github.com/Curt-Park/rainbow-is-all-you-need)\n- [Kaixhin: Rainbow (GitHub)](https://github.com/Kaixhin/Rainbow/tree/1745b184c3dfc03d4ffa3ce2342ced9996b39a60)\n\nPPO - \n- [ericyangyu: PPO for beginners (GitHub)](https://github.com/ericyangyu/PPO-for-Beginners)\n- [vwxyzjn: PPO implementation details (GitHub)](https://github.com/vwxyzjn/ppo-implementation-details)\n\nIntrinsic Motivation - \n- [Alexander Zai and Brandon Brown: Deep Reinforcement Learning In Action, Chapter 8: Curiosity-Driven Exploration (book)](https://livebook.manning.com/book/deep-reinforcement-learning-in-action/chapter-8/)\n- [navneet-nmk: pytorch-rl (GitHub) - for empowerment](https://github.com/navneet-nmk/pytorch-rl)\n\nA special thanks to each of these developers for creating incredible work in the RL space. Please check out their original work using the links above.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fachronus%2Frl_atari_games","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fachronus%2Frl_atari_games","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fachronus%2Frl_atari_games/lists"}