{"id":18271252,"url":"https://github.com/camargo/dmu-project","last_synced_at":"2026-04-30T14:36:59.057Z","repository":{"id":165754881,"uuid":"622768044","full_name":"camargo/dmu-project","owner":"camargo","description":"Decision Making Under Uncertainty (DMU) final project.","archived":false,"fork":false,"pushed_at":"2023-05-15T23:12:21.000Z","size":22945,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"develop","last_synced_at":"2025-12-02T00:40:37.931Z","etag":null,"topics":["atari","deep-learning","games","policy-gradient","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/camargo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-03T02:49:42.000Z","updated_at":"2023-05-12T22:12:11.000Z","dependencies_parsed_at":"2023-06-03T01:00:16.904Z","dependency_job_id":null,"html_url":"https://github.com/camargo/dmu-project","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/camargo/dmu-project","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/camargo%2Fdmu-project","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/camargo%2Fdmu-project/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/camargo%2Fdmu-project/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/camargo%2Fdmu-project/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/camargo","download_url":"https://codeload.github.com/camargo/dmu-project/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/camargo%2Fdmu-project/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32468009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"ssl_error","status_checked_at":"2026-04-30T13:12:06.837Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atari","deep-learning","games","policy-gradient","reinforcement-learning"],"created_at":"2024-11-05T11:39:10.855Z","updated_at":"2026-04-30T14:36:59.041Z","avatar_url":"https://github.com/camargo.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dmu-project\n\nRepository for my Spring 2023 Decision Making Under Uncertainty final project. I investigated the performance of a reinforcement learning agent trained using a policy gradient to play Atari Pong. You can read the [final report here](./final-report.pdf).\n\n## Agent Demo\n\nHere is a demo of one of the trained agents (a53b7b) playing and winning against the computer.\n\nhttps://user-images.githubusercontent.com/683355/235566498-97fada0a-9d09-41cb-87b5-6b44b51dc6ea.mp4\n\n## Source Files\n\n| File                          | Description                                                             |\n| ----------------------------- | ----------------------------------------------------------------------- |\n| model_helpers.py              | Helper functions for saving and loading PyTorch models.                 |\n| play.py                       | Fun script for playing games in the Atari gym.                          |\n| pong_test.py                  | Test script that tests the trained Pong agent.                          |\n| pong_train.py                 | Train script that trains Pong agent via policy gradient.                |\n| visualization_and_stats.ipynb | Notebook for help visualizing models, and computing various statistics. |\n\n## Models\n\nModels were trained with PyTorch using Reinforcement Learning and a policy gradient. They are stored in the [models](./models) directory. Complete metrics were collected with [Aim](https://github.com/aimhubio/aim).\n\n| Agent | ID                       | Train Time  | Win Rate | Total Fames | Reward-to-Go | Baseline Subtraction | Max Steps / Episode | Total Layers | Hidden Dim | Episodes | Gamma | Learning Rate |\n| ----- | ------------------------ | ----------- | -------- | ----------- | ------------ | -------------------- | ------------------- | ------------ | ---------- | -------- | ----- | ------------- |\n| 4     | f0d3a1baf6a04e9380841bc5 | 109hrs      | ~62%     | 506,007,959 | ✅           | ✅                   | None                | 3            | 300        | 20000    | 0.99  | 0.0001        |\n| 3     | 2b9c7df2eca04bb49e31404f | 35hrs       | ~95%     | 367,556,707 | ✅           | ✅                   | 5000                | 3            | 200        | 20000    | 0.99  | 0.0001        |\n| 2     | a53b7b3457f14f4e99172150 | 38hrs       | ~95%     | 335,928,545 | ✅           | ❌                   | 5000                | 3            | 200        | 20000    | 0.99  | 0.0001        |\n| 1     | b60ba6f06be54de99c2f890f | 12hrs 29min | ~28%     | 79,861,221  | ✅           | ❌                   | 1000                | 3            | 200        | 20000    | 0.99  | 0.0001        |\n\n- Win rate calculated over the result of 100 games played. See [pong_test.py](./pong_test.py) for how this is computed.\n- Agent 4 not included in final report since it took too long to train.\n\n## Create and Activate Environment\n\n```sh\npython3 -m venv env\nsource env/bin/activate\n```\n\n## Install Dependencies\n\n```sh\npython -m pip install -r requirements.txt\npip install gym\\[accept-rom-license\\]\n```\n\n## Run Model\n\n```sh\npython pong_test.py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcamargo%2Fdmu-project","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcamargo%2Fdmu-project","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcamargo%2Fdmu-project/lists"}