{"id":18647256,"url":"https://github.com/ttitcombe/dqn","last_synced_at":"2026-04-11T08:03:38.846Z","repository":{"id":115039864,"uuid":"197337619","full_name":"TTitcombe/DQN","owner":"TTitcombe","description":"PyTorch implementation of Deep Q Learning","archived":false,"fork":false,"pushed_at":"2020-01-06T22:06:34.000Z","size":25325,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-05-17T19:08:28.990Z","etag":null,"topics":["cartpole","deep-learning","deep-q-learning","deep-q-network","deep-reinforcement-learning","dqn","dqn-pytorch","openai-gym","pytorch","pytorch-implementation","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TTitcombe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-17T07:25:23.000Z","updated_at":"2025-03-21T16:08:37.000Z","dependencies_parsed_at":null,"dependency_job_id":"d237733b-56a5-4d1e-bad6-6731ca6baefd","html_url":"https://github.com/TTitcombe/DQN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TTitcombe/DQN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FDQN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FDQN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FDQN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FDQN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TTitcombe","download_url":"https://codeload.github.com/TTitcombe/DQN/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FDQN/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31673068,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-10T17:19:37.612Z","status":"online","status_checked_at":"2026-04-11T02:00:05.776Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cartpole","deep-learning","deep-q-learning","deep-q-network","deep-reinforcement-learning","dqn","dqn-pytorch","openai-gym","pytorch","pytorch-implementation","reinforcement-learning"],"created_at":"2024-11-07T06:25:22.430Z","updated_at":"2026-04-11T08:03:38.823Z","avatar_url":"https://github.com/TTitcombe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DQN\n\n![DDQN CarRacing](results/videos/carracing_good.gif)\n\nThis is a basic implementation of Deep Q Learning. We have implemented linear and convolutional DQN and DDQN models, with \nDQN and double DQN algorithms\n\nNext steps:\n* Model trained on Atari games\n* Prioritized Experience Replay\n\n## To run\nTo begin, setup [OpenAI gym](https://gym.openai.com/) and install the packages in `requirements.txt`.\n\nWe have an example script which trains a model on the CartPoleSwingUp environment (this requires gym \u003c= 0.9.4).\nRun `python -m examples.cartpoleswingup_linear` in the top-level directory.\n\n(To run Box2D environments, I used [this Docker container](https://github.com/TTitcombe/docker_openai_gym) - \ncheck it out if you are also having problems installing Gym)\n\n## Results\nThe best models trained on each env are present in `results/models/`. There you will find the saved pytorch model as a `.pth` file and\na graph comparing the reward per episode against random play.\n\n**Linear models**\n\n| Model | Env             |      Score      |\n|-------|-----------------|:---------------:|\n| DQN | CartPole-v1     |  500            |\n| DQN | CartPoleSwingUp\\* |  872 +/- 3          |\n\nYou can see how dependant the linear model is on various hyperparameters in the following graph\n\n![DQN linear investigation](results/CartpoleSwingUp_investigation.png)\n\n\\**Note: While we can train high-performing models on CartPoleSwingUp, these are very unstable, even when training for millions of frames and with a large (100000) capacity memory. \nIt is not clear why this is the case.*\n\n**Conv model**\n\n| Model | Env | Score |\n|-------|-----|:-----:|\n\n![DQN_CartPoleSwingUp_Example](results/videos/double_dqn_cartpoleswingup.gif)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fttitcombe%2Fdqn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fttitcombe%2Fdqn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fttitcombe%2Fdqn/lists"}