{"id":18647263,"url":"https://github.com/ttitcombe/a3c","last_synced_at":"2026-04-30T16:33:24.139Z","repository":{"id":115039601,"uuid":"203525206","full_name":"TTitcombe/A3C","owner":"TTitcombe","description":"PyTorch implementation of Asynchronous (and Synchronous) Advantage Actor Critic","archived":false,"fork":false,"pushed_at":"2019-08-21T11:07:30.000Z","size":143,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-03T14:07:46.276Z","etag":null,"topics":["a2c","a3c","advantage-actor-critic","pytorch","pytorch-implementation","reinforcement-learning","reinforcement-learning-algorithms"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TTitcombe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-21T06:55:03.000Z","updated_at":"2019-08-21T11:08:23.000Z","dependencies_parsed_at":null,"dependency_job_id":"18260660-77c1-4a02-80d8-c540f40f4fc7","html_url":"https://github.com/TTitcombe/A3C","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TTitcombe/A3C","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FA3C","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FA3C/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FA3C/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FA3C/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TTitcombe","download_url":"https://codeload.github.com/TTitcombe/A3C/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TTitcombe%2FA3C/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32470879,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"ssl_error","status_checked_at":"2026-04-30T13:12:06.837Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["a2c","a3c","advantage-actor-critic","pytorch","pytorch-implementation","reinforcement-learning","reinforcement-learning-algorithms"],"created_at":"2024-11-07T06:25:23.327Z","updated_at":"2026-04-30T16:33:24.113Z","avatar_url":"https://github.com/TTitcombe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A3C PyTorch\n\nThis repo contains a synchronous (A2C) and asynchronous (A3C) PyTorch implementation of Advantage Actor Critic,\npresented by [Mnih et al.](https://arxiv.org/pdf/1602.01783.pdf).\n\nThe [OpenAI baseline](https://github.com/openai/baselines/blob/master/baselines/a2c/a2c.py) was used as a reference.\n\n## What is A3C?\nA3C is a *policy gradient, Actor-critic* algorithm.\n\nThe **Critic** model estimates the value of a state.\\\nThe **Actor** model enacts the policy, i.e. which actions to take.\n\nIn A3C, we estimate the **Advantage function** to reduce the variance of the policy gradient \n(taking fewer \"bad\" steps when we update the parameters).\\\nThe Advantage function is the difference between our value-action estimates and value estimates.\n\nUnlike DQN, A3C doesn't need expensive sweeps over a replay memory to train.\n\n## How to run\nThere is a test script for running both versions of the algorithm on `CartPole-v1` OpenAI gym environment. \nWhen in the top level of this repository, run:\n```bash\npython -m examples.A2C_test \n```\nThis will train A2C for ~4'000 episodes, achieving an average score of around ~400 in ~30seconds on an average CPU.\n\nRun:\n```bash\npython -m examples.A3C_test\n```\ninstead to launch the asynchronous algorithm over 2 processes.\n\nOther scripts in the **examples** directory test various A3/2C hyperparameters, such as number of frames in between updates,\nor whether the algorithm performs better when the actor and critic share some parameters.\n\n### Results\nCurrently A2C has only been tested on `CartPole-v1`.\n\n![cartpole_results](results/A2C_CartPole.png)\n\nAs you can see, the agent improves well over the first few episodes, only becoming unstable when it gets an average \nepisode reward of 120.\n\nNo benchmarking has been performed on A3C.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fttitcombe%2Fa3c","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fttitcombe%2Fa3c","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fttitcombe%2Fa3c/lists"}