{"id":13935890,"url":"https://github.com/gsurma/cartpole","last_synced_at":"2025-04-03T02:10:19.806Z","repository":{"id":90787491,"uuid":"145177400","full_name":"gsurma/cartpole","owner":"gsurma","description":"OpenAI's cartpole env solver.","archived":false,"fork":false,"pushed_at":"2023-02-17T21:27:04.000Z","size":1087,"stargazers_count":154,"open_issues_count":4,"forks_count":113,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-24T08:19:05.991Z","etag":null,"topics":["ai","cartpole","cartpole-v1","dqn","dqn-solver","machine-learning","openai","openai-gym","python","python27","reinforcement-learning"],"latest_commit_sha":null,"homepage":"https://gsurma.github.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gsurma.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null},"funding":{"patreon":"gsurma"}},"created_at":"2018-08-18T00:25:15.000Z","updated_at":"2025-03-21T16:07:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"d84aaac6-463b-4233-ae75-a87f5732ac05","html_url":"https://github.com/gsurma/cartpole","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gsurma%2Fcartpole","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gsurma%2Fcartpole/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gsurma%2Fcartpole/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gsurma%2Fcartpole/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gsurma","download_url":"https://codeload.github.com/gsurma/cartpole/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246922247,"owners_count":20855345,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","cartpole","cartpole-v1","dqn","dqn-solver","machine-learning","openai","openai-gym","python","python27","reinforcement-learning"],"created_at":"2024-08-07T23:02:10.671Z","updated_at":"2025-04-03T02:10:19.780Z","avatar_url":"https://github.com/gsurma.png","language":"Python","funding_links":["https://patreon.com/gsurma"],"categories":["Python"],"sub_categories":[],"readme":"\u003ch3 align=\"center\"\u003e\n  \u003cimg src=\"assets/cartpole_icon_web.png\" width=\"300\"\u003e\n\u003c/h3\u003e\n\n# Cartpole\n\nReinforcement Learning solution of the [OpenAI's Cartpole](https://gym.openai.com/envs/CartPole-v0/).\n\nCheck out corresponding Medium article: [Cartpole - Introduction to Reinforcement Learning (DQN - Deep Q-Learning)](https://towardsdatascience.com/cartpole-introduction-to-reinforcement-learning-ed0eb5b58288)\n\n## About\n\n\u003e A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The system is controlled by applying a force of +1 or -1 to the cart. The pendulum starts upright, and the goal is to prevent it from falling over. A reward of +1 is provided for every timestep that the pole remains upright. The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center. [source](https://gym.openai.com/envs/CartPole-v0/)\n\n## DQN\nStandard DQN with Experience Replay.\n\n### Hyperparameters:\n\n* GAMMA = 0.95\n* LEARNING_RATE = 0.001\n* MEMORY_SIZE = 1000000\n* BATCH_SIZE = 20\n* EXPLORATION_MAX = 1.0\n* EXPLORATION_MIN = 0.01\n* EXPLORATION_DECAY = 0.995\n\n### Model structure:\n\n1. Dense layer - input: **4**, output: **24**, activation: **relu**\n2. Dense layer - input **24**, output: **24**, activation: **relu**\n3. Dense layer - input **24**, output: **2**, activation: **linear**\n\n* **MSE** loss function\n* **Adam** optimizer\n\n\n## Performance\n\n\u003e CartPole-v0 defines \"solving\" as getting average reward of 195.0 over 100 consecutive trials. [source](https://gym.openai.com/envs/CartPole-v0/)\n\u003e \n\n##### Example trial gif\n\n\u003cimg src=\"assets/cartpole_example.gif\" width=\"200\"\u003e\n\n\n##### Example trial chart\n\n\u003cimg src=\"scores/scores.png\"\u003e\n\n##### Solved trials chart\n\n\u003cimg src=\"scores/solved.png\"\u003e\n\n## Author\n\n**Greg (Grzegorz) Surma**\n\n[**PORTFOLIO**](https://gsurma.github.io)\n\n[**GITHUB**](https://github.com/gsurma)\n\n[**BLOG**](https://medium.com/@gsurma)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgsurma%2Fcartpole","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgsurma%2Fcartpole","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgsurma%2Fcartpole/lists"}