{"id":20432357,"url":"https://github.com/captaine/rlsolutionssutton","last_synced_at":"2026-05-26T23:04:27.025Z","repository":{"id":134234113,"uuid":"205565624","full_name":"CaptainE/RLSolutionsSutton","owner":"CaptainE","description":"attempted solutions for Sutton-barto version 2.0","archived":false,"fork":false,"pushed_at":"2019-08-31T16:01:40.000Z","size":2312,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-15T18:40:06.557Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CaptainE.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-31T15:58:44.000Z","updated_at":"2020-11-13T20:46:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"8356d0c2-d3ab-4ad5-82dc-09cdd987c07f","html_url":"https://github.com/CaptainE/RLSolutionsSutton","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainE%2FRLSolutionsSutton","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainE%2FRLSolutionsSutton/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainE%2FRLSolutionsSutton/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainE%2FRLSolutionsSutton/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CaptainE","download_url":"https://codeload.github.com/CaptainE/RLSolutionsSutton/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241975156,"owners_count":20051431,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T08:14:43.059Z","updated_at":"2025-12-04T00:03:21.391Z","avatar_url":"https://github.com/CaptainE.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"### Overview\n\nThis repository provides code, exercises and solutions for popular Reinforcement Learning algorithms. These are meant to serve as a learning tool to complement the theoretical materials from\n\n- [Reinforcement Learning: An Introduction (2nd Edition)](http://incompleteideas.net/book/bookdraft2018jan1.pdf)\n- [David Silver's Reinforcement Learning Course](http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html)\n\nEach folder in corresponds to one or more chapters of the above textbook and/or course. In addition to exercises and solution, each folder also contains a list of learning goals, a brief concept summary, and links to the relevant readings.\n\nAll code is written in Python 3 and uses RL environments from [OpenAI Gym](https://gym.openai.com/). Advanced techniques use [Tensorflow](https://www.tensorflow.org/) for neural network implementations.\n\n\n### Table of Contents\n\n- [Introduction to RL problems \u0026 OpenAI Gym](Introduction/)\n- [MDPs and Bellman Equations](MDP/)\n- [Dynamic Programming: Model-Based RL, Policy Iteration and Value Iteration](DP/)\n- [Monte Carlo Model-Free Prediction \u0026 Control](MC/)\n- [Temporal Difference Model-Free Prediction \u0026 Control](TD/)\n- [Function Approximation](FA/)\n- [Deep Q Learning](DQN/) (WIP)\n- [Policy Gradient Methods](PolicyGradient/) (WIP)\n- Learning and Planning (WIP)\n- Exploration and Exploitation (WIP)\n\n\n### List of Implemented Algorithms\n\n- [Dynamic Programming Policy Evaluation](DP/Policy%20Evaluation%20Solution.ipynb)\n- [Dynamic Programming Policy Iteration](DP/Policy%20Iteration%20Solution.ipynb)\n- [Dynamic Programming Value Iteration](DP/Value%20Iteration%20Solution.ipynb)\n- [Monte Carlo Prediction](MC/MC%20Prediction%20Solution.ipynb)\n- [Monte Carlo Control with Epsilon-Greedy Policies](MC/MC%20Control%20with%20Epsilon-Greedy%20Policies%20Solution.ipynb)\n- [Monte Carlo Off-Policy Control with Importance Sampling](MC/Off-Policy%20MC%20Control%20with%20Weighted%20Importance%20Sampling%20Solution.ipynb)\n- [SARSA (On Policy TD Learning)](TD/SARSA%20Solution.ipynb)\n- [Q-Learning (Off Policy TD Learning)](TD/Q-Learning%20Solution.ipynb)\n- [Q-Learning with Linear Function Approximation](FA/Q-Learning%20with%20Value%20Function%20Approximation%20Solution.ipynb)\n- [Deep Q-Learning for Atari Games](DQN/Deep%20Q%20Learning%20Solution.ipynb)\n- [Double Deep-Q Learning for Atari Games](DQN/Double%20DQN%20Solution.ipynb)\n- Deep Q-Learning with Prioritized Experience Replay (WIP)\n- [Policy Gradient: REINFORCE with Baseline](PolicyGradient/CliffWalk%20REINFORCE%20with%20Baseline%20Solution.ipynb)\n- [Policy Gradient: Actor Critic with Baseline](PolicyGradient/CliffWalk%20Actor%20Critic%20Solution.ipynb)\n- [Policy Gradient: Actor Critic with Baseline for Continuous Action Spaces](PolicyGradient/Continuous%20MountainCar%20Actor%20Critic%20Solution.ipynb)\n- Deterministic Policy Gradients for Continuous Action Spaces (WIP)\n- Deep Deterministic Policy Gradients (DDPG) (WIP)\n- [Asynchronous Advantage Actor Critic (A3C)](PolicyGradient/a3c)\n\n\n### Resources\n\nTextbooks:\n\n- [Reinforcement Learning: An Introduction (2nd Edition)](http://incompleteideas.net/book/bookdraft2018jan1.pdf)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcaptaine%2Frlsolutionssutton","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcaptaine%2Frlsolutionssutton","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcaptaine%2Frlsolutionssutton/lists"}