{"id":24538894,"url":"https://github.com/valinsogna/reinforcemate","last_synced_at":"2026-05-22T05:08:23.696Z","repository":{"id":180460021,"uuid":"665181149","full_name":"valinsogna/ReinForceMate","owner":"valinsogna","description":"Advanced RL algorithms for two simplified versions of chess. Shortest Path finds the minimal moves between two cells based on piece capabilities. Capture Pieces trains against random opponents aiming for maximal captures in set moves. Features Deep Q-Learning, Policy Iteration, TD and more.","archived":false,"fork":false,"pushed_at":"2023-09-18T11:03:41.000Z","size":6935,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-22T15:36:40.149Z","etag":null,"topics":["deep-q-learning","expected-sarsa","q-learning","reinforcement-learning","sarsa","sarsa-lambda"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/valinsogna.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-11T16:03:06.000Z","updated_at":"2023-09-18T10:31:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"2a599f70-4ab4-465e-b41d-f254945577d0","html_url":"https://github.com/valinsogna/ReinForceMate","commit_stats":null,"previous_names":["valinsogna/reinforcemate"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valinsogna%2FReinForceMate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valinsogna%2FReinForceMate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valinsogna%2FReinForceMate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valinsogna%2FReinForceMate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/valinsogna","download_url":"https://codeload.github.com/valinsogna/ReinForceMate/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243817896,"owners_count":20352626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-q-learning","expected-sarsa","q-learning","reinforcement-learning","sarsa","sarsa-lambda"],"created_at":"2025-01-22T15:29:27.876Z","updated_at":"2026-05-22T05:08:18.676Z","avatar_url":"https://github.com/valinsogna.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# README\n\n## Overview\n\nThis script contains a suite of reinforcement learning algorithms, all applied to a game of chess. The game is viewed as a Markov Decision Process (MDP), and the algorithms learn a policy to play the game. The algorithms included are Policy Iteration, Temporal Difference, Expected Temporal Difference, Temporal Difference Lambda, and Q-Learning.\n\n## Getting Started\n\nYou need Python installed on your computer to run the script. If you don't have Python installed, you can download it from the [official website](https://www.python.org/downloads/).\n\nAlso, this script relies on the following Python libraries:\n- torch\n- matplotlib\n- numpy\n- tqdm\n- chess\n\nYou can install them using pip:\n\n```\npip install torch matplotlib numpy tqdm chess\n```\n\n## Installation\nYou can clone the ReinForceMate repository and install the package using the following commands:\n\n```bash\n\u003e git clone https://github.com/valinsogna/ReinForceMate\n\u003e cd ReinForceMate\n\u003e ./install.sh\n```\n\n## Usage\n\nThe script is structured into different sections, each applying a different algorithm.\n\nFirst, the script imports the necessary modules from the ReinForceMate package, along with the other necessary libraries.\n\n```python\nfrom ReinForceMate import Q_LearningMove\nfrom ReinForceMate import TemporalDifference\nfrom ReinForceMate import TemporalDifferenceLambda\nfrom ReinForceMate import PolicyIteration\nfrom ReinForceMate import ExpectedTemporalDifference\nimport torch\nimport time\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom tqdm import tqdm\n```\n\nThe script then applies several reinforcement learning algorithms to learn a policy for playing chess.\n\nFor example, to apply the Policy Iteration algorithm:\n\n```python\nr = PolicyIteration(piece='bishop')\npolicy_iter_rewards = r.run_episode()\n```\n\nThe `run_episode` method plays a full game of chess using the current policy and returns the total reward obtained.\n\nAfter each algorithm has been run, the script will visualize the learnt policy, for example:\n\n```python\nr.visualize_policy()\n```\n\nThe script also evaluates the performance of the algorithms. It plays a certain number of episodes with each algorithm and measures the time it takes to complete. The average cumulative reward for each algorithm is then plotted:\n\n```python\nfor result in results:\n    plt.plot(np.cumsum(result['rewards']) / np.arange(1, n_of_episodes+1), label=result['name'])\nplt.legend()\nplt.title('Average Cumulative Reward')\nplt.show()\n```\n\nThe performance of different piece types with Policy Iteration is evaluated in a similar way:\n\n```python\nalgorithms = [\n    PolicyIteration(piece='king'),\n    PolicyIteration(piece='rook'),\n    PolicyIteration(piece='knight'),\n    PolicyIteration(piece='bishop')\n]\n```\n\nFinally, Q-Learning performance is evaluated for different values of alpha:\n\n```python\nfor alpha in tqdm([0.001, 0.01, 0.1, 0.2, 0.3, 0.4, 0.5]):\n    td = Q_LearningMove(piece='king')\n    for episode in range(n_of_episodes):\n        reward = td.run_episode(episode, alpha=alpha)\n    rewards.append(reward)\n```\n\nPlease refer to the source code comments for a more in-depth understanding of the workings of each algorithm.\n\n## Customization\n\nYou can customize the algorithms by modifying the parameters they accept. For example, you can change the `piece` parameter in the `PolicyIteration` instantiation to apply the algorithm to a different chess piece:\n\n```python\nr = PolicyIteration(piece='rook')\n```\n\nYou can also change the number of episodes played in the evaluation stage by modifying the `n_of_episodes` variable:\n\n```python\nn_of_episodes = 200\n```\n\nFor Q-Learning, you can change the values of alpha to experiment with:\n\n```python\nfor alpha in tqdm([0.001, 0.01, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]):\n```\n\n## Additional Details\n\nThis script also generates csv files with performance results and chess piece values, as well as a PGN file containing the game played during the Q-Learning stage. Please ensure that the directory you are running the script in allows file writing to access these generated files.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalinsogna%2Freinforcemate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvalinsogna%2Freinforcemate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalinsogna%2Freinforcemate/lists"}