{"id":23078422,"url":"https://github.com/denselance/mcts-simple","last_synced_at":"2025-10-05T05:10:17.908Z","repository":{"id":57440358,"uuid":"461227556","full_name":"DenseLance/mcts-simple","owner":"DenseLance","description":"mcts-simple is a Python3 library that implements Monte Carlo Tree Search and its variants to solve a host of problems, most commonly for reinforcement learning.","archived":false,"fork":false,"pushed_at":"2025-08-08T04:16:15.000Z","size":12018,"stargazers_count":29,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-25T04:58:32.799Z","etag":null,"topics":["game","mcts","mcts-algorithm","python","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DenseLance.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-02-19T15:13:02.000Z","updated_at":"2025-08-08T04:16:18.000Z","dependencies_parsed_at":"2023-12-01T15:26:40.947Z","dependency_job_id":"a72c1165-d948-4c51-80c9-9108bbd3ed7b","html_url":"https://github.com/DenseLance/mcts-simple","commit_stats":{"total_commits":11,"total_committers":2,"mean_commits":5.5,"dds":0.09090909090909094,"last_synced_commit":"10d8d4ce46df9a5eec0d646a2980a02be37bf1a6"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/DenseLance/mcts-simple","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DenseLance%2Fmcts-simple","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DenseLance%2Fmcts-simple/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DenseLance%2Fmcts-simple/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DenseLance%2Fmcts-simple/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DenseLance","download_url":"https://codeload.github.com/DenseLance/mcts-simple/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DenseLance%2Fmcts-simple/sbom","scorecard":{"id":39350,"data":{"date":"2025-08-11","repo":{"name":"github.com/DenseLance/mcts-simple","commit":"9af9e4575af86f0f05f7172aa1994bd421cb4c1c"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.2,"checks":[{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Maintained","score":2,"reason":"3 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":0,"reason":"Found 1/19 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":9,"reason":"1 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2017-74"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 2 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-14T21:06:37.438Z","repository_id":57440358,"created_at":"2025-08-14T21:06:37.438Z","updated_at":"2025-08-14T21:06:37.438Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278411261,"owners_count":25982368,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-05T02:00:06.059Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["game","mcts","mcts-algorithm","python","reinforcement-learning"],"created_at":"2024-12-16T10:54:31.748Z","updated_at":"2025-10-05T05:10:17.903Z","avatar_url":"https://github.com/DenseLance.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mcts-simple\n\n[![Python](https://img.shields.io/pypi/pyversions/mcts-simple.svg?style=plastic)](https://badge.fury.io/py/mcts-simple) [![Version](https://img.shields.io/pypi/v/mcts-simple.svg?logo=pypi)](https://badge.fury.io/py/mcts-simple) [![GitHub license](https://img.shields.io/github/license/denselance/mcts-simple.svg)](https://github.com/DenseLance/mcts-simple/blob/main/LICENSE) [![PyPI Downloads](https://static.pepy.tech/badge/mcts-simple/month)](https://pepy.tech/projects/mcts-simple)\n\n*mcts-simple* is a Python3 library that implements Monte Carlo Tree Search and its variants to solve a host of problems, most commonly for reinforcement learning.\n\n### Version Updates (v1.1)\n\n* Reduced time taken to train MCTS and its variants.\n  \n  * \u003cb\u003eTic-Tac-Toe Example:\u003c/b\u003e 9 hours → 100 seconds ($10^6$ iterations)\n  \n  * \u003cb\u003eCartPole Example:\u003c/b\u003e 15 hours → 22 minutes ($10^6$ iterations)\n\n* Ported over to `msgpack` for saving and loading of MCTS objects.\n  \n  * Eliminates security vulnerabilities that come with `pickle`.\n  \n  * `save()` and `load()` is expected to be slightly slower with larger memory footprint.\n\n* Added the `seed` parameter and `set_seed(seed)` method for reproducible MCTS training.\n  \n  * Direct usage of `random` and global random seed will no longer affect MCTS training. However, global random seed may still affect game internals.\n  \n  * Random number generator (RNG) for MCTS is now independent of the game's internal RNG.\n  \n  * RNG is saved when exporting a trained MCTS.\n\n* Support for Python 3.7 has been deprecated.\n\n### Monte Carlo Tree Search (MCTS)\n\nMCTS attempts to identify the most promising moves at each state by choosing random actions at that state for every episode (playouts/rollouts). The final game result of each episode is then used to determine the weight of all nodes traversed during that episode so that the probability of choosing an action that yields higher current and potential rewards is increased.\n\n**There are 4 stages to the MCTS:**\n\n1. Selection\n   - Traverse through the search tree from the root node to a leaf node, while only selecting the most promising child nodes. Leaf node in this case refers to a node that has not yet gone through the expansion stage, rather than its traditional definition which is \"a node without child nodes\".\n2. Expansion\n   - If the leaf node does not lead to an outcome to the episode (e.g. win/lose/draw), create at least one child node for that leaf node and choose one child node from those created.\n3. Simulation\n   - Complete one episode starting from the chosen child node, where random actions are chosen for future states. An episode is only completed when an outcome can be yielded from it.\n4. Backpropagation\n   - The outcome yielded from the simulated episode in stage 3 should be used to update information in traversed nodes.\n\n**Note:**\n\n* We assume that states are unique.\n* Root node's score is almost never evaluated, and at most only the number of visits, *n*, is used.\n* We assume the node with the highest average returns is the most visited node.\n\n### Upper Confidence bounds applied to Trees (UCT)\n\nUCT, a variation of MCTS, is often used instead of vanilla MCTS for a few reasons, mainly:\n\n1. MCTS emphasizes entirely on exploitation. On the other hand, UCT is able to balance exploration and exploitation.\n2. MCTS may favour a losing move despite the presence of one or few forced refutations. UCT attempts to deal with this limitation of the original MCTS.\n\nUCT uses the UCB1 formula to evaluate actions at each state. The exploration parameter c in the UCB1 formula is theoretically equal to $\\sqrt{2}$, but it can be changed depending on each situation.\n\n### Open Loop MCTS\n\nMost of the time, a closed loop MCTS is sufficient in dealing with reinforcement learning problems. However, when it comes to games that have non-deterministic or non-discrete states, an open loop MCTS is required. Open loop MCTS would completely eliminate the need for chance nodes. Transpositions will also not be considered since we would ignore the game state entirely. Since the tree is now significantly smaller in an open loop MCTS, the branching factor is also a lot smaller and evaluations may be less accurate. This would also mean that results are more likely to converge at a faster rate. This variant of MCTS can be used for deterministic games as well.\n\n**Note:**\n\n* Since there is no transposition table in an open loop MCTS, we cannot change the initial starting state.\n\n### How to use mcts-simple\n\n*mcts-simple* only supports Python 3.8 and above. If you want to use *mcts-simple* for other versions of Python, it may not function as intended.\n\n#### Dependencies\n\n*mcts-simple* requires the following libraries, which is automatically installed together with it:\n\n* `tqdm`\n* `msgpack`\n\nThe following libraries are used in our examples:\n\n* `gymnasium` and `imageio` (for CartPole)\n\n#### User installation\n\nTo install the module:\n\n```cmd\npip install mcts-simple\n```\n\nTo import the module:\n\n```python\nfrom mcts_simple import *\n```\n\n#### Creating your own game environment\n\nCreate a class for your game by inheriting the `Game` class from *mcts-simple*, and define the following methods for your class:\n\n| Method                     | What it does                                                                                                                                                               |\n|:--------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|\n| `__init__()`               | Initializes the object.                                                                                                                                                    |\n| `render()`                 | Returns a visual representation of the current state of the game.                                                                                                          |\n| `get_state()`              | Returns current state of the game.\u003cbr\u003e\u003cb\u003eNote:\u003c/b\u003e MCTS will automatically apply `str()` on the state so that it becomes hashable and compatible with Python's dictionary. |\n| `number_of_players()`      | Returns number of players.                                                                                                                                                 |\n| `current_player()`         | Returns the player that is taking an action this turn.\u003cbr\u003e**Note:** Players are labelled from `0` to `number of players - 1`.                                              |\n| `possible_actions()`       | Returns the possible actions that can be taken by current player this turn.                                                                                                |\n| `take_action(action: int)` | Player takes action.\u003cbr\u003e**Note:** Even if action leads to the end of the game, next player should still be chosen.                                                         |\n| `has_outcome()`            | Returns `True` if game has ended. Returns `False` if game is still ongoing.                                                                                                |\n| `winner()`                 | Returns empty list if all players lose. Returns list of players if game ends in a draw. Returns list of winners if at least 1 player wins.                                 |\n\nAfter creating your game environment, you're basically done! You can train and export your MCTS with just 3 lines of code. To reproduce results, you can set the `seed` parameter, or use the `set_seed()` method. Note that `YourGame()` should be the game environment that you have created.\n\n```python\nmcts = MCTS(YourGame(), training = True)\nmcts.self_play(iterations = 50000)\nmcts.save(\"game.mcts\")\n```\n\nYou can import your trained MCTS and see how it performs with another 3 lines of code.\n\n```python\nmcts = MCTS(YourGame(), training = False)\nmcts.load(\"game.mcts\")\nmcts.self_play()\n```\n\nWe recommend enabling `transposition_table` (if the option is available), as it can substantially reduce the number of nodes created and improve efficiency. If you have any issues in creating your environment, you can browse the source code or check out the examples provided \u003ca href = \"https://github.com/DenseLance/mcts-simple/tree/main/examples\"\u003ehere\u003c/a\u003e.\n\n### Contributions\n\nI appreciate it if you can contribute to this project as this is my first public Python package. If you think that something is wrong with my code, you can open an issue and I'll try my best to resolve it!\n\n### Work in Progress\n\n- [ ] Implement PUCT (Predictor + UCT).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdenselance%2Fmcts-simple","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdenselance%2Fmcts-simple","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdenselance%2Fmcts-simple/lists"}