{"id":17522971,"url":"https://github.com/google-deepmind/searchless_chess","last_synced_at":"2025-10-15T00:18:10.224Z","repository":{"id":232578814,"uuid":"784223082","full_name":"google-deepmind/searchless_chess","owner":"google-deepmind","description":"Grandmaster-Level Chess Without Search","archived":false,"fork":false,"pushed_at":"2025-01-10T12:51:32.000Z","size":2215,"stargazers_count":580,"open_issues_count":1,"forks_count":30,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-06-17T00:48:30.386Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2402.04494","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-deepmind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-09T12:35:41.000Z","updated_at":"2025-06-14T17:44:34.000Z","dependencies_parsed_at":"2024-06-14T11:52:53.204Z","dependency_job_id":null,"html_url":"https://github.com/google-deepmind/searchless_chess","commit_stats":null,"previous_names":["google-deepmind/searchless_chess"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/google-deepmind/searchless_chess","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsearchless_chess","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsearchless_chess/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsearchless_chess/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsearchless_chess/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-deepmind","download_url":"https://codeload.github.com/google-deepmind/searchless_chess/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsearchless_chess/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279031830,"owners_count":26089378,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-20T13:01:54.703Z","updated_at":"2025-10-15T00:18:10.189Z","avatar_url":"https://github.com/google-deepmind.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Amortized Planning with Large-Scale Transformers: A Case Study on Chess\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/google-deepmind/searchless_chess/master/overview.svg\" alt=\"Overview figure\"/\u003e\n\u003c/p\u003e\n\n\nThis repository provides an implementation of our NeurIPS 2024 paper [Amortized Planning with Large-Scale Transformers: A Case Study on Chess](https://arxiv.org/abs/2402.04494).\n\n\u003e This paper uses chess, a landmark planning problem in AI, to assess transformers’ performance on a planning task where memorization is futile — even at a large scale.\nTo this end, we release ChessBench, a large-scale benchmark dataset of 10 million chess games with legal move and value annotations (15 billion data points) provided by Stockfish 16, the state-of-the-art chess engine.\nWe train transformers with up to 270 million parameters on ChessBench via supervised learning and perform extensive ablations to assess the impact of dataset size, model size, architecture type, and different prediction targets (state-values, action-values, and behavioral cloning).\nOur largest models learn to predict action-values for novel boards quite accurately, implying highly non-trivial generalization.\nDespite performing no explicit search, our resulting chess policy solves challenging chess puzzles and achieves a surprisingly strong Lichess blitz Elo of 2895 against humans (grandmaster level).\nWe also compare to Leela Chess Zero and AlphaZero (trained without supervision via self-play) with and without search.\nWe show that, although a remarkably good approximation of Stockfish’s search-based algorithm can be distilled into large-scale transformers via supervised learning, perfect distillation is still beyond reach, thus making ChessBench well-suited for future research.\n\n\n## Contents\n\n```\n.\n|\n├── BayesElo                        - Elo computation (need to be installed)\n|\n├── checkpoints                     - Model checkpoints (need to be downloaded)\n|   ├── 136M\n|   ├── 270M\n|   └── 9M\n|\n├── data                            - Datasets (need to be downloaded)\n|   ├── eco_openings.csv\n|   ├── test\n|   ├── train\n|   └── puzzles.csv\n|\n├── lc0                             - Leela Chess Zero (needs to be installed)\n|\n├── src\n|   ├── engines\n|   |   ├── constants.py            - Engine constants\n|   |   ├── engine.py               - Engine interface\n|   |   ├── lc0_engine.py           - Leela Chess Zero engine\n|   |   ├── neural_engines.py       - Neural engines\n|   |   └── stockfish_engine.py     - Stockfish engine\n|   |\n|   ├── bagz.py                     - Readers for our .bag data files\n|   ├── config.py                   - Experiment configurations\n|   ├── constants.py                - Constants, interfaces, and types\n|   ├── data_loader.py              - Data loader\n|   ├── metrics_evaluator.py        - Metrics (e.g., Kendall's tau) evaluator\n|   ├── puzzles.py                  - Puzzle evaluation script\n|   ├── searchless_chess.ipynb      - Model analysis notebook\n|   ├── tokenizer.py                - Chess board tokenization\n|   ├── tournament.py               - Elo tournament script\n|   ├── train.py                    - Example training + evaluation script\n|   ├── training.py                 - Training loop\n|   ├── training_utils.py           - Training utility functions\n|   ├── transformer.py              - Decoder-only Transformer\n|   └── utils.py                    - Utility functions\n|\n├── Stockfish                       - Stockfish (needs to be installed)\n|\n├── README.md\n└── requirements.txt                - Dependencies\n```\n\n\n## Installation\n\nClone the source code into a local directory:\n\n```bash\ngit clone https://github.com/google-deepmind/searchless_chess.git\ncd searchless_chess\n```\n\nThis repository requires Python 3.10.\n`pip install -r requirements.txt` will install all required dependencies.\nThis is best done inside a [conda environment](https://www.anaconda.com/).\nTo that end, install [Anaconda](https://www.anaconda.com/download#downloads).\nThen, create and activate the conda environment:\n\n```bash\nconda create --name searchless_chess python=3.10\nconda activate searchless_chess\n```\n\nInstall `pip` and use it to install all the dependencies:\n\n```bash\nconda install pip\npip install -r requirements.txt\n```\n\nIf you have a GPU available (highly recommended for fast training), then you can install JAX with CUDA support.\n\n```bash\npip install --upgrade \"jax[cuda12_pip]\" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html\n```\nNote that the jax version must correspond to the existing CUDA installation you wish to use (CUDA 12 in the example above).\nPlease see the [JAX documentation](https://github.com/jax-ml/jax#installation) for more details.\n\n### Installing Stockfish\n\nDownload and compile the latest version of Stockfish (for Unix-like systems):\n\n```bash\ngit clone https://github.com/official-stockfish/Stockfish.git\ncd Stockfish/src\nmake -j profile-build ARCH=x86-64-avx2\ncd ../..\n```\n\n\n### Installing Leela Chess Zero\n\nFollow the [Lc0 download instructions](https://github.com/LeelaChessZero/lc0?tab=readme-ov-file#downloading-source), i.e.,\n\n```bash\ngit clone -b release/0.30 --recurse-submodules https://github.com/LeelaChessZero/lc0.git\n```\n\nThen build the engine as described in the [Lc0 build instructions](https://github.com/LeelaChessZero/lc0?tab=readme-ov-file#building-and-running-lc0).\n\nWe evaluate Lc0 with the largest-possible network from [Lc0's model catalogue](https://lczero.org/play/networks/bestnets/), i.e., the `Large` network.\nTo download that network, run the following command:\n\n```bash\ncd lc0/build/release\nwget https://storage.lczero.org/files/768x15x24h-t82-swa-7464000.pb.gz\ngzip -d 768x15x24h-t82-swa-7464000.pb.gz\ncd ../../..\n```\n\n### Installing BayesElo\n\nTo compute the Elos for the different agents, we require [BayesElo](https://www.remi-coulom.fr/Bayesian-Elo/), which can be installed as follows:\n\n```bash\nwget https://www.remi-coulom.fr/Bayesian-Elo/bayeselo.tar.bz2\ntar -xvjf bayeselo.tar.bz2\ncd BayesElo\nmake bayeselo\ncd ..\n```\n\n\n### Downloading the Datasets\n\nTo download our datasets to the correct locations, run the following command:\n\n```bash\ncd data\n./download.sh\ncd ..\n```\n\nWe also provide the individual dataset download links in the following table\n(the action-value dataset is sharded into 2148 files due to its size and only\nthe link to the first shard is listed below):\n\n| Split | Action-Value | Behavioral Cloning | State-Value | Puzzles |\n|------ | ------------ | ------------------ | ----------- | ------- |\n| Train | [1.2 GB](https://storage.googleapis.com/searchless_chess/data/train/action_value-00000-of-02148_data.bag) (of 1.1 TB) | [34 GB](https://storage.googleapis.com/searchless_chess/data/train/behavioral_cloning_data.bag) | [36 GB](https://storage.googleapis.com/searchless_chess/data/train/state_value_data.bag) | - |\n| Test  | [141 MB](https://storage.googleapis.com/searchless_chess/data/test/action_value_data.bag) | [4.1 MB](https://storage.googleapis.com/searchless_chess/data/test/behavioral_cloning_data.bag) | [4.4 MB](https://storage.googleapis.com/searchless_chess/data/test/state_value_data.bag) | [4.5 MB](https://storage.googleapis.com/searchless_chess/data/puzzles.csv) |\n\n\n### Downloading the Model Checkpoints\n\nTo download the pretrained models to the correct locations, run the following command:\n\n```bash\ncd checkpoints\n./download.sh\ncd ..\n```\n\n\n## Usage\n\nBefore running any code, make sure to activate the conda environment and set the `PYTHONPATH`:\n\n```bash\nconda activate searchless_chess\nexport PYTHONPATH=$(pwd)/..\n```\n\n### Training\n\nTo train a model locally, run the following command:\n\n```bash\ncd src\npython train.py\ncd ..\n```\nThe model checkpoints will be saved to `/checkpoints/local`.\n\n### Puzzles\n\nTo evaluate a model's puzzle accuracy, run the following command:\n\n```bash\ncd src\npython puzzles.py --num_puzzles 10 --agent=local\ncd ..\n```\n\n`puzzles.py` supports the following agents:\n\n* the locally trained model: `local`\n* the pretrained models: `9M`, `136M`, and `270M`\n* the Stockfish engines: `stockfish` and `stockfish_all_moves`\n* the Lc0 engines: `leela_chess_zero_depth_1`, `leela_chess_zero_policy_net`, and `leela_chess_zero_400_sims`\n\n\n### Tournament Elo\n\nTo compute the Elo for the different agents, run the tournament to play games between them and then compute the Elo for the PGN file generated by the tournament (more information on BayesElo can be found [here](https://www.remi-coulom.fr/Bayesian-Elo/)):\n\n```bash\ncd src\npython tournament.py --num_games=200\n\ncd ../BayesElo\n\n./bayeselo\n\u003e ...\nResultSet\u003ereadpgn ../data/tournament.pgn\n\u003e N game(s) loaded, 0 game(s) with unknown result ignored.\nResultSet\u003eelo\nResultSet-EloRating\u003emm\n\u003e 00:00:00,00\nResultSet-EloRating\u003eexactdist\n\u003e 00:00:00,00\nResultSet-EloRating\u003eratings\n\u003e ...\n\ncd ..\n```\n\n### Analysis Notebook\n\nTo investigate the model's behavior (e.g., to compute the win percentage for all legal moves), start a notebook server and then open `src/searchless_chess.ipynb` in your browser:\n\n```bash\njupyter notebook\n```\n\n\n## Citing this work\n\n```latex\n@inproceedings{ruoss2024amortized,\n  author       = {Anian Ruoss and\n                  Gr{\\'{e}}goire Del{\\'{e}}tang and\n                  Sourabh Medapati and\n                  Jordi Grau{-}Moya and\n                  Li Kevin Wenliang and\n                  Elliot Catt and\n                  John Reid and\n                  Cannada A. Lewis and\n                  Joel Veness and\n                  Tim Genewein},\n  title        = {Amortized Planning with Large-Scale Transformers: A Case Study\n                  on Chess},\n  booktitle    = {NeurIPS},\n  year         = {2024}\n}\n```\n\n## License and disclaimer\n\nCopyright 2024 DeepMind Technologies Limited\n\nAll software is licensed under the Apache License, Version 2.0 (Apache 2.0);\nyou may not use this file except in compliance with the Apache 2.0 license.\nYou may obtain a copy of the Apache 2.0 license at:\nhttps://www.apache.org/licenses/LICENSE-2.0\n\nThe model weights are licensed under Creative Commons Attribution 4.0 (CC-BY).\nYou may obtain a copy of the CC-BY license at:\nhttps://creativecommons.org/licenses/by/4.0/legalcode\n\nSome portions of the dataset are in the public domain by a\nCreative Commons CC0 license from lichess.org.\nThe remainder of the dataset is licensed under\nCreative Commons Attribution 4.0 (CC-BY).\nYou may obtain a copy of the CC-BY license at:\nhttps://creativecommons.org/licenses/by/4.0/legalcode.\n\nUnless required by applicable law or agreed to in writing, software and\nmaterials distributed under the Apache 2.0 or CC-BY licenses are\ndistributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,\neither express or implied. See the licenses for the specific language governing\npermissions and limitations under those licenses.\n\nThis is not an official Google product.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fsearchless_chess","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-deepmind%2Fsearchless_chess","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fsearchless_chess/lists"}