{"id":21685867,"url":"https://github.com/sodascience/lichess_db","last_synced_at":"2026-05-02T23:35:56.150Z","repository":{"id":239745393,"uuid":"714969278","full_name":"sodascience/lichess_db","owner":"sodascience","description":"Lichess game header data as parquet files","archived":false,"fork":false,"pushed_at":"2025-01-03T11:21:58.000Z","size":1376,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-01-25T12:09:35.893Z","etag":null,"topics":["database","lichess","lichess-database"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sodascience.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-06T08:12:45.000Z","updated_at":"2024-12-06T14:23:46.000Z","dependencies_parsed_at":"2024-05-30T13:05:08.565Z","dependency_job_id":"191ca588-fd50-4808-9664-c4503813a1ca","html_url":"https://github.com/sodascience/lichess_db","commit_stats":null,"previous_names":["sodascience/lichess_db"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sodascience%2Flichess_db","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sodascience%2Flichess_db/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sodascience%2Flichess_db/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sodascience%2Flichess_db/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sodascience","download_url":"https://codeload.github.com/sodascience/lichess_db/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244609358,"owners_count":20480780,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","lichess","lichess-database"],"created_at":"2024-11-25T16:23:26.941Z","updated_at":"2025-10-15T08:53:53.244Z","avatar_url":"https://github.com/sodascience.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lichess database to parquet converter\n\nThIn this repository, you can find code to efficiently download millions of chess games from [database.lichess.org](https://database.lichess.org) to your computer using Apache `parquet` files, which can be loaded to tables efficiently using [`polars`](https://pola.rs):\n\n```python\nimport polars as pl\nchess_df = pl.scan_parquet(\"lichess_parquet/*.parquet\")\nchess_df.head().collect()\n```\n\n|    | ID       | UTCDate             | UTCTime   | White            | Black             | Result   |   WhiteElo |   BlackElo |   WhiteRatingDiff |   BlackRatingDiff | ECO   | Opening                                     | TimeControl   | Termination   |\n|---:|:---------|:--------------------|:----------|:-----------------|:------------------|:---------|-----------:|-----------:|------------------:|------------------:|:------|:--------------------------------------------|:--------------|:--------------|\n|  0 | j1dkb5dw | 2012-12-31 00:00:00 | 23:01:03  | BFG9k            | mamalak           | 1-0      |       1639 |       1403 |                 5 |                -8 | C00   | French Defense: Normal Variation            | 600+8         | Normal        |\n|  1 | a9tcp02g | 2012-12-31 00:00:00 | 23:04:12  | Desmond_Wilson   | savinka59         | 1-0      |       1654 |       1919 |                19 |               -22 | D04   | Queen's Pawn Game: Colle System, Anti-Colle | 480+2         | Normal        |\n|  2 | szom2tog | 2012-12-31 00:00:00 | 23:03:15  | Kozakmamay007    | VanillaShamanilla | 1-0      |       1643 |       1747 |                13 |               -94 | C50   | Four Knights Game: Italian Variation        | 420+17        | Normal        |\n|  3 | rklpc7mk | 2012-12-31 00:00:00 | 23:04:57  | Naitero_Nagasaki | 800               | 0-1      |       1824 |       1973 |                -6 |                 8 | B12   | Caro-Kann Defense: Goldman Variation        | 60+1          | Normal        |\n|  4 | 1xb3os63 | 2012-12-31 00:00:00 | 23:02:37  | nichiren1967     | Naitero_Nagasaki  | 0-1      |       1765 |       1815 |        -9 |                 9 | C00   | French Defense: La Bourdonnais Variation    | 60+1          | Normal        |\n\n## Installation and usage\n\n1. Clone or download this repository\n2. Install requirements: `pip install -r requirements.txt`\n3. `python ingest_lichess.py`\n\n    The command accepts the following arguments:\n    + `--start` start year for download (default: 2013)\n    + `--end` end year (default: current year)\n    + `--months` months to download; list months by number, seperated by spaces (example `--months 1 2 3` for first quarter) (optional; defaults to all months)\n    + `--parquet-dir` path to write Parquet-files to (default: `./lichess_parquet`)\n    + `--include-moves` whether to include each game's moves in the data (default: False). Take care, this increases the size of the data dramatically.\n    + `--debug` display debug info while downloading (default: False)\n\n4. Wait a good while (this results in tens of gigabytes of data!!). To avoid memory problems while downloading, there is a limit of 1M games per Parquet file. Hence, there will be multiple files per year/month (`2023_05_001.parquet`, `2023_05_002.parquet`, etc).\n\n5. Open [`eda.ipynb`](eda.ipynb) for examples on how to read, filter and visualize the data\n\n## Some plots\n\nThe daily number of games played is increasing strongly over the years.\n![](img/gamecount_plot.png)\n\nThe most popular openings are as follows:\n![](img/opening_plot.png)\n\n\n\n## Contributing\n\nContributions are what make the open source community an amazing place\nto learn, inspire, and create. Any contributions you make are **greatly\nappreciated**.\n\nPlease refer to the\n[CONTRIBUTING](https://github.com/sodascience/lichess_db/blob/main/CONTRIBUTING.md)\nfile for more information on issues and pull requests.\n\n\n\n## License and citation\n\nThe package `lichess_db` is published under an MIT license. \nThe [lichess data](https://database.lichess.org/) is licensed under the [Creative Commons CC0 license](https://creativecommons.org/publicdomain/zero/1.0/).\n\n\n## Contact\n\nThis project is developed and maintained by the [ODISSEI Social Data\nScience (SoDa)](https://odissei-data.nl/nl/soda/) team.\n\n\u003cimg src=\"soda_logo.png\" alt=\"SoDa logo\" width=\"250px\"/\u003e\n\nDo you have questions, suggestions, or remarks? File an issue in the issue\ntracker or feel free to contact the team via\nhttps://odissei-data.nl/en/using-soda/.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsodascience%2Flichess_db","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsodascience%2Flichess_db","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsodascience%2Flichess_db/lists"}