{"id":25435768,"url":"https://github.com/quentin18/gymnasium-2048","last_synced_at":"2025-11-01T03:30:19.634Z","repository":{"id":219413164,"uuid":"677786154","full_name":"Quentin18/gymnasium-2048","owner":"Quentin18","description":"Gymnasium environment for the game 2048","archived":false,"fork":false,"pushed_at":"2024-12-01T10:56:22.000Z","size":2575,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-01T11:31:05.735Z","etag":null,"topics":["2048","2048-ai","2048-game","gymnasium","n-tuple-networks","pygame","temporal-differencing-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Quentin18.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-08-12T16:21:58.000Z","updated_at":"2024-12-01T10:56:27.000Z","dependencies_parsed_at":"2024-01-27T10:16:11.762Z","dependency_job_id":"ac0b9a5f-7369-4d35-bf07-3ed1ee91d2e0","html_url":"https://github.com/Quentin18/gymnasium-2048","commit_stats":null,"previous_names":["quentin18/gymnasium-2048"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quentin18%2Fgymnasium-2048","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quentin18%2Fgymnasium-2048/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quentin18%2Fgymnasium-2048/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quentin18%2Fgymnasium-2048/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Quentin18","download_url":"https://codeload.github.com/Quentin18/gymnasium-2048/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239252914,"owners_count":19607936,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["2048","2048-ai","2048-game","gymnasium","n-tuple-networks","pygame","temporal-differencing-learning"],"created_at":"2025-02-17T07:31:55.335Z","updated_at":"2025-11-01T03:30:19.508Z","avatar_url":"https://github.com/Quentin18.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gymnasium 2048\n\n[![CI](https://github.com/Quentin18/gymnasium-2048/actions/workflows/build.yml/badge.svg)](https://github.com/Quentin18/gymnasium-2048/actions/workflows/build.yml)\n[![Python](https://img.shields.io/pypi/pyversions/gymnasium-2048.svg)](https://badge.fury.io/py/gymnasium-2048)\n[![PyPI](https://badge.fury.io/py/gymnasium-2048.svg)](https://badge.fury.io/py/gymnasium-2048)\n[![PyPI Downloads](https://static.pepy.tech/badge/gymnasium-2048)](https://pepy.tech/projects/gymnasium-2048)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white)](https://pre-commit.com/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat\u0026labelColor=ef8336)](https://pycqa.github.io/isort/)\n\nGymnasium environment for the [Game 2048](https://en.wikipedia.org/wiki/2048_(video_game)) and game-playing agents using\ntemporal difference learning of n-tuple networks.\n\nhttps://github.com/Quentin18/gymnasium-2048/assets/58831477/c630a605-d1da-412a-a284-75f5c28bab46\n\n\u003ctable\u003e\n    \u003ctbody\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eAction Space\u003c/td\u003e\n            \u003ctd\u003e\u003ccode\u003espaces.Discrete(4)\u003c/code\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eObservation Space\u003c/td\u003e\n            \u003ctd\u003e\u003ccode\u003espaces.Box(low=0, high=1, shape=(4, 4, 16), dtype=np.uint8)\u003c/code\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eImport\u003c/td\u003e\n            \u003ctd\u003e\u003ccode\u003egymnasium.make(\"gymnasium_2048:gymnasium_2048/TwentyFortyEight-v0\")\u003c/code\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/tbody\u003e\n\u003c/table\u003e\n\n## Installation\n\nTo install `gymnasium-2048` with pip, execute:\n\n```bash\npip install gymnasium_2048\n```\n\nFrom source:\n\n```bash\ngit clone https://github.com/Quentin18/gymnasium-2048\ncd gymnasium-2048/\npip install -e .\n```\n\n## Environment\n\n### Action Space\n\nThe action is an integer representing the direction to slide the tiles:\n\n| Direction | Action |\n|-----------|--------|\n| 0         | UP     |\n| 1         | RIGHT  |\n| 2         | DOWN   |\n| 3         | LEFT   |\n\n### Observation Space\n\nThe observation is a 3D `ndarray` encoding the board state. It is encoded into 16 channels, where each channel is a 4x4\nbinary image. The i-th channel marks each cell of the game position that contains the i-th tile as 1, and 0 otherwise.\nEach channel represents the positions of empty cells, 2-tiles, 4-tiles, ... , and 32768-tiles, respectively.\n\n![Observation](./figures/observation.png)\n\nThis representation is mostly used for deep convolutional neural networks (DCNN).\n\n### Rewards\n\nAt each step, for each tile merge, the player gains a reward\nequal to the value of the new tile.\nThe total reward, corresponding to the game score, is the\nsum of rewards obtained throughout the game.\n\n### Starting State\n\nThe game starts with two randomly generated tiles. A 2-tile can be generated with probability 0.9 and a 4-tile with\nprobability 0.1.\n\n### Episode End\n\nThe episode ends if there are no legal moves, i.e., all squares are occupied and there are no two adjacent tiles sharing\nthe same value.\n\n### Arguments\n\n- `size`: the size of the game board. The default value is 4.\n- `max_pow`: the maximum power of 2 allowed. The default value is 16.\n\n```python\nimport gymnasium as gym\n\ngym.make(\"gymnasium_2048:gymnasium_2048/TwentyFortyEight-v0\", size=4, max_pow=16)\n```\n\n## Usage\n\nTo use the training and evaluation scripts, install the `training` dependencies:\n\n```bash\npip install .[training]\n```\n\n### Play Manually\n\nTo play the game manually with the four arrows of your keyboard, execute:\n\n```bash\npython -m scripts.play\n```\n\nSee the arguments with the help command:\n\n```bash\npython -m scripts.play -h\n```\n\n### Train an Agent\n\nTo train an agent using temporal difference learning of n-tuple networks, execute:\n\n```bash\npython -m scripts.train \\\n  --algo tdl \\\n  -n 100000 \\\n  --eval-freq 5000 \\\n  --eval-episode 1000 \\\n  --save-freq 5000 \\\n  --seed 42 \\\n  -o models/tdl\n```\n\nSee the arguments with the help command:\n\n```bash\npython -m scripts.train -h\n```\n\n### Plot Training Metrics\n\nTo plot training metrics from logs, execute:\n\n```bash\npython -m scripts.plot \\\n  -i train.log \\\n  -t \"Temporal Difference Learning\" \\\n  -o figures/training_tdl.png\n```\n\nSee the arguments with the help command:\n\n```bash\npython -m scripts.plot -h\n```\n\nHere are the training metrics of trained policies over episodes:\n\n| TDL small                                      | TDL                                |\n|------------------------------------------------|------------------------------------|\n| ![TDL small](./figures/training_tdl_small.png) | ![TDL](./figures/training_tdl.png) |\n\n### Enjoy a Trained Agent\n\nTo see a trained agent in action, execute:\n\n```bash\npython -m scripts.enjoy \\\n  --algo tdl \\\n  -i models/tdl/best_n_tuple_network_policy.zip \\\n  -n 1 \\\n  --seed 42\n```\n\nSee the arguments with the help command:\n\n```bash\npython -m scripts.enjoy -h\n```\n\n### Evaluate a Trained Agent\n\nTo evaluate the performance of a trained agent, execute:\n\n```bash\npython -m scripts.evaluate \\\n  --algo tdl \\\n  -i models/tdl/best_n_tuple_network_policy.zip \\\n  -n 1000 \\\n  --seed 42 \\\n  -t \"Temporal Difference Learning\" \\\n  -o figures/stats_tdl.png\n```\n\nSee the arguments with the help command:\n\n```bash\npython -m scripts.evaluate -h\n```\n\nHere are the performances of trained policies:\n\n| TDL small                                   | TDL                             |\n|---------------------------------------------|---------------------------------|\n| ![TDL small](./figures/stats_tdl_small.png) | ![TDL](./figures/stats_tdl.png) |\n\n\u003cdetails\u003e\n\u003csummary\u003eRandom policy performances\u003c/summary\u003e\n\n![Random policy](./figures/stats_random_policy.png)\n\n\u003c/details\u003e\n\n## Tests\n\nTo run tests, execute:\n\n```bash\npytest\n```\n\n## Citing\n\nTo cite the repository in publications:\n\n```bibtex\n@misc{gymnasium-2048,\n  author = {Quentin Deschamps},\n  title = {Gymnasium 2048},\n  year = {2023},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https://github.com/Quentin18/gymnasium-2048}},\n}\n```\n\n## References\n\n- [Gymnasium](https://github.com/Farama-Foundation/Gymnasium)\n- [Szubert and Jaśkowski: Temporal Difference Learning of N-Tuple Networks\n  for the Game 2048](https://www.cs.put.poznan.pl/wjaskowski/pub/papers/Szubert2014_2048.pdf)\n- [Guei and Wu: On Reinforcement Learning for the Game of 2048](https://arxiv.org/pdf/2212.11087.pdf)\n\n## Author\n\n[Quentin Deschamps](mailto:quentindeschamps18@gmail.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquentin18%2Fgymnasium-2048","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquentin18%2Fgymnasium-2048","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquentin18%2Fgymnasium-2048/lists"}