{"id":28531462,"url":"https://github.com/sverrenystad/muzero","last_synced_at":"2026-02-26T18:37:46.285Z","repository":{"id":297718199,"uuid":"930243637","full_name":"SverreNystad/MuZero","owner":"SverreNystad","description":"An implementation of the MuZero algorithm by Google Deepmind.","archived":false,"fork":false,"pushed_at":"2025-08-27T09:50:23.000Z","size":39704,"stargazers_count":3,"open_issues_count":7,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-27T14:43:59.603Z","etag":null,"topics":["ai","deep-reinforcement-learning","distributed-computing","muzero","openai-gym-agents","parameter-server","rey"],"latest_commit_sha":null,"homepage":"https://arxiv.org/pdf/1911.08265","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SverreNystad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-10T10:12:51.000Z","updated_at":"2025-08-27T09:50:26.000Z","dependencies_parsed_at":"2025-06-07T02:48:51.074Z","dependency_job_id":"1346b463-0ee9-4776-bea7-1c7699161fb7","html_url":"https://github.com/SverreNystad/MuZero","commit_stats":null,"previous_names":["sverrenystad/muzero"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SverreNystad/MuZero","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SverreNystad%2FMuZero","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SverreNystad%2FMuZero/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SverreNystad%2FMuZero/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SverreNystad%2FMuZero/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SverreNystad","download_url":"https://codeload.github.com/SverreNystad/MuZero/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SverreNystad%2FMuZero/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29867564,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-26T18:27:06.972Z","status":"ssl_error","status_checked_at":"2026-02-26T18:26:57.848Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deep-reinforcement-learning","distributed-computing","muzero","openai-gym-agents","parameter-server","rey"],"created_at":"2025-06-09T15:10:37.720Z","updated_at":"2026-02-26T18:37:46.268Z","avatar_url":"https://github.com/SverreNystad.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MuZero\n\nThis project is a Python implementation of the [**MuZero**](https://arxiv.org/pdf/1911.08265) algorithm created by Google Deepmind, designed to be used with the OpenAI Gym environment. It is built to be modular and extensible, allowing for easy integration with different Gymnasium environments and neural network configurations.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003e📋 Table of contents \u003c/b\u003e\u003c/summary\u003e\n\n- [MuZero](#muzero)\n  - [Overview](#overview)\n  - [Prerequisites](#prerequisites)\n  - [Usage](#usage)\n  - [Testing](#testing)\n  - [📖 Documentations](#-documentations)\n  - [Contributors](#contributors)\n\n\u003c/details\u003e\n\nCheck out our presentation video below for an in-depth overview:\n\n[![Watch MuZero Presentation](https://img.youtube.com/vi/2GOIBhZgXrA/maxresdefault.jpg)](https://youtu.be/2GOIBhZgXrA)\n\n\nOur MuZero in action, demonstrating performance on the Car Racing game in OpenAI Gym:\n\n[![Watch Results](https://img.youtube.com/vi/pYpWq2wOTsU/0.jpg)](https://youtu.be/pYpWq2wOTsU)\n\n## Overview\nMuZero is an advanced **model-based** reinforcement learning algorithm that jointly learns **a dynamics model**, **a value function**, and **a policy** through self-play training. Unlike traditional systems like AlphaZero, MuZero does not require explicit knowledge of the true environment dynamics. Instead the model gets the last 32 frames as input and it learns its own abstract internal representation of the environment. This allows for using Monte Carlo Tree Search (MCTS) to plan ahead in latent space making it capable of sophisticated planning, resulting in superhuman performance across various complex environments.\n\nAs the model does not require the true environment dynamics, it can be applied to a much wider range of problems. This being very useful in real-world applications where the environment is almost never fully known or is too complex to model explicitly. However the lack of explicit environment model makes it much more difficult to train, as the model has to learn the dynamics of the environment from scratch and it will start of by scrambling the signals it receives from the environment.\n\nWe needed to improve the sampling efficiency of the model training on the data the model thinks is most surprising and what episodes that it did the best on. This is done by using a [**Prioritized Experience Replay**](https://arxiv.org/pdf/1511.05952) which allows the model to focus on the most important experiences and learn from them more effectively. The model is trained on a large number of episodes, and it learns to prioritize the most important experiences based on their impact on the learning process.\n\n![Distributed Prioritized Experience Replay](docs/diagrams/DPER.png)\n\nBut as the model needs lots of training data we scaled the training process by using a [**Distributed Prioritized Experience Replay**](https://arxiv.org/pdf/1803.00933). Where we had several instances of the program continuously generating training data in parallel. Each instance of the program runs its own environment and collects experiences, which are then stored in a shared experience replay buffer. While we had a parameter server that manages the model parameters and synchronizes them across all instances. With a trainer that samples from the shared experience replay buffer and updates the model parameters based on the sampled experiences.\n\n\n\n## Prerequisites\n- Ensure that git is installed on your machine. [Download Git](https://git-scm.com/downloads)\n- Docker is used for the backend and database setup. [Download Docker](https://www.docker.com/products/docker-desktop)\n\n\n## Usage\nTo start, run the following command in the root directory of the project:\n```bash\npython main.py\n```\n\n\n## Testing\n\nTo run the tests, run the following command in the root directory of the project:\n```bash\npytest\n```\n\nTo run the tests with coverage, run the following command in the root directory of the project:\n```bash\ncoverage run -m pytest\n```\n\nTo see the coverage report, run the following command in the root directory of the project:\n```bash\ncoverage html -i\n```\n\n## 📖 Documentations\n\n- [Developer Setup Guild](docs/manuals/developer_setup.md)\n\n\n\n## Contributors\n\n\n\u003ctable align=\"center\"\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n        \u003ca href=\"https://github.com/Knolaisen\"\u003e\n            \u003cimg src=\"https://github.com/Knolaisen.png?size=600\" width=\"100px;\" alt=\"Kristoffer Nohr Olaisen\"/\u003e\u003cbr /\u003e\n            \u003csub\u003e\u003cb\u003eKristoffer Nohr Olaisen\u003c/b\u003e\u003c/sub\u003e\n        \u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\n        \u003ca href=\"https://github.com/olavsl\"\u003e\n            \u003cimg src=\"https://github.com/olavsl.png?size=600\" width=\"100px;\" alt=\"Olav Selnes Lorentzen\"/\u003e\u003cbr /\u003e\n            \u003csub\u003e\u003cb\u003eOlav Selnes Lorentzen\u003c/b\u003e\u003c/sub\u003e\n        \u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003ca href=\"https://github.com/sandviklee\"\u003e\n          \u003cimg src=\"https://github.com/sandviklee.png?size=600\" width=\"100px;\"/\u003e\u003cbr /\u003e\n          \u003csub\u003e\u003cb\u003eSimon Sandvik Lee\u003c/b\u003e\u003c/sub\u003e\n      \u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\n        \u003ca href=\"https://github.com/SverreNystad\"\u003e\n            \u003cimg src=\"https://github.com/SverreNystad.png?size=600\" width=\"100px;\"/\u003e\u003cbr /\u003e\n            \u003csub\u003e\u003cb\u003eSverre Nystad\u003c/b\u003e\u003c/sub\u003e\n        \u003c/a\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsverrenystad%2Fmuzero","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsverrenystad%2Fmuzero","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsverrenystad%2Fmuzero/lists"}