{"id":13958527,"url":"https://github.com/YeWR/EfficientZero","last_synced_at":"2025-07-21T00:31:08.931Z","repository":{"id":37710057,"uuid":"419600574","full_name":"YeWR/EfficientZero","owner":"YeWR","description":"Open-source codebase for EfficientZero, from \"Mastering Atari Games with Limited Data\" at NeurIPS 2021.","archived":false,"fork":false,"pushed_at":"2023-12-20T07:31:16.000Z","size":2209,"stargazers_count":871,"open_issues_count":32,"forks_count":137,"subscribers_count":47,"default_branch":"main","last_synced_at":"2024-11-28T02:34:47.746Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/YeWR.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-21T06:03:14.000Z","updated_at":"2024-11-27T02:25:15.000Z","dependencies_parsed_at":"2024-11-28T02:32:24.033Z","dependency_job_id":"390ce6d2-1bce-4e1a-8f1e-bff988435535","html_url":"https://github.com/YeWR/EfficientZero","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/YeWR/EfficientZero","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YeWR%2FEfficientZero","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YeWR%2FEfficientZero/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YeWR%2FEfficientZero/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YeWR%2FEfficientZero/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/YeWR","download_url":"https://codeload.github.com/YeWR/EfficientZero/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YeWR%2FEfficientZero/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266221257,"owners_count":23894965,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-08T13:01:41.977Z","updated_at":"2025-07-21T00:31:08.519Z","avatar_url":"https://github.com/YeWR.png","language":"Python","readme":"# EfficientZero (NeurIPS 2021)\nOpen-source codebase for EfficientZero, from [\"Mastering Atari Games with Limited Data\"](https://arxiv.org/abs/2111.00210) at NeurIPS 2021.\n\n## Environments\nEfficientZero requires python3 (\u003e=3.6) and pytorch (\u003e=1.8.0) with the development headers. \n\nWe recommend to use torch amp (`--amp_type torch_amp`) to accelerate training.\n\n### Prerequisites\nBefore starting training, you need to build the c++/cython style external packages. (GCC version 7.5+ is required.)\n```\ncd core/ctree\nbash make.sh\n``` \nThe distributed framework of this codebase is built on [ray](https://docs.ray.io/en/releases-1.0.0/auto_examples/overview.html).\n\n### Installation\nAs for other packages required for this codebase, please run `pip install -r requirements.txt`.\n\n## Usage\n### Quick start\n* Train: `python main.py --env BreakoutNoFrameskip-v4 --case atari --opr train --amp_type torch_amp --num_gpus 1 --num_cpus 10 --cpu_actor 1 --gpu_actor 1 --force`\n* Test: `python main.py --env BreakoutNoFrameskip-v4 --case atari --opr test --amp_type torch_amp --num_gpus 1 --load_model --model_path model.p \\`\n### Bash file\nWe provide `train.sh` and `test.sh` for training and evaluation.\n* Train: \n  * With 4 GPUs (3090): `bash train.sh`\n* Test: `bash test.sh`\n\n|Required Arguments | Description|\n|:-------------|:-------------|\n| `--env`                             |Name of the environment|\n| `--case {atari}`                    |It's used for switching between different domains(default: atari)|\n| `--opr {train,test}`                |select the operation to be performed|\n| `--amp_type {torch_amp,none}`       |use torch amp for acceleration|\n\n|Other Arguments | Description|\n|:-------------|:-------------|\n| `--force`                           |will rewrite the result directory\n| `--num_gpus 4`                      |how many GPUs are available\n| `--num_cpus 96`                     |how many CPUs are available\n| `--cpu_actor 14`                    |how many cpu workers\n| `--gpu_actor 20`                    |how many gpu workers\n| `--seed 0`                          |the seed\n| `--use_priority`                    |use priority in replay buffer sampling\n| `--use_max_priority`                |use the max priority for the newly collectted data\n| `--amp_type 'torch_amp'`            |use torch amp for acceleration\n| `--info 'EZ-V0'`                    |some tags for you experiments\n| `--p_mcts_num 8`                    |set the parallel number of envs in self-play \n| `--revisit_policy_search_rate 0.99` |set the rate of reanalyzing policies\n| `--use_root_value`                  |use root values in value targets (require more GPU actors)\n| `--render`                          |render in evaluation\n| `--save_video`                      |save videos for evaluation\n \n## Architecture Designs\nThe architecture of the training pipeline is shown as follows:\n![](static/imgs/archi.png)\n\n### Some suggestions\n* To use a smaller model, you can choose smaller dim of the projection layers (Eg: 256/64) and the LSTM hidden layer (Eg: 64) in the config. \n* For GPUs with 10G memory instead of 20G memory, you can allocate 0.25 gpu for each GPU maker (`@ray.remote(num_gpus=0.25)`) in `core/reanalyze_worker.py`.\n\n### New environment registration\nIf you wan to apply EfficientZero to a new environment like `mujoco`. Here are the steps for registration:\n1. Follow the directory `config/atari` and create dir for the env at `config/mujoco`.\n2. Implement your `MujocoConfig(BaseConfig)` class and implement the models as well as your environment wrapper.\n3. Register the case at `main.py`.\n\n## Results \nEvaluation with 32 seeds for 3 different runs (different seeds).\n![](static/imgs/total_results.png)\n\n## Citation\nIf you find this repo useful, please cite our paper:\n```\n@inproceedings{ye2021mastering,\n  title={Mastering Atari Games with Limited Data},\n  author={Weirui Ye, and Shaohuai Liu, and Thanard Kurutach, and Pieter Abbeel, and Yang Gao},\n  booktitle={NeurIPS},\n  year={2021}\n}\n```\n\n## Contact\nIf you have any question or want to use the code, please contact ywr20@mails.tsinghua.edu.cn .\n\n## Acknowledgement\nWe appreciate the following github repos a lot for their valuable code base implementations:\n\nhttps://github.com/koulanurag/muzero-pytorch\n\nhttps://github.com/werner-duvaud/muzero-general\n\nhttps://github.com/pytorch/ELF\n","funding_links":[],"categories":["时间序列"],"sub_categories":["网络服务_其他"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYeWR%2FEfficientZero","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FYeWR%2FEfficientZero","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYeWR%2FEfficientZero/lists"}