{"id":36928496,"url":"https://github.com/Kevin-thu/Epona","last_synced_at":"2026-01-28T17:01:47.771Z","repository":{"id":301400344,"uuid":"1009038213","full_name":"Kevin-thu/Epona","owner":"Kevin-thu","description":"Official Code for Epona: Autoregressive Diffusion World Model for Autonomous Driving (ICCV 2025)","archived":false,"fork":false,"pushed_at":"2025-07-15T03:42:53.000Z","size":123559,"stargazers_count":74,"open_issues_count":3,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-07-15T08:46:05.902Z","etag":null,"topics":["autonomous-driving","diffusion-models","video-generation","world-models"],"latest_commit_sha":null,"homepage":"https://kevin-thu.github.io/Epona/","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kevin-thu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-26T13:31:27.000Z","updated_at":"2025-07-15T08:25:26.000Z","dependencies_parsed_at":"2025-07-15T05:24:22.034Z","dependency_job_id":null,"html_url":"https://github.com/Kevin-thu/Epona","commit_stats":null,"previous_names":["kevin-thu/epona"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Kevin-thu/Epona","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kevin-thu%2FEpona","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kevin-thu%2FEpona/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kevin-thu%2FEpona/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kevin-thu%2FEpona/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kevin-thu","download_url":"https://codeload.github.com/Kevin-thu/Epona/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kevin-thu%2FEpona/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28847053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T15:15:36.453Z","status":"ssl_error","status_checked_at":"2026-01-28T15:15:13.020Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autonomous-driving","diffusion-models","video-generation","world-models"],"created_at":"2026-01-13T01:00:35.844Z","updated_at":"2026-01-28T17:01:47.762Z","avatar_url":"https://github.com/Kevin-thu.png","language":null,"funding_links":[],"categories":["Papers","Decision-Coupled / Sequential / Token Feature Sequence"],"sub_categories":["2025"],"readme":"\u003cp align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003e\u003ci\u003eEpona\u003c/i\u003e: Autoregressive Diffusion World Model for Autonomous Driving\u003c/h1\u003e\n  \u003ch3 align=\"center\"\u003eICCV 2025\u003c/h3\u003e\n  \u003cp align=\"center\"\u003e\n                \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://kevin-thu.github.io/homepage/\" target=\"_blank\"\u003eKaiwen Zhang\u003c/a\u003e\u003csup\u003e*\u003c/sup\u003e,\n              \u003c/span\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://github.com/Tzy010822/\" target=\"_blank\"\u003eZhenyu Tang\u003c/a\u003e\u003csup\u003e*\u003c/sup\u003e,\n              \u003c/span\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://huxiaotaostasy.github.io/\" target=\"_blank\"\u003eXiaotao Hu\u003c/a\u003e,\n              \u003c/span\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://xingangpan.github.io/\" target=\"_blank\"\u003eXingang Pan\u003c/a\u003e,\n              \u003c/span\u003e\u003cbr\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://xy-guo.github.io/\" target=\"_blank\"\u003eXiaoyang Guo\u003c/a\u003e,\n              \u003c/span\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://liuyuan-pal.github.io/\" target=\"_blank\"\u003eYuan Liu\u003c/a\u003e,\n              \u003c/span\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://scholar.google.com/citations?user=7eJBk1UAAAAJ\u0026hl=zh-CN\" target=\"_blank\"\u003eJingwei Huang\u003c/a\u003e,\n              \u003c/span\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://yuanli2333.github.io/\" target=\"_blank\"\u003eYuan Li\u003c/a\u003e,\n              \u003c/span\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://scholar.google.com/citations?user=pCY-bikAAAAJ\u0026hl=en\u0026oi=ao\" target=\"_blank\"\u003eQian Zhang\u003c/a\u003e,\n              \u003c/span\u003e\u003cbr\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://www.xxlong.site/\" target=\"_blank\"\u003eXiaoxiao Long\u003c/a\u003e\u003csup\u003e✝\u003c/sup\u003e,\n              \u003c/span\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://cite.nju.edu.cn/People/Faculty/20190621/i5054.html\" target=\"_blank\"\u003eXun Cao\u003c/a\u003e,\n              \u003c/span\u003e\n              \u003cspan class=\"author-block\"\u003e\n                \u003ca href=\"https://yvanyin.xyz/\" target=\"_blank\"\u003eWei Yin\u003c/a\u003e\u003csup\u003e§\u003c/sup\u003e\n  \u003c/p\u003e\n\n  \u003cp align=\"center\"\u003e\n    \u003csep\u003e*\u003c/sep\u003eEqual Contribution\n    \u003csep\u003e✝\u003c/sep\u003eProject Adviser\n    \u003csep\u003e§\u003c/sep\u003eProject Lead, Corresponding Author\n  \u003c/p\u003e\n\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://arxiv.org/pdf/2506.24113\"\u003e\u003cimg alt='arXiv' src=\"https://img.shields.io/badge/arXiv-2506.24113-b31b1b.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://kevin-thu.github.io/Epona/\"\u003e\u003cimg alt='page' src=\"https://img.shields.io/badge/Project-Website-orange\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://huggingface.co/Kevin-thu/Epona\"\u003e\u003cimg alt=\"Huggingface\" src=\"https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Epona-orange\"\u003e\u003c/a\u003e\n    \u003c!-- \u003ca href=\"https://twitter.com/sze68zkw\"\u003e\u003cimg alt='Twitter' src=\"https://img.shields.io/twitter/follow/sze68zkw?label=%40KaiwenZhang\"\u003e\u003c/a\u003e --\u003e\n  \u003c/p\u003e\n\n  \u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./assets/teaser.png\", width=\"800\"\u003e\n    \u003cp align=\"left\"\u003e\u003cb\u003eVersatile capabilities of \u003ci\u003eEpona\u003c/i\u003e\u003c/b\u003e: Given historical driving context, our Epona can generate consistent \u003cb\u003eminutes-long driving videos\u003c/b\u003e at high resolution (A). It can be \u003cb\u003econtrolled by diverse trajectories\u003c/b\u003e (B), and understand real-world traffic knowledge (C). In addition, our world model can \u003cb\u003epredict future trajectories\u003c/b\u003e and serve as an end-to-end real-time motion planner (D).\n\u003c/p\u003e\n  \u003c/div\u003e\n\u003c/p\u003e\n\n\n## 🚀 Getting Started\n### Installation\n```bash\nconda create -n epona python=3.10\nconda activate epona\npip install -r requirements.txt\n```\nTo run the code with CUDA properly, you can comment out `torch` and `torchvision` in `requirement.txt`, and install the appropriate version of `torch\u003e=2.1.0+cu121` and `torchvision\u003e=0.16.0+cu121` according to the instructions on [PyTorch](https://pytorch.org/get-started/locally/).\n\n\n### Data Preparation\nPlease refer to [data preparation](./data_preparation/README.md) for more details to prepare and preprocess data.\n\nAfter preprocessing, please change the `datasets_paths` in the config files (under `configs` folder) to your own data path.\n\n### Inference\nYou can first download our pre-trained models (including the world models and the finetuned temporal-aware DCAE) from [Huggingface](https://huggingface.co/Kevin-thu/Epona).\n\nIn addition to our finetuned temporal-aware DCAE, you may also experiment with the original [DCAEs](https://github.com/mit-han-lab/efficientvit/blob/master/applications/dc_ae/README.md) provided by MIT Han Lab as the autoencoder: [dc-ae-f32c32-mix-1.0](https://huggingface.co/mit-han-lab/dc-ae-f32c32-mix-1.0) and [dc-ae-f32c32-sana-1.1](https://huggingface.co/mit-han-lab/dc-ae-f32c32-sana-1.1). After downloading, please change the `vae_ckpt` in the config files to your own autoencoder checkpoint path.\n\nThen, you can run different scripts in `scripts/test` folder to test *Epona* for different uses:\n| Script Name        | Dataset      | Trajectory Type                 | Video Length    | Use Case Description                                         |\n| ------------------ | ------------ | ------------------------------- | --------------- | ------------------------------------------------------------ |\n| `test_nuplan.py`   | NuPlan       | Fixed (from dataset)            | Fixed           | Evaluation on NuPlan test set with fixed setup.     |\n| `test_free.py`     | NuPlan       | Self-predicted                  | Variable (free) | **Long-term video generation** with autonomous predictions.      |\n| `test_ctrl.py`     | NuPlan       | User-provided (`poses`, `yaws`) | Variable (free) | **Trajectory-controlled video generation**; requires manual inputs in the script.    |\n| `test_traj.py`     | NuPlan       | Prediction only                 | N/A             | Evaluates the model’s **trajectory prediction** accuracy.        |\n| `test_nuscenes.py` | NuScenes     | Fixed (from dataset)            | Fixed           | Evaluation on nuScenes validation set with fixed setup.            |\n| `test_demo.py`     | Custom input | Self-predicted                  | Variable (free) | Run *Epona* on your own input data. |\n\n\n\u003c!-- 1. **`test_nuplan.py`:** test the model on NuPlan test set with *fixed trajectories and fixed video length* in the dataset;\n1. **`test_free.py`:** test the model on NuPlan test set with *self-predicted trajectories and free video length* (for *long-term video generation*);\n2. **`test_demo.py`:** test the model on *your own input data* with self-predicted trajectories and free video length;\n3. **`test_ctrl.py`:** test the model on NuPlan test set with *your input trajectories* (for *trajectory-controlled video generation*, need to set the `poses` and `yaws` according to the guidance in the script);\n4. **`test_traj.py`:** test the model's trajectory prediction performance on NuPlan test set;\n5. **`test_nuscenes.py`:** test the model on nuScenes test set with *fixed trajectories and fixed video length* in the dataset. --\u003e\n\nFor example, to test the model on NuPlan test set, you can run:\n```bash\npython3 scripts/test/test_nuplan.py \\\n  --exp_name \"test-nuplan\" \\\n  --start_id 0 --end_id 100 \\\n  --resume_path \"pretrained/epona_nuplan.pkl\" \\\n  --config configs/dit_config_dcae_nuplan.py\n```\nwhere:\n- `exp_name` is the name of the experiment;\n- `start_id` and `end_id` are the range of the test samples;\n- `resume_path` is the path to the pre-trained world model;\n- `config` is the path to the config file.\n\nAll the inference scripts can be run on a single NVIDIA 4090 GPU.\n\n### Training / Finetuning\nWe also provide a simple script `scripts/train_deepspeed.py` for you to train or finetune the world model with DeepSpeed.\nFor example, to train the world model on NuPlan dataset, you can run:\n```bash\nexport NODES_NUM=4\nexport GPUS_NUM=8\ntorchrun --nnodes=$NODES_NUM --nproc_per_node=$GPUS_NUM \\\nscripts/train_deepspeed.py \\\n  --batch_size 2 \\\n  --lr 2e-5 \\\n  --exp_name \"train-nuplan\" \\\n  --config configs/dit_config_dcae_nuplan.py \\\n  --resume_path \"pretrained/epona_nuplan.pkl\" \\ # set `resume_path` to resume training on previous checkpoint\n  --eval_steps 2000\n```\nYou can customize the configuration file in the `configs` folder (e.g., adjust image resolution, number of condition frames, model size, etc.).\nAdditionally, you can finetune our base world model on your own dataset by modifying the `dataset` folder to implement a custom dataset class.\n\n## ❤️ Ackowledgement\nOur implementation is based on [DrivingWorld](https://github.com/YvanYin/DrivingWorld), [Flux](https://github.com/black-forest-labs/flux) and [DCAE](https://github.com/mit-han-lab/efficientvit/tree/master/applications/dc_ae). Thanks for these great open-source works!\n\n## 📌 Citation\nIf any part of our paper or code is helpful to your research, please consider citing our work 📝 and give us a star ⭐. Thanks for your support!\n```bibtex\n@inproceedings{zhang2025epona,\n  author = {Zhang, Kaiwen and Tang, Zhenyu and Hu, Xiaotao and Pan, Xingang and Guo, Xiaoyang and Liu, Yuan and Huang,\n  Jingwei and Yuan, Li and Zhang, Qian and Long, Xiao-Xiao and Cao, Xun and Yin, Wei},\n  title = {Epona: Autoregressive Diffusion World Model for Autonomous Driving},\n  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},\n  year = {2025}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKevin-thu%2FEpona","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FKevin-thu%2FEpona","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKevin-thu%2FEpona/lists"}