{"id":27884310,"url":"https://github.com/skyworkai/skyreels-a2","last_synced_at":"2025-05-05T06:36:14.923Z","repository":{"id":285922465,"uuid":"953821511","full_name":"SkyworkAI/SkyReels-A2","owner":"SkyworkAI","description":"SkyReels-A2: Compose anything in video diffusion transformers","archived":false,"fork":false,"pushed_at":"2025-04-15T13:59:13.000Z","size":71713,"stargazers_count":301,"open_issues_count":17,"forks_count":24,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-04-15T14:44:59.503Z","etag":null,"topics":["controllable-generation","image2video","video-diffusion-transformers"],"latest_commit_sha":null,"homepage":"https://www.skyreels.ai/home/tools/to-video?refer=home","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SkyworkAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-24T06:09:34.000Z","updated_at":"2025-04-15T13:59:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"4563e08e-93f2-42b2-b685-5d122f4711bf","html_url":"https://github.com/SkyworkAI/SkyReels-A2","commit_stats":null,"previous_names":["skyworkai/skyreels-a2"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SkyworkAI%2FSkyReels-A2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SkyworkAI%2FSkyReels-A2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SkyworkAI%2FSkyReels-A2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SkyworkAI%2FSkyReels-A2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SkyworkAI","download_url":"https://codeload.github.com/SkyworkAI/SkyReels-A2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252454576,"owners_count":21750493,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["controllable-generation","image2video","video-diffusion-transformers"],"created_at":"2025-05-05T06:36:14.445Z","updated_at":"2025-05-05T06:36:14.908Z","avatar_url":"https://github.com/SkyworkAI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/logo.png\" alt=\"Skyreels Logo\" width=\"50%\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eSkyReels-A2: Compose Anything in Video Diffusion Transformers\u003c/h1\u003e \n\n\u003cdiv align='center'\u003e\n    \u003ca href='https://scholar.google.com/citations?user=_43YnBcAAAAJ\u0026hl=zh-CN' target='_blank'\u003eZhengcong Fei\u003csup\u003e * \u003c/sup\u003e\u003c/a\u003e\u0026emsp;\n    \u003ca href='https://debangli.github.io/' target='_blank'\u003eDebang Li\u003csup\u003e * \u0026dagger;\u003c/sup\u003e\u003c/a\u003e\u0026emsp;\n    \u003ca href='https://scholar.google.com/citations?user=6D_nzucAAAAJ\u0026hl=en' target='_blank'\u003eDi Qiu\u003csup\u003e * \u0026dagger;\u003c/sup\u003e\u003c/a\u003e\u0026emsp;\n    \u003ca href='' target='_blank'\u003eJiahua Wang\u003c/a\u003e\u0026emsp;\n    \u003ca href='' target='_blank'\u003eYikun Dou\u003c/a\u003e\u0026emsp;\n    \u003ca href='' target='_blank'\u003eRui Wang\u003c/a\u003e\u0026emsp;\n\u003c/div\u003e\n\n\u003cdiv align='center'\u003e\n    \u003ca href='https://scholar.google.com/citations?user=bL_o7I8AAAAJ\u0026hl=en' target='_blank'\u003eJingtao Xu\u003c/a\u003e\u0026emsp;\n    \u003ca href='' target='_blank'\u003eMingyuan Fan\u003c/a\u003e\u0026emsp;\n    \u003ca href='https://scholar.google.com.hk/citations?hl=zh-CN\u0026user=HukWSw4AAAAJ' target='_blank'\u003eGuibin Chen\u003c/a\u003e\u0026emsp;\n    \u003ca href='' target='_blank'\u003eYang Li\u003c/a\u003e\u0026emsp;\n    \u003ca href='' target='_blank'\u003eYahui Zhou\u003c/a\u003e\u0026emsp;\n\u003c/div\u003e\n\u003cdiv align='center'\u003e\n    \u003cbr\u003e\u003csup\u003e * \u003c/sup\u003eEqual contribution,\u003csup\u003e \u0026dagger; \u003c/sup\u003eProject lead\n    \u003cbr\u003e\u003cstrong\u003eSkywork AI, Kunlun Inc.\u003c/strong\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003c!-- \u003ca href='LICENSE'\u003e\u003cimg src='https://img.shields.io/badge/license-MIT-yellow'\u003e\u003c/a\u003e --\u003e\n  \u003ca href='https://arxiv.org/pdf/2504.02436'\u003e\u003cimg src='https://img.shields.io/badge/arXiv-SkyReels A2-red'\u003e\u003c/a\u003e\n  \u003ca href='https://skyworkai.github.io/skyreels-a2.github.io/'\u003e\u003cimg src='https://img.shields.io/badge/Project-SkyReels A2-green'\u003e\u003c/a\u003e\n  \u003ca href='https://huggingface.co/Skywork/SkyReels-A2'\u003e\u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue'\u003e\u003c/a\u003e\n  \u003ca href='https://huggingface.co/datasets/Skywork/A2-Bench'\u003e\u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-A2Bench-blue'\u003e\u003c/a\u003e\n  \u003ca href='https://huggingface.co/spaces/Skywork/SkyReels_A2_Bench'\u003e\u003cimg src='https://img.shields.io/badge/Online-Leaderboard-purple'\u003e\u003c/a\u003e\n  \u003ca href='https://www.skyreels.ai'\u003e\u003cimg src='https://img.shields.io/badge/Demo-Spaces-yellow'\u003e\u003c/a\u003e\n  \u003cbr\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/demo.gif\" alt=\"showcase\" width=\"800\"\u003e\n  \u003cbr\u003e\n  🔥 For more results, visit our \u003ca href=\"https://skyworkai.github.io/skyreels-a2.github.io/\"\u003e\u003cstrong\u003ehomepage\u003c/strong\u003e\u003c/a\u003e 🔥\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    👋 Join our \u003ca href=\"https://discord.gg/PwM6NYtccQ\" target=\"_blank\"\u003e\u003cstrong\u003eDiscord\u003c/strong\u003e\u003c/a\u003e \n\u003c/p\u003e\n\nThis repo, named **SkyReels-A2**, contains the official PyTorch implementation of our paper [SkyReels-A2: Compose Anything in Video Diffusion Transformers](https://arxiv.org/pdf/2504.02436).\n\n## 🎉 News\n\n* Apr 21, 2025: 👋 We release the Infinite-Length Film Generative Model, [SkyReels-V2](https://huggingface.co/collections/Skywork/skyreels-v2-6801b1b93df627d441d0d0d9) Series Models and the video captioning model [SkyCaptioner-V1](https://huggingface.co/Skywork/SkyCaptioner-V1) .\n* Apr 8, 2025: 🎉  We open the A2-Bench online evaluation and leaderboard. Test it on [A2-Bench Leaderboard](https://huggingface.co/spaces/Skywork/SkyReels_A2_Bench) !\n* Apr 7, 2025: 🔥 [ComfyUI](https://github.com/Yuan-ManX/ComfyUI-SkyReels-A2) is now available.\n* Apr 3, 2025: 🔥 We release pre-view version of checkpoints, code of model inference and gradio demo.\n* Previously, we released [SkyReels-A1](https://github.com/SkyworkAI/SkyReels-A1). This is an open-sourced and effective framework for portrait image animation.\n\n## 📑 TODO List\n- [x] Support A2-Bench evaluation and leaderboard.\n- [x] ComfyUI\n- [x] Parallel Inference on Multi-GPUs\n- [x] [TeaCache](https://github.com/ali-vilab/TeaCache) for faster inference\n- [x] User-Level GPU Inference on RTX4090\n- [ ] Release all model sequence, including infinity-long version.\n- [ ] Diffusers\n\n## 🪄 Models\n| Models                   | Download Link                                                                                                                                                                           | Video Size         |\n|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------|\n| A2-Wan2.1-14B-Preview | [Huggingface](https://huggingface.co/Skywork/SkyReels-A2) 🤗                                                                                                                                                              | ~ 81 x 480 x 832    | \n| A2-Wan2.1-14B         | [To be released](https://github.com/SkyworkAI/SkyReels-A2)  | ~ 81 x 480 x 832    | \n| A2-Wan2.1-14B-Pro         | [To be released](https://github.com/SkyworkAI/SkyReels-A2)  | ~ 97 x 544 x 960    | \n| A2-Wan2.1-14B-Infinity         | [To be released](https://github.com/SkyworkAI/SkyReels-A2)  | ~ Inf x 720 x 1080   | \n\n\n\n\n## 1. Getting Started 🏁 \n\n### 1.1 Clone the code and prepare the environment 🛠️\n\nFirst git clone the repository with code: \n```bash\ngit clone https://github.com/SkyworkAI/SkyReels-A2.git\ncd SkyReels-A2\n\n# create env using conda\nconda create -n skyreels-a2 python=3.10\nconda activate skyreels-a2\n```\nThen, install the remaining dependencies:\n```bash\npip install -r requirements.txt\n```\n\n### 1.2 Download pretrained weights 📥\n\nYou can download the pretrained weights from HuggingFace as:\n```bash\n# !pip install -U \"huggingface_hub[cli]\"\nhuggingface-cli download Skywork/SkyReels-A2 --local-dir local_path --exclude \"*.git*\" \"README.md\" \"docs\"\n```\nor download from webpage mannually. \n\n\n### 1.3 Inference 🚀\n\nYou can first set the model path and reference images path and then simply run the inference scripts as: \n```bash\npython infer.py\n```\n\nIf the script runs successfully, you will get an output mp4 file. This file includes the following results: driving video, input image or video, and generated result.\n\n\nWe also support multi-GPU inference scripts for faster inference, as:\n```bash\ntorchrun --nproc_per_node=$GPU_NUM infer_MGPU.py\n```\nSet the offload_switch of infer_MGPU.py to True, and you can run it on RTX4090\n\n\n#### Gradio Interface 🤗\n\nWe also provide a [Gradio](https://huggingface.co/docs/hub/spaces-sdks-gradio) interface for a better user experience, just run by:\n\n```bash\npython app.py\n```\n\nThe graphical interactive interface is shown as below.  \n\n\n\n## 2. A2-Bench Evaluation 👓\n\nWe public the evaluation data in [Huggingface](https://huggingface.co/datasets/Skywork/A2-Bench), you can infer with results and then submit to [leaderboard](https://huggingface.co/spaces/Skywork/SkyReels_A2_Bench) to obtain the results automatically. \nMore detail about metric computation code will coming soon.\n\n\n\n## Acknowledgements 💐\n\nWe would like to thank the contributors of [Wan](https://github.com/Wan-Video/Wan2.1) and [finetrainers](https://github.com/a-r-r-o-w/finetrainers) repositories, for their open research and contributions. \n\n## Citation 💖\nIf you find SkyReels-A2 useful for your research, welcome to 🌟 this repo and cite our work using the following BibTeX:\n```bibtex\n@article{fei2025skyreels,\n  title={SkyReels-A2: Compose Anything in Video Diffusion Transformers},\n  author={Fei, Zhengcong and Li, Debang and Qiu, Di and Wang, Jiahua and Dou, Yikun and Wang, Rui and Xu, Jingtao and Fan, Mingyuan and Chen, Guibin and Li, Yang and others},\n  journal={arXiv preprint arXiv:2504.02436},\n  year={2025}\n}\n```\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=SkyworkAI/SkyReels-A2\u0026type=Date)](https://www.star-history.com/#SkyworkAI/SkyReels-A2\u0026Date)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskyworkai%2Fskyreels-a2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskyworkai%2Fskyreels-a2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskyworkai%2Fskyreels-a2/lists"}