{"id":20953891,"url":"https://github.com/continue-revolution/sd-forge-animatediff","last_synced_at":"2025-05-14T04:32:18.258Z","repository":{"id":221073902,"uuid":"753343783","full_name":"continue-revolution/sd-forge-animatediff","owner":"continue-revolution","description":"AnimateDiff for Stable Diffusion WebUI Forge, mirror for https://github.com/continue-revolution/sd-webui-animatediff/tree/forge/master","archived":false,"fork":false,"pushed_at":"2024-03-21T08:40:48.000Z","size":299,"stargazers_count":89,"open_issues_count":6,"forks_count":5,"subscribers_count":8,"default_branch":"forge-master","last_synced_at":"2024-06-19T03:11:42.064Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/continue-revolution.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":"conrevo","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":["https://paypal.me/conrevo","https://afdian.net/a/conrevo"]}},"created_at":"2024-02-05T23:40:24.000Z","updated_at":"2024-06-16T08:22:25.000Z","dependencies_parsed_at":"2024-02-06T02:43:35.943Z","dependency_job_id":"01004bd2-ae2d-4246-b9b4-9282bc936e8f","html_url":"https://github.com/continue-revolution/sd-forge-animatediff","commit_stats":null,"previous_names":["continue-revolution/sd-forge-animatediff"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/continue-revolution%2Fsd-forge-animatediff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/continue-revolution%2Fsd-forge-animatediff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/continue-revolution%2Fsd-forge-animatediff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/continue-revolution%2Fsd-forge-animatediff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/continue-revolution","download_url":"https://codeload.github.com/continue-revolution/sd-forge-animatediff/tar.gz/refs/heads/forge-master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225275669,"owners_count":17448389,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T01:04:14.726Z","updated_at":"2024-11-19T01:04:15.274Z","avatar_url":"https://github.com/continue-revolution.png","language":"Python","funding_links":["https://ko-fi.com/conrevo","https://paypal.me/conrevo","https://afdian.net/a/conrevo"],"categories":[],"sub_categories":[],"readme":"# AnimateDiff for Stable Diffusion WebUI Forge\nThis branch is specifically designed for [Stable Diffusion WebUI Forge](https://github.com/lllyasviel/stable-diffusion-webui-forge) by lllyasviel. See [here](docs/how-to-use.md#preparation) for how to install forge and this extension. See [Update](#update) for current status.\n\nThis extension aim for integrating [AnimateDiff](https://github.com/guoyww/AnimateDiff/) with [CLI](https://github.com/s9roll7/animatediff-cli-prompt-travel) into [lllyasviel's Forge Adaption of AUTOMATIC1111 Stable Diffusion WebUI](https://github.com/lllyasviel/stable-diffusion-webui-forge) and form the most easy-to-use AI video toolkit. You can generate GIFs in exactly the same way as generating images after enabling this extension.\n\nThis extension implements AnimateDiff in a different way. It makes heavy use of [Unet Patcher](https://github.com/lllyasviel/stable-diffusion-webui-forge?tab=readme-ov-file#unet-patcher), so that you do not need to reload your model weights if you don't want to, and I can almostly get rid of monkey-patching WebUI and ControlNet.\n\nYou might also be interested in another extension I created: [Segment Anything for Stable Diffusion WebUI](https://github.com/continue-revolution/sd-webui-segment-anything). This extension will also be redesigned for forge later..\n\n[TusiArt](https://tusiart.com/) (for users inside P.R.China mainland) and [TensorArt](https://tensor.art/) (for others) offers online service of this extension.\n\n\n## Table of Contents\n[Update](#update) | [Future Plan](#future-plan) | [Model Zoo](#model-zoo) | [Documentation](#documentation) | [Tutorial](#tutorial) | [Thanks](#thanks) | [Star History](#star-history) | [Sponsor](#sponsor)\n\n\n## Update\n- [v2.0.0-f](https://github.com/continue-revolution/sd-webui-animatediff/tree/v2.0.0-f) in `02/05/2023`: txt2img, prompt travel, infinite generation, all kinds of optimizations have been proven to be working properly and elegantly.\n- [v2.0.1-f](https://github.com/continue-revolution/sd-webui-animatediff/tree/v2.0.1-f) in `02/11/2023`: [ControlNet V2V](docs/features.md#controlnet-v2v) in txt2img panel is working properly and elegantly. You can also try adding mask and inpaint.\n- [v2.0.2-f](https://github.com/continue-revolution/sd-webui-animatediff/tree/v2.0.2-f) in `03/18/2023`: Motion LoRA, i2i batch and GroupNorm hack have been restored. Motion LoRA is built under [KohakuBlueleaf](https://github.com/KohakuBlueleaf)'s [LyCORIS](https://github.com/KohakuBlueleaf/a1111-sd-webui-lycoris) extension. GroupNorm hack is currently in [this](https://github.com/lllyasviel/stable-diffusion-webui-forge/tree/conrevo/gn-patcher-for-early-ad) branch.\n\nWe believe that all features in OG A1111 version (except IP-Adapter prompt travel / SparseCtrl / ControlNet keyframe / FreeInit) have been available in Forge version. We will synchronize ControlNet updates from OG A1111 version, add SparseCtrl and Magic Animate, and add more paramters as soon as we can.\n\nBREAKING CHANGE:\n- You need PyTorch \u003e= 2.0.0 to run this extension.\n- You need [a1111-sd-webui-lycoris](https://github.com/KohakuBlueleaf/a1111-sd-webui-lycoris) extension to run Motion LoRA. Consider ALL LoRAs as LyCORIS models (specify ALL LoRAs as `\u003clyco:whatever:x.y\u003e` instead of `\u003clora:whatever:x.y\u003e`, not only when you use AnimateDiff) if you install LyCORIS extension in Foge.\n- You need to download [Motion LoRA](https://huggingface.co/conrevo/AnimateDiff-A1111/tree/main/lora_v2), [Hotshot-XL](https://huggingface.co/conrevo/AnimateDiff-A1111/resolve/main/motion_module/mm_sdxl_hs.safetensors?download=true), [AnimateDiff V3 Motion Adapter](https://huggingface.co/conrevo/AnimateDiff-A1111/resolve/main/lora_v2/mm_sd15_v3_adapter.safetensors?download=true), [SparseCtrl](https://huggingface.co/conrevo/AnimateDiff-A1111/tree/main/control) from [my HuggingFace repository](https://huggingface.co/conrevo/AnimateDiff-A1111).\n\n## Future Plan\nAlthough [OpenAI Sora](https://openai.com/sora) is far better at following complex text prompts and generating complex scenes, we believe that OpenAI will NOT open source Sora or any other other products they released recently. My current plan is to continue developing this extension until when an open-sourced video model is released, with strong ability to generate complex scenes, easy customization and good ecosystem like SD1.5.\n\nWe will try our best to bring interesting researches into both WebUI and Forge as long as we can. Not all researches will be implemented. You are welcome to submit a feature request if you find an interesting one. We are also open to learn from other equivalent software.\n\nThat said, due to the notorious difficulty in maintaining [sd-webui-controlnet](https://github.com/Mikubill/sd-webui-controlnet), we do NOT plan to implement ANY new research into WebUI if it touches \"reference control\", such as [Magic Animate](https://github.com/magic-research/magic-animate). Such features will be Forge only. Also, some advanced features in [ControlNet Forge Intergrated](https://github.com/lllyasviel/stable-diffusion-webui-forge/tree/main/extensions-builtin/sd_forge_controlnet), such as ControlNet per-frame mask, will also be Forge only. I really hope that I could have bandwidth to rework sd-webui-controlnet, but it requires a huge amount of time.\n\n\n## Model Zoo\nI am maintaining a [huggingface repo](https://huggingface.co/conrevo/AnimateDiff-A1111/tree/main) to provide all official models in fp16 \u0026 safetensors format. You are highly recommended to use my link. You MUST use my link to download [Motion LoRA](https://huggingface.co/conrevo/AnimateDiff-A1111/tree/main/lora_v2), [Hotshot-XL](https://huggingface.co/conrevo/AnimateDiff-A1111/resolve/main/motion_module/mm_sdxl_hs.safetensors?download=true), [AnimateDiff V3 Motion Adapter](https://huggingface.co/conrevo/AnimateDiff-A1111/resolve/main/lora_v2/mm_sd15_v3_adapter.safetensors?download=true), [SparseCtrl](https://huggingface.co/conrevo/AnimateDiff-A1111/tree/main/control). You may still use the old links if you want, for all other models\n\n- \"Official\" models by [@guoyww](https://github.com/guoyww): [Google Drive](https://drive.google.com/drive/folders/1EqLC65eR1-W-sGD0Im7fkED6c8GkiNFI) | [HuggingFace](https://huggingface.co/guoyww/animatediff/tree/main) | [CivitAI](https://civitai.com/models/108836)\n- \"Stabilized\" community models by [@manshoety](https://huggingface.co/manshoety): [HuggingFace](https://huggingface.co/manshoety/AD_Stabilized_Motion/tree/main)\n- \"TemporalDiff\" models by [@CiaraRowles](https://huggingface.co/CiaraRowles): [HuggingFace](https://huggingface.co/CiaraRowles/TemporalDiff/tree/main)\n\n\n## Documentation\n- [How to Use](docs/how-to-use.md) -\u003e [Preparation](docs/how-to-use.md#preparation) | [WebUI](docs/how-to-use.md#webui) | [API](docs/how-to-use.md#api) | [Parameters](docs/how-to-use.md#parameters)\n- [Features](docs/features.md) -\u003e [Img2Vid](docs/features.md#img2vid) | [Prompt Travel](docs/features.md#prompt-travel) | [ControlNet V2V](docs/features.md#controlnet-v2v) | [ [Model Spec](docs/features.md#model-spec) -\u003e [Motion LoRA](docs/features.md#motion-lora) | [V3](docs/features.md#v3) | [SDXL](docs/features.md#sdxl) ]\n- [Performance](docs/performance.md) -\u003e [ [Optimizations](docs/performance.md#optimizations) -\u003e [Attention](docs/performance.md#attention) | [FP8](docs/performance.md#fp8) | [LCM](docs/performance.md#lcm) ] | [VRAM](docs/performance.md#vram) | [Batch Size](docs/performance.md#batch-size)\n- [Demo](docs/demo.md) -\u003e [Basic Usage](docs/demo.md#basic-usage) | [Motion LoRA](docs/demo.md#motion-lora) | [Prompt Travel](docs/demo.md#prompt-travel) | [AnimateDiff V3](docs/demo.md#animatediff-v3) | [AnimateDiff XL](docs/demo.md#animatediff-xl) | [ControlNet V2V](docs/demo.md#controlnet-v2v)\n\n\n## Tutorial \nThere are a lot of wonderful video tutorials on YouTube and bilibili, and you should check those out for now. For the time being, there are a series of updates on the way and I don't want to work on my own before I am satisfied. An official tutorial should come when I am satisfied with the available features.\n\n\n## Thanks\nWe thank all developers and community users who contribute to this repository in many ways, especially\n- [@guoyww](https://github.com/guoyww) for creating AnimateDiff\n- [@limbo0000](https://github.com/limbo0000) for responding to my questions about AnimateDiff\n- [@neggles](https://github.com/neggles) and [@s9roll7](https://github.com/s9roll7) for developing [AnimateDiff CLI Prompt Travel](https://github.com/s9roll7/animatediff-cli-prompt-travel)\n- [@zappityzap](https://github.com/zappityzap) for developing the majority of the [output features](https://github.com/continue-revolution/sd-webui-animatediff/blob/master/scripts/animatediff_output.py)\n- [@thiswinex](https://github.com/thiswinex) for developing FreeInit\n- [@lllyasviel](https://github.com/lllyasviel) for adding me as a collaborator of sd-webui-controlnet and offering technical support for Forge\n- [@KohakuBlueleaf](https://github.com/KohakuBlueleaf) for helping with FP8 and LCM development\n- [@TDS4874](https://github.com/TDS4874) and [@opparco](https://github.com/opparco) for resolving the grey issue which significantly improve the performance\n- [@streamline](https://twitter.com/kaizirod) for providing ControlNet V2V dataset and workflow. His workflow is extremely amazing and definitely worth checking out.\n\n\n## Star History\n\u003ca href=\"https://star-history.com/#continue-revolution/sd-webui-animatediff\u0026Date\"\u003e\n  \u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://api.star-history.com/svg?repos=continue-revolution/sd-webui-animatediff\u0026type=Date\u0026theme=dark\" /\u003e\n    \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://api.star-history.com/svg?repos=continue-revolution/sd-webui-animatediff\u0026type=Date\" /\u003e\n    \u003cimg alt=\"Star History Chart\" src=\"https://api.star-history.com/svg?repos=continue-revolution/sd-webui-animatediff\u0026type=Date\" /\u003e\n  \u003c/picture\u003e\n\u003c/a\u003e\n\n\n## Sponsor\nYou can sponsor me via WeChat, AliPay or [PayPal](https://paypal.me/conrevo). You can also support me via [ko-fi](https://ko-fi.com/conrevo) or [afdian](https://afdian.net/a/conrevo).\n\n| WeChat | AliPay | PayPal |\n| --- | --- | --- |\n| ![216aff0250c7fd2bb32eeb4f7aae623](https://user-images.githubusercontent.com/63914308/232824466-21051be9-76ce-4862-bb0d-a431c186fce1.jpg) | ![15fe95b4ada738acf3e44c1d45a1805](https://user-images.githubusercontent.com/63914308/232824545-fb108600-729d-4204-8bec-4fd5cc8a14ec.jpg) | ![IMG_1419_](https://github.com/continue-revolution/sd-webui-animatediff/assets/63914308/eaa7b114-a2e6-4ecc-a29f-253ace06d1ea) |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontinue-revolution%2Fsd-forge-animatediff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcontinue-revolution%2Fsd-forge-animatediff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontinue-revolution%2Fsd-forge-animatediff/lists"}