{"id":20161915,"url":"https://github.com/ailab-cvc/freenoise","last_synced_at":"2025-04-06T13:10:14.532Z","repository":{"id":203230704,"uuid":"708946286","full_name":"AILab-CVC/FreeNoise","owner":"AILab-CVC","description":"[ICLR 2024] Code for FreeNoise based on VideoCrafter","archived":false,"fork":false,"pushed_at":"2024-01-31T08:37:37.000Z","size":85611,"stargazers_count":327,"open_issues_count":7,"forks_count":24,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-05-16T00:00:24.615Z","etag":null,"topics":["aigc","diffusion","generative-model","video-diffusion-model"],"latest_commit_sha":null,"homepage":"http://haonanqiu.com/projects/FreeNoise.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AILab-CVC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-23T17:47:05.000Z","updated_at":"2024-07-11T15:12:33.323Z","dependencies_parsed_at":"2023-10-29T07:26:07.944Z","dependency_job_id":"49843931-7c3e-4f0b-8c88-38ba0fb50fa5","html_url":"https://github.com/AILab-CVC/FreeNoise","commit_stats":null,"previous_names":["arthur-qiu/longercrafter","ailab-cvc/freenoise"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AILab-CVC%2FFreeNoise","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AILab-CVC%2FFreeNoise/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AILab-CVC%2FFreeNoise/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AILab-CVC%2FFreeNoise/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AILab-CVC","download_url":"https://codeload.github.com/AILab-CVC/FreeNoise/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247485287,"owners_count":20946398,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aigc","diffusion","generative-model","video-diffusion-model"],"created_at":"2024-11-14T00:21:52.639Z","updated_at":"2025-04-06T13:10:14.515Z","avatar_url":"https://github.com/AILab-CVC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## ___***FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling***___\n\n### 🔥🔥🔥 The LongerCrafter for longer high-quality video generation are now released!\n\n\u003cdiv align=\"center\"\u003e\n\u003cp style=\"font-weight: bold\"\u003e\n✅ totally \u003cspan style=\"color: red; font-weight: bold\"\u003eno\u003c/span\u003e tuning \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n✅ less than \u003cspan style=\"color: red; font-weight: bold\"\u003e20%\u003c/span\u003e extra time \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n✅ support \u003cspan style=\"color: red; font-weight: bold\"\u003e512\u003c/span\u003e frames \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n\u003c/p\u003e\n\n \u003ca href='https://arxiv.org/abs/2310.15169'\u003e\u003cimg src='https://img.shields.io/badge/arXiv-2310.15169-b31b1b.svg'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n \u003ca href='http://haonanqiu.com/projects/FreeNoise.html'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-Green'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/MoonQiu/FreeNoise) \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n [![Replicate](https://replicate.com/cjwbw/longercrafter/badge)](https://replicate.com/cjwbw/longercrafter)\n\n\n_**[Haonan Qiu](http://haonanqiu.com/), [Menghan Xia*](https://menghanxia.github.io), [Yong Zhang](https://yzhang2016.github.io), [Yingqing He](https://github.com/YingqingHe), \n\u003cbr\u003e\n[Xintao Wang](https://xinntao.github.io), [Ying Shan](https://scholar.google.com/citations?hl=zh-CN\u0026user=4oXBp9UAAAAJ), and [Ziwei Liu*](https://liuziwei7.github.io/)**_\n\u003cbr\u003e\u003cbr\u003e\n(* corresponding author)\n\nFrom Tencent AI Lab and Nanyang Technological University.\n\n\u003cimg src=assets/t2v/hd01.gif\u003e\n\u003cp\u003eInput: \"A chihuahua in astronaut suit floating in space, cinematic lighting, glow effect\"; \n\u003cbr\u003e\nResolution: 1024 x 576; Frames: 64.\u003c/p\u003e\n\u003cimg src=assets/t2v/hd02.gif\u003e\n\u003cp\u003eInput: \"Campfire at night in a snowy forest with starry sky in the background\"; \n\u003cbr\u003e\nResolution: 1024 x 576; Frames: 64.\u003c/p\u003e\n\u003c/div\u003e\n \n## 🔆 Introduction\n\n\n🤗🤗🤗 LongerCrafter (FreeNoise) is a tuning-free and time-efficient paradigm for longer video generation based on pretrained video diffusion models.\n\n### 1. Longer Single-Prompt Text-to-video Generation\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=assets/t2v/sp512.gif\u003e\n\u003cp\u003eLonger single-prompt results. Resolution: 256 x 256; Frames: 512. (Compressed)\u003c/p\u003e\n\u003c/div\u003e\n\n### 2. Longer Multi-Prompt Text-to-video Generation\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=assets/t2v/mp256.gif\u003e\n\u003cp\u003eLonger multi-prompt results. Resolution: 256 x 256; Frames: 256. (Compressed)\u003c/p\u003e\n\u003c/div\u003e\n\n## 📝 Changelog\n- __[2024.01.28]__: 🔥🔥 Support FreeNoise on VideoCrafter2!\n- __[2024.01.23]__: 🔥🔥 Support FreeNoise on other two video frameworks AnimateDiff and LaVie!\n- __[2023.10.25]__: 🔥🔥 Release the 256x256 model and support multi-prompt generation!\n- __[2023.10.24]__: 🔥🔥 Release the LongerCrafter (FreeNoise), longer video generation!\n\u003cbr\u003e\n\n\n## 🧰 Models\n\n|Model|Resolution|Checkpoint|Description\n|:---------|:---------|:--------|:--------|\n|VideoCrafter (Text2Video)|576x1024|[Hugging Face](https://huggingface.co/VideoCrafter/Text2Video-1024-v1.0/blob/main/model.ckpt)|Support 64 frames on NVIDIA A100 (40GB)\n|VideoCrafter (Text2Video)|256x256|[Hugging Face](https://huggingface.co/VideoCrafter)|Support 512 frames on NVIDIA A100 (40GB)\n|VideoCrafter2 (Text2Video)|320x512|[Hugging Face](https://huggingface.co/VideoCrafter/VideoCrafter2/blob/main/model.ckpt)|Support 128 frames on NVIDIA A100 (40GB)\n\n(Reduce the number of frames when you have smaller GPUs, e.g. 256x256 resolutions with 64 frames.)\n\n## ⚙️ Setup\n\n### Install Environment via Anaconda (Recommended)\n```bash\nconda create -n freenoise python=3.8.5\nconda activate freenoise\npip install -r requirements.txt\n```\n\n\n## 💫 Inference \n### 1. Longer Text-to-Video\n\n\u003c!-- 1) Download pretrained T2V models via [Hugging Face](https://huggingface.co/VideoCrafter/Text2Video-512-v1/blob/main/model.ckpt), and put the `model.ckpt` in `checkpoints/base_512_v1/model.ckpt`.\n2) Input the following commands in terminal.\n```bash\n  sh scripts/run_text2video_freenoise_512.sh\n``` --\u003e\n\n1) Download pretrained T2V models via [Hugging Face](https://huggingface.co/VideoCrafter/Text2Video-1024-v1.0/blob/main/model.ckpt), and put the `model.ckpt` in `checkpoints/base_1024_v1/model.ckpt`.\n2) Input the following commands in terminal.\n```bash\n  sh scripts/run_text2video_freenoise_1024.sh\n```\n\n### 2. Longer Multi-Prompt Text-to-Video\n\n1) Download pretrained T2V models via [Hugging Face](https://huggingface.co/VideoCrafter), and put the `model.ckpt` in `checkpoints/base_256_v1/model.ckpt`.\n2) Input the following commands in terminal.\n```bash\n  sh scripts/run_text2video_freenoise_mp_256.sh\n```\n\n\n## 🧲 Support For Other Models\n\nFreeNoise is supposed to work on other similar frameworks. An easy way to test compatibility is by shuffling the noise to see whether a new similar video can be generated (set eta to 0). If your have any questions about applying FreeNoise to other frameworks, feel free to contact [Haonan Qiu](http://haonanqiu.com/).\n\nCurrent official implementation: [FreeNoise-VideoCrafter](https://github.com/AILab-CVC/FreeNoise), [FreeNoise-AnimateDiff](https://github.com/arthur-qiu/FreeNoise-AnimateDiff), [FreeNoise-LaVie](https://github.com/arthur-qiu/FreeNoise-LaVie) \n\n\n## 👨‍👩‍👧‍👦 Crafter Family\n[VideoCrafter](https://github.com/AILab-CVC/VideoCrafter): Framework for high-quality video generation.\n\n[ScaleCrafter](https://github.com/YingqingHe/ScaleCrafter): Tuning-free method for high-resolution image/video generation.\n\n[TaleCrafter](https://github.com/AILab-CVC/TaleCrafter): An interactive story visualization tool that supports multiple characters.  \n\n\n## 😉 Citation\n```bib\n@misc{qiu2023freenoise,\n      title={FreeNoise: Tuning-Free Longer Video Diffusion Via Noise Rescheduling}, \n      author={Haonan Qiu and Menghan Xia and Yong Zhang and Yingqing He and Xintao Wang and Ying Shan and Ziwei Liu},\n      year={2023},\n      eprint={2310.15169},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\n\n## 📢 Disclaimer\nWe develop this repository for RESEARCH purposes, so it can only be used for personal/research/non-commercial purposes.\n****\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Failab-cvc%2Ffreenoise","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Failab-cvc%2Ffreenoise","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Failab-cvc%2Ffreenoise/lists"}