{"id":18279000,"url":"https://github.com/sczhou/upscale-a-video","last_synced_at":"2025-05-16T06:04:13.289Z","repository":{"id":210899267,"uuid":"725725007","full_name":"sczhou/Upscale-A-Video","owner":"sczhou","description":" [CVPR 2024] Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution","archived":false,"fork":false,"pushed_at":"2024-09-27T10:50:59.000Z","size":11340,"stargazers_count":1190,"open_issues_count":33,"forks_count":64,"subscribers_count":76,"default_branch":"master","last_synced_at":"2025-04-06T17:11:21.721Z","etag":null,"topics":["aigc-enhancement","deflicker","video-diffusion-model","video-super-resolution"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sczhou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-30T18:38:24.000Z","updated_at":"2025-04-05T23:44:51.000Z","dependencies_parsed_at":"2024-01-18T22:40:11.713Z","dependency_job_id":"9ef7dfbc-059e-454d-80db-ba90e0b7ef16","html_url":"https://github.com/sczhou/Upscale-A-Video","commit_stats":null,"previous_names":["sczhou/upscale-a-video"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sczhou%2FUpscale-A-Video","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sczhou%2FUpscale-A-Video/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sczhou%2FUpscale-A-Video/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sczhou%2FUpscale-A-Video/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sczhou","download_url":"https://codeload.github.com/sczhou/Upscale-A-Video/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254478163,"owners_count":22077675,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aigc-enhancement","deflicker","video-diffusion-model","video-super-resolution"],"created_at":"2024-11-05T12:26:59.572Z","updated_at":"2025-05-16T06:04:13.223Z","avatar_url":"https://github.com/sczhou.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003ch1\u003e\n    Upscale-A-Video:\u003cbr\u003e \n    Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution\n\u003c/h1\u003e\n\n\u003cdiv\u003e\n    \u003ca href='https://shangchenzhou.com/' target='_blank'\u003eShangchen Zhou\u003csup\u003e∗\u003c/sup\u003e\u003c/a\u003e\u0026emsp;\n    \u003ca href='https://pq-yang.github.io/' target='_blank'\u003ePeiqing Yang\u003csup\u003e∗\u003c/sup\u003e\u003c/a\u003e\u0026emsp;\n    \u003ca href='https://iceclear.github.io/' target='_blank'\u003eJianyi Wang\u003c/a\u003e\u0026emsp;\n    \u003ca href='https://github.com/Luo-Yihang' target='_blank'\u003eYihang Luo\u003c/a\u003e\u0026emsp;\n    \u003ca href='https://www.mmlab-ntu.com/person/ccloy/' target='_blank'\u003eChen Change Loy\u003c/a\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n    S-Lab, Nanyang Technological University\n\u003c/div\u003e\n\n\u003cdiv\u003e\n    \u003cstrong\u003eCVPR 2024 (Highlight)\u003c/strong\u003e\n\u003c/div\u003e\n\n\u003cdiv\u003e\n    \u003ch4 align=\"center\"\u003e\n        \u003ca href=\"https://shangchenzhou.com/projects/upscale-a-video/\" target='_blank'\u003e\n        \u003cimg src=\"https://img.shields.io/badge/🐳-Project%20Page-blue\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://arxiv.org/abs/2312.06640\" target='_blank'\u003e\n        \u003cimg src=\"https://img.shields.io/badge/arXiv-2312.06640-b31b1b.svg\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://www.youtube.com/watch?v=b9J3lqiKnLM\" target='_blank'\u003e\n        \u003cimg src=\"https://img.shields.io/badge/Demo%20Video-%23FF0000.svg?logo=YouTube\u0026logoColor=white\"\u003e\n        \u003c/a\u003e\n        \u003ca href=\"https://replicate.com/sczhou/upscale-a-video\" target='_blank'\u003e\n        \u003cimg src=\"https://replicate.com/sczhou/upscale-a-video/badge\"\u003e\n        \u003c/a\u003e\n        \u003cimg src=\"https://api.infinitescript.com/badgen/count?name=sczhou/Upscale-A-Video\"\u003e\n    \u003c/h4\u003e\n\u003c/div\u003e\n\n\u003cstrong\u003eUpscale-A-Video is a diffusion-based model that upscales videos by taking the low-resolution video and text prompts as inputs.\u003c/strong\u003e\n\n\u003cdiv style=\"width: 100%; text-align: center; margin:auto;\"\u003e\n    \u003cimg style=\"width:100%\" src=\"assets/teaser.png\"\u003e\n\u003c/div\u003e\n\n:open_book: For more visual results, go checkout our \u003ca href=\"##\" target=\"_blank\"\u003eproject page\u003c/a\u003e\n\n---\n\u003c/div\u003e\n\n\n## 🔥 Update\n- [2024.09] Inference code is released.\n- [2024.02] YouHQ dataset is made publicly available.\n- [2023.12] This repo is created.\n\n## 🎬 Overview\n![overall_structure](assets/pipeline.png)\n\n## 🔧 Dependencies and Installation\n1. Clone Repo\n    ```bash\n    git clone https://github.com/sczhou/Upscale-A-Video.git\n    cd Upscale-A-Video\n    ```\n\n2. Create Conda Environment and Install Dependencies\n    ```bash\n    # create new conda env\n    conda create -n UAV python=3.9 -y\n    conda activate UAV\n\n    # install python dependencies\n    pip install -r requirements.txt\n    ```\n\n3. Download Models\n\n   (a) Download pretrained models and configs from [Google Drive](https://drive.google.com/drive/folders/1O8pbeR1hsRlFUU8O4EULe-lOKNGEWZl1?usp=sharing) and put them under the `pretrained_models/upscale_a_video` folder.\n\n   The [`pretrained_models`](./pretrained_models) directory structure should be arranged as:\n\n    ```\n    ├── pretrained_models\n    │   ├── upscale_a_video\n    │   │   ├── low_res_scheduler\n    │   │       ├── ...\n    │   │   ├── propagator\n    │   │       ├── ...\n    │   │   ├── scheduler\n    │   │       ├── ...\n    │   │   ├── text_encoder\n    │   │       ├── ...\n    │   │   ├── tokenizer\n    │   │       ├── ...\n    │   │   ├── unet\n    │   │       ├── ...\n    │   │   ├── vae\n    │   │       ├── ...\n    ```\n    \n    (a) (Optional) LLaVA can be downloaded automatically when set `--use_llava` to `True`, for users with access to huggingface.\n\n\n## ☕️ Quick Inference\n\nThe `--input_path` can be either the path to a single video or a folder containing multiple videos.\n\nWe provide several examples in the [`inputs`](./inputs) folder. \nRun the following commands to try it out:\n\n```shell\n## AIGC videos\npython inference_upscale_a_video.py \\\n-i ./inputs/aigc_1.mp4 -o ./results -n 150 -g 6 -s 30 -p 24,26,28\n\npython inference_upscale_a_video.py \\\n-i ./inputs/aigc_2.mp4 -o ./results -n 150 -g 6 -s 30 -p 24,26,28\n\npython inference_upscale_a_video.py \\\n-i ./inputs/aigc_3.mp4 -o ./results -n 150 -g 6 -s 30 -p 20,22,24\n```\n\n```shell\n## old videos/movies/animations \npython inference_upscale_a_video.py \\\n-i ./inputs/old_video_1.mp4 -o ./results -n 150 -g 9 -s 30\n\npython inference_upscale_a_video.py \\\n-i ./inputs/old_movie_1.mp4 -o ./results -n 100 -g 5 -s 20 -p 17,18,19\n\npython inference_upscale_a_video.py \\\n-i ./inputs/old_movie_2.mp4 -o ./results -n 120 -g 6 -s 30 -p 8,10,12\n\npython inference_upscale_a_video.py \\\n-i ./inputs/old_animation_1.mp4 -o ./results -n 120 -g 6 -s 20 --use_video_vae\n```\n\nIf you notice any color discrepancies between the output and the input, you can set `--color_fix` to `\"AdaIn\"` or `\"Wavelet\"`. By default, it is set to `\"None\"`.\n\n\n\n## 🎞️ YouHQ Dataset\nThe datasets are hosted on Google Drive\n\n| Dataset | Link | Description|\n| :----- | :--: | :---- | \n| YouHQ-Train | [Google Drive](https://drive.google.com/file/d/1f8g8gTHzQq-cKt4s94YQXDwJcdjL59lK/view?usp=sharing)| 38,576 videos for training, each of which has around 32 frames.|\n| YouHQ40-Test| [Google Drive](https://drive.google.com/file/d/1rkeBQJMqnRTRDtyLyse4k6Vg2TilvTKC/view?usp=sharing) | 40 video clips for evaluation, each of which has around 32 frames.|\n\n## 📑 Citation\n\n   If you find our repo useful for your research, please consider citing our paper:\n\n   ```bibtex\n   @inproceedings{zhou2024upscaleavideo,\n      title={{Upscale-A-Video}: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution},\n      author={Zhou, Shangchen and Yang, Peiqing and Wang, Jianyi and Luo, Yihang and Loy, Chen Change},\n      booktitle={CVPR},\n      year={2024}\n   }\n   ```\n\n\n## 📝 License\n\nThis project is licensed under \u003ca rel=\"license\" href=\"./LICENSE\"\u003eNTU S-Lab License 1.0\u003c/a\u003e. Redistribution and use should follow this license.\n\n\n## 📧 Contact\nIf you have any questions, please feel free to reach us at `shangchenzhou@gmail.com` or `peiqingyang99@outlook.com`. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsczhou%2Fupscale-a-video","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsczhou%2Fupscale-a-video","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsczhou%2Fupscale-a-video/lists"}