{"id":18322502,"url":"https://github.com/tencentarc/vqfr","last_synced_at":"2025-08-16T08:04:31.093Z","repository":{"id":51054322,"uuid":"490120217","full_name":"TencentARC/VQFR","owner":"TencentARC","description":"ECCV 2022, Oral, VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder","archived":false,"fork":false,"pushed_at":"2022-12-15T15:26:48.000Z","size":10290,"stargazers_count":331,"open_issues_count":24,"forks_count":44,"subscribers_count":26,"default_branch":"main","last_synced_at":"2025-03-31T12:08:26.618Z","etag":null,"topics":["face-restoration","vector-quantization"],"latest_commit_sha":null,"homepage":"https://ycgu.site/projects/vqfr","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TencentARC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-05-09T03:19:57.000Z","updated_at":"2025-03-03T00:45:10.000Z","dependencies_parsed_at":"2023-01-29T03:30:57.591Z","dependency_job_id":null,"html_url":"https://github.com/TencentARC/VQFR","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FVQFR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FVQFR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FVQFR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FVQFR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TencentARC","download_url":"https://codeload.github.com/TencentARC/VQFR/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247675597,"owners_count":20977376,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["face-restoration","vector-quantization"],"created_at":"2024-11-05T18:24:53.909Z","updated_at":"2025-04-07T15:07:39.677Z","avatar_url":"https://github.com/TencentARC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# VQFR (ECCV 2022 Oral)\n\n[![download](https://img.shields.io/github/downloads/TencentARC/VQFR/total.svg)](https://github.com/TencentARC/VQFR/releases)\n[![Open issue](https://img.shields.io/github/issues/TencentARC/VQFR)](https://github.com/TencentARC/VQFR/issues)\n[![Closed issue](https://img.shields.io/github/issues-closed/TencentARC/VQFR)](https://github.com/TencentARC/VQFR/issues)\n[![LICENSE](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/TencentARC/VQFR/blob/master/LICENSE)\n\u003ca href=\"https://colab.research.google.com/drive/1Nd_PUrHaYmeEAOF5f_Zi0VuOxlJ62gLr?usp=sharing\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e\n\n\n1. [Colab Demo](https://colab.research.google.com/drive/1Nd_PUrHaYmeEAOF5f_Zi0VuOxlJ62gLr?usp=sharing) for VQFR\n1. Online demo: [Replicate.ai](https://replicate.com/tencentarc/vqfr) (may need to sign in, return the whole image)\n\n:triangular_flag_on_post: **Updates**\n\n- :white_check_mark: **2022.10.16** Clean research codes \u0026 Update VQFR-v2. In this version, we emphasize the restoration **quality** of the texture branch and **balance fidelity with user control**. \u003ca href=\"https://colab.research.google.com/drive/1Nd_PUrHaYmeEAOF5f_Zi0VuOxlJ62gLr?usp=sharing\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e\n\n- :white_check_mark: Support enhancing non-face regions (background) with [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN).\n- :white_check_mark: The Colab Demo of VQFR is created.\n- :white_check_mark: The training/inference codes and pretrained models in paper are released.\n\nThis paper aims at investigating the **potential and limitation of Vector-Quantized (VQ) dictionary** for blind face restoration.\n\u003cbr\u003e\nWe propose a new framework VQFR – incoporating the **Vector-Quantized Dictionary** and the **Parallel Decoder**.\nCompare with previous arts, VQFR produces more realistic facial details and keep the comparable fidelity.\n\n---\n\n### VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder\n\n\u003e [[Paper](https://arxiv.org/abs/2205.06803)] \u0026emsp; [[Project Page](https://ycgu.site/projects/vqfr)] \u0026emsp; [Video] \u0026emsp; [B站] \u0026emsp; [Poster] \u0026emsp; [Slides]\u003cbr\u003e\n\u003e [Yuchao Gu](https://ycgu.site/), [Xintao Wang](https://xinntao.github.io/), [Liangbin Xie](https://liangbinxie.github.io/), [Chao Dong](https://scholar.google.com.hk/citations?user=OSDCB0UAAAAJ), Gen Li, [Ying Shan](https://scholar.google.com/citations?user=4oXBp9UAAAAJ\u0026hl=en),\n[Ming-Ming Cheng](https://scholar.google.com/citations?user=huWpVyEAAAAJ\u0026hl=en)\u003cbr\u003e\n\u003e Nankai University; Tencent ARC Lab; Tencent Online Video; Shanghai AI Laboratory;\u003cbr\u003e\nShenzhen Institutes of Advanced Technology, Chinese Academy of Sciences\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/teaser.jpg\"\u003e\n\u003c/p\u003e\n\n---\n\n## :wrench: Dependencies and Installation\n\n- Python \u003e= 3.7 (Recommend to use [Anaconda](https://www.anaconda.com/download/#linux) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html))\n- [PyTorch \u003e= 1.7](https://pytorch.org/)\n- Option: NVIDIA GPU + [CUDA](https://developer.nvidia.com/cuda-downloads)\n- Option: Linux\n\n### Installation\n\n1. Clone repo\n\n    ```bash\n    git clone https://github.com/TencentARC/VQFR.git\n    cd VQFR\n    ```\n\n1. Install dependent packages\n\n    ```bash\n    # Build VQFR with extension\n    pip install -r requirements.txt\n    VQFR_EXT=True python setup.py develop\n\n    # Following packages are required to run demo.py\n\n    # Install basicsr - https://github.com/xinntao/BasicSR\n    pip install basicsr\n\n    # Install facexlib - https://github.com/xinntao/facexlib\n    # We use face detection and face restoration helper in the facexlib package\n    pip install facexlib\n\n    # If you want to enhance the background (non-face) regions with Real-ESRGAN,\n    # you also need to install the realesrgan package\n    pip install realesrgan\n    ```\n\n## :zap: Quick Inference\n\nDownload pre-trained **VQFRv1/v2 models** [[Google Drive](https://drive.google.com/drive/folders/1lczKYEbARwe27FJlKoFdng7UnffGDjO2?usp=sharing)].\n\n**Inference**\n\n```bash\n# for real-world image\npython demo.py -i inputs/whole_imgs -o results -v 2.0 -s 2 -f 0.1\n\n# for cropped face\npython demo.py -i inputs/cropped_faces/ -o results -v 2.0 -s 1 -f 0.1 --aligned\n```\n\n```console\nUsage: python demo.py -i inputs/whole_imgs -o results -v 2.0 -s 2 -f 0.1 [options]...\n\n  -h                   show this help\n  -i input             Input image or folder. Default: inputs/whole_imgs\n  -o output            Output folder. Default: results\n  -v version           VQFR model version. Option: 1.0. Default: 1.0\n  -f fidelity_ratio    VQFRv2 model supports user control fidelity ratio, range from [0,1]. 0 for the best quality and 1 for the best fidelity. Default: 0\n  -s upscale           The final upsampling scale of the image. Default: 2\n  -bg_upsampler        background upsampler. Default: realesrgan\n  -bg_tile             Tile size for background sampler, 0 for no tile during testing. Default: 400\n  -suffix              Suffix of the restored faces\n  -only_center_face    Only restore the center face\n  -aligned             Input are aligned faces\n  -ext                 Image extension. Options: auto | jpg | png, auto means using the same extension as inputs. Default: auto\n```\n\n## :computer: Training\n\nWe provide the training codes for VQFR (used in our paper).\n\n- Dataset preparation: [FFHQ](https://github.com/NVlabs/ffhq-dataset)\n- Download lpips weights [[Google Drive](https://drive.google.com/drive/folders/1weXfn5mdIwp2dEfDbNNUkauQgo8fx-2D?usp=sharing)] into **experiments/pretrained_models/**\n\n**Codebook Training**\n\n- Pre-train VQ codebook on FFHQ datasets.\n```bash\n python -m torch.distributed.launch --nproc_per_node=8 --master_port=2022 vqfr/train.py -opt options/train/VQGAN/train_vqgan_v1_B16_800K.yml --launcher pytorch\n```\n\n- Or download our pretrained **VQ codebook** [Google Drive](https://drive.google.com/drive/folders/1lczKYEbARwe27FJlKoFdng7UnffGDjO2?usp=sharing) and put them in the `experiments/pretrained_models` folder.\n\n**Restoration Training**\n\n- Modify the configuration file `options/train/VQFR/train_vqfr_v1_B16_200K.yml` accordingly.\n\n- Training\n```bash\npython -m torch.distributed.launch --nproc_per_node=8 --master_port=2022 vqfr/train.py -opt options/train/VQFR/train_vqfr_v1_B16_200K.yml --launcher pytorch\n```\n\n## :straight_ruler: Evaluation\n\nWe evaluate VQFR on one synthetic dataset **CelebA-Test**, and three real-world datasets **LFW-Test**, **CelebChild** and **Webphoto-Test**.\nFor reproduce our evaluation results, you need to perform the following steps:\n\n1. Download testing datasets (or VQFR results) by the following links:\n\n\u003ctable\u003e\n\u003ctr\u003e\n    \u003cth\u003eName\u003c/th\u003e\n    \u003cth\u003eDatasets\u003c/th\u003e\n    \u003cth\u003eShort Description\u003c/th\u003e\n    \u003cth\u003eDownload\u003c/th\u003e\n    \u003cth\u003eVQFR Results\u003c/th\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd rowspan=\"5\"\u003eTesting Datasets\u003c/td\u003e\n    \u003ctd\u003eCelebA-Test(LQ/HQ)\u003c/td\u003e\n    \u003ctd\u003e\u003csub\u003e3000 (LQ, HQ) synthetic images for testing\u003c/sub\u003e\u003c/td\u003e\n    \u003ctd rowspan=\"4\"\u003e\u003ca href=\"https://drive.google.com/drive/folders/1CPdaCijkhl8ojg9ZDCPXntQrnw-8tUeD?usp=sharing\"\u003eGoogle Drive\u003c/a\u003e \u003c/td\u003e\n    \u003ctd rowspan=\"4\"\u003e\u003ca href=\"https://drive.google.com/drive/folders/1w-qo5BsKBzSmE0hwS9Hl5G9fn4VKVweU?usp=sharing\"\u003eGoogle Drive\u003c/a\u003e \u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003eLFW-Test(LQ)\u003c/td\u003e\n    \u003ctd\u003e\u003csub\u003e1711 real-world images for testing\u003c/sub\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003eCelebChild(LQ)\u003c/td\u003e\n    \u003ctd\u003e\u003csub\u003e180 real-world images for testing\u003c/sub\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003eWebphoto-Test(LQ)\u003c/td\u003e\n    \u003ctd\u003e\u003csub\u003e469 real-world images for testing\u003c/sub\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n2. Install related package and download pretrained models for different metrics:\n\n```bash\n    # LPIPS\n    pip install lpips\n\n    # Deg.\n    cd metric_paper/\n    git clone https://github.com/ronghuaiyang/arcface-pytorch.git\n    mv arcface-pytorch/ arcface/\n    rm arcface/config/__init__.py arcface/models/__init__.py\n\n    # put pretrained models of different metrics to \"experiments/pretrained_models/metric_weights/\"\n```\n\u003ctable\u003e\n\u003ctr\u003e\n    \u003cth\u003eMetrics\u003c/th\u003e\n    \u003cth\u003ePretrained Weights\u003c/th\u003e\n    \u003cth\u003eDownload\u003c/th\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003eFID\u003c/td\u003e\n    \u003ctd\u003einception_FFHQ_512.pth\u003c/td\u003e\n    \u003ctd rowspan=\"3\"\u003e\u003ca href=\"https://drive.google.com/drive/folders/1k3RCSliF6PsujCMIdCD1hNM63EozlDIZ?usp=sharing\"\u003eGoogle Drive\u003c/a\u003e \u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003eDeg\u003c/td\u003e\n    \u003ctd\u003eresnet18_110.pth\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n    \u003ctd\u003eLMD\u003c/td\u003e\n    \u003ctd\u003ealignment_WFLW_4HG.pth\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n3. Generate restoration results:\n\n- Specify the dataset_lq/dataset_gt to the testing dataset root in **test_vqfr_v1.yml**.\n\n- Then run the following command:\n```bash\n    python vqfr/test.py -opt options/test/VQFR/test_vqfr_v1.yml\n```\n\n4. Run evaluation:\n```bash\n    # LPIPS|PSNR/SSIM|LMD|Deg.\n    python metric_paper/[calculate_lpips.py|calculate_psnr_ssim.py|calculate_landmark_distance.py|calculate_cos_dist.py]\n    -restored_folder folder_to_results -gt_folder folder_to_gt\n\n    # FID|NIQE\n    python metric_paper/[calculate_fid_folder.py|calculate_niqe.py] -restored_folder folder_to_results\n```\n\n## :scroll: License\n\nVQFR is released under Apache License Version 2.0.\n\n## :eyes: Acknowledgement\n\nThanks to the following open-source projects:\n\n[Taming-transformers](https://github.com/CompVis/taming-transformers)\n\n[GFPGAN](https://github.com/TencentARC/GFPGAN)\n\n[DistSup](https://github.com/distsup/DistSup)\n\n\n## :clipboard: Citation\n\n    @inproceedings{gu2022vqfr,\n      title={VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder},\n      author={Gu, Yuchao and Wang, Xintao and Xie, Liangbin and Dong, Chao and Li, Gen and Shan, Ying and Cheng, Ming-Ming},\n      year={2022},\n      booktitle={ECCV}\n    }\n\n## :e-mail: Contact\n\nIf you have any question, please email `yuchaogu9710@gmail.com`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fvqfr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftencentarc%2Fvqfr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fvqfr/lists"}