Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/jkwang28/OSDFace


https://github.com/jkwang28/OSDFace

Last synced: about 2 months ago
JSON representation

Awesome Lists containing this project

README 

        
# OSDFace: One-Step Diffusion Model for Face Restoration



[Jingkai Wang](https://github.com/jkwang28), [Jue Gong](https://github.com/gobunu), [Lin Zhang](https://github.com/wanliyungui), [Zheng Chen](https://zhengchen1999.github.io/), Xing Liu, Hong Gu, [Yutong Liu](https://isabelleliu630.github.io/), [Yulun Zhang](http://yulunzhang.com/), and [Xiaokang Yang](https://scholar.google.com/citations?user=yDEavdMAAAAJ), "One-Step Diffusion Model for Face Restoration", arXiv, 2024



[![arXiv](https://img.shields.io/badge/arXiv-Paper-.svg)](https://arxiv.org/abs/2411.17163) 

[![download](https://img.shields.io/github/downloads/jkwang28/OSDFace/total.svg)](https://github.com/jkwang28/OSDFace/releases)

![visitors](https://visitor-badge.glitch.me/badge?page_id=jkwang28.OSDFace&left_color=green&right_color=red)

[![GitHub Stars](https://img.shields.io/github/stars/jkwang28/OSDFace?style=social)](https://github.com/jkwang28/OSDFace)



[[supplementary material](https://github.com/jkwang28/OSDFace/releases/tag/v1)] [[project page](https://jkwang28.github.io/OSDFace-web/)] [pretrained models]



#### 🔥🔥🔥 News



- **2024-11-25:** This repo is released.



---



> **Abstract:** Diffusion models have demonstrated impressive performance in face restoration. Yet, their multi-step inference process remains computationally intensive, limiting their applicability in real-world scenarios. Moreover, existing methods often struggle to generate face images that are harmonious, realistic, and consistent with the subject’s identity. In this work, we propose OSDFace, a novel one-step diffusion model for face restoration. Specifically, we propose a visual representation embedder (VRE) to better capture prior information and understand the input face. In VRE, low-quality faces are processed by a visual tokenizer and subsequently embedded with a vector-quantized dictionary to generate visual prompts. Additionally, we incorporate a facial identity loss derived from face recognition to further ensure identity consistency. We further employ a generative adversarial network (GAN) as a guidance model to encourage distribution alignment between the restored face and the ground truth. Experimental results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics, generating high-fidelity, natural face images with high identity consistency.



![](images/overall-osdface.png)



---



[](https://imgsli.com/MzIxNTU3) [](https://imgsli.com/MzIxNTU5) [](https://imgsli.com/MzIxNTYw) [](https://imgsli.com/MzIxNTYy)



[](https://imgsli.com/MzIxNTYz) [](https://imgsli.com/MzIxNTY4) [](https://imgsli.com/MzIxNTY5) [](https://imgsli.com/MzIxNTcz)



---



## ⚒️ TODO



* [ ] Release code and pretrained models



## 🔗 Contents



- [ ] Datasets

- [ ] Models

- [ ] Testing

- [ ] Training

- [x] [Results](#Results)

- [x] [Citation](#Citation)

- [ ] [Acknowledgements](#Acknowledgements)



## 🔎 Results



We achieved state-of-the-art performance on synthetic and real-world blur dataset. Detailed results can be found in the paper.



Quantitative Comparisons (click to expand)

  •  Results in Table 1 on synthetic dataset (CelebA-Test) from main paper. 

    

    

    

    • 

  •  Results in Table 2 on real-world datasets (Wider-Test, LFW-Test, WebPhoto-Test) from main paper. 

    

    

    

    • 


    Visual Comparisons (click to expand)

  •  Results in Figure 5 on synthetic dataset (CelebA-Test) from main paper.

    

    

    

    • 

  •  Results in Figure 6 on real-world dataset (Wider-Test, LFW-Test, WebPhoto-Test) from main paper.

    

    

    

    • 


    More Comparisons on Synthetic Dataset...

  • Results in Figure 4, 5, 6 on synthetic dataset (CelebA-Test) from supplemental material.

    

    

    

    

    

    

    

    

    

    • 


    More Comparisons on Real-World Dataset...

  • Results in Figure 7, 8, 9, 10 on real-world datasets (Wider-Test, LFW-Test, WebPhoto-Test) from supplemental material.

    

    

    

    

    

    

    

    

    

    

    

    

    • 


    ## 📎 Citation
    


    If you find the code helpful in your resarch or work, please cite the following paper(s).
    


    ```

    @article{wang2024osdface,

        title={One-Step Diffusion Model for Face Restoration},

        author={Wang, Jingkai and Gong, Jue and Zhang, Lin and Chen, Zheng and Liu, Xing and Gu, Hong and Liu, Yutong and Zhang, Yulun and Yang, Xiaokang},

        journal={arXiv preprint arXiv:2411.17163},

        year={2024}

    }

    ```
    


    ## 💡 Acknowledgements
    


    [TBD]
    


    ![Visitor Count](https://profile-counter.glitch.me/jkwang28/count.svg)