Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/vitae-transformer/vitae-transformer-matting

A comprehensive list of our research works related to image matting, including papers, codes, datasets, demos, and citations. Note: The repo for [IJCV'23] "Rethinking Portrait Matting with Privacy Preserving" has been moved to: https://github.com/ViTAE-Transformer/P3M-Net
https://github.com/vitae-transformer/vitae-transformer-matting

computer-vision deep-learning image-matting privacy-preserving survey vision-transformer

Last synced: about 7 hours ago
JSON representation

A comprehensive list of our research works related to image matting, including papers, codes, datasets, demos, and citations. Note: The repo for [IJCV'23] "Rethinking Portrait Matting with Privacy Preserving" has been moved to: https://github.com/ViTAE-Transformer/P3M-Net

Awesome Lists containing this project

README 

        
Image Matting



This repo contains a comprehensive list of our research works related to **image matting**, including papers, codes, datasets, demos, and citations. For any related questions, please contact Jizhizi Li at [[email protected]](mailto:[email protected]) and Sihan Ma at [[email protected]](mailto:[email protected]).



***

>

>
🚀 News


>

> [2023-04-10]: Publish the paper [Deep Image Matting: A Comprehensive Survey](https://arxiv.org/abs/2304.04672) on arXiv.

> 

> [2023-03-28]: The paper [Rethinking Portrait Matting with Privacy Preserving](https://arxiv.org/abs/2203.16828) has been accepted by the International Journal of Computer Vision ([IJCV](https://www.springer.com/journal/11263)) 🎉

> 

> [2023-02-28]: The paper [Referring Image Matting](https://arxiv.org/abs/2206.05149) has been accepted by the Computer Vision and Pattern Recognition Conference ([CVPR](https://cvpr2023.thecvf.com/)) 🎉

***



## Overview



[1. Deep Image Matting: A Comprehensive Survey, arXiv, 2023](#survey)    



[2. Rethinking Portrait Matting with Privacy Preserving, IJCV, 2023](#rethink_p3m)     



[3. Referring Image Matting, CVPR, 2023](#rim)     



[4. Bridging Composite and Real: Towards End-to-end Deep Image Matting, IJCV, 2022](#gfm)       



[5. Privacy-preserving Portrait Matting, ACM MM, 2021](#p3m)     



[6. Deep Automatic Natural Image Matting, IJCAI, 2021](#aim)     



## Projects



### 📘 Deep Image Matting: A Comprehensive Survey [arXiv-2023]



Jizhizi Li, Jing Zhang, and Dacheng Tao



[Paper](https://arxiv.org/abs/2304.04672) | [Github Code](https://github.com/JizhiziLi/matting-survey) | [BibTex](./assets/arXiv_2023_Survey/matting_survey.bib)



Image matting refers to extracting precise alpha matte from natural images, and it plays a critical role in various downstream applications, such as image editing. The emergence of deep learning has revolutionized the field of image matting and given birth to multiple new techniques, including automatic, interactive, and referring image matting. Here we present a comprehensive review of recent advancements in image matting in the era of deep learning.






***



### 📘 Rethinking Portrait Matting with Privacy Preserving [IJCV-2023]






Sihan Ma∗, Jizhizi Li∗, Jing Zhang, He Zhang, and Dacheng Tao. (*equal contribution)



[Paper](https://arxiv.org/abs/2203.16828) | [Github Code](https://github.com/ViTAE-Transformer/P3M-Net) | [Dataset](https://github.com/ViTAE-Transformer/P3M-Net#ppt-setting-and-p3m-10k-dataset) | [Demo](https://colab.research.google.com/drive/1pD_XKx31Lgd7zwq46dRpz2jGdsH1ZIay?usp=sharing) | [BibTex](./assets/IJCV_2023_Rethink_P3M/rethink_p3m.bib)



This paper introduces three variants of P3M-Net based on both transformer and CNN backbones to solve the portrait matting problem with privacy preserving. Also a simple yet effective Copy and Paste strategy (P3M-CP) is devised to enable the matting model to process both face-blurred and normal images without extra effort during inference.






---



### 📘 Referring Image Matting [CVPR-2023]






Jizhizi Li, Jing Zhang, and Dacheng Tao



[Paper](https://arxiv.org/abs/2206.05149) |  [Github Code](https://github.com/JizhiziLi/rim/) | [Dataset](https://github.com/JizhiziLi/rim/#refmatte) | [BibTex](./assets/CVPR_2023_RIM/rim.bib)



Image matting refers to extracting the accurate foregrounds in the image. Current automatic methods tend to extract all the salient objects in the image indiscriminately. In this paper, we propose a new task named Referring Image Matting (RIM), referring to extracting the meticulous alpha matte of the specific object that can best match the given natural language description. We then propose a large-scale dataset RefMatte and a carefully designed method CLIPMat to serve as a baseline suite for RIM. We believe the new task RIM along with the RefMatte dataset and the method CLIPMat will open new research directions in this area and facilitate future studies.






---



### 📘 Bridging Composite and Real: Towards End-to-end Deep Image Matting [IJCV-2022]






Jizhizi Li1∗, Jing Zhang1∗, Stephen J. Maybank, and Dacheng Tao. (*equal contribution)



[Paper](https://arxiv.org/abs/2010.16188) |  [Github Code](https://github.com/JizhiziLi/gfm/) | [Dataset](https://github.com/JizhiziLi/gfm/#am-2k) | [Demo](https://colab.research.google.com/drive/1EaQ5h4u9Q_MmDSFTDmFG0ZOeSsFuRTsJ?usp=sharing) | [BibTex](./assets/IJCV_2022_GFM/gfm.bib)



We propose a novel Glance and Focus Matting network (GFM), which employs a shared encoder and two separate decoders to learn both tasks in a collaborative manner for end-to-end image matting. We also establish a novel Animal Matting dataset (AM-2k) to serve for end-to-end matting task. Furthermore, we investigate the domain gap issue between composition images and natural images systematically, propose a carefully designed composite route RSSN and a large-scale high-resolution background dataset (BG-20k) to serve as better candidates for composition.






---



### 📘 Privacy-Preserving Portrait Matting [ACM MM-21]






Jizhizi Li∗, Sihan Ma∗, Jing Zhang, and Dacheng Tao. (*equal contribution)



[Paper](https://arxiv.org/abs/2104.14222) | [Github Code](https://github.com/JizhiziLi/P3M) | [Dataset](https://github.com/JizhiziLi/p3m/#ppt-setting-and-p3m-10k-dataset) | [BibTex](./assets/ACM_MM_2021_P3M/p3m.bib)



This work presents P3M-10k, which is the first large-scale anonymized benchmark for Privacy-Preserving Portrait Matting, to solve the increasing concerns about the privacy in image matting. They also propose P3M-Net, which leverages the power of a unified framework for both semantic perception and detail matting, and specifically emphasizes the interaction between them and the encoder to facilitate the matting process. 






---



### 📘 Deep Automatic Natural Image Matting [IJCAI-21]






Jizhizi Li, Jing Zhang, and Dacheng Tao



[Paper](https://arxiv.org/abs/2107.07235) | [Github Code](https://github.com/JizhiziLi/aim/) | [Dataset](https://github.com/jizhiziLi/aim#aim-500) | [BibTex](./assets/IJCAI_2021_AIM/aim.bib)



We investigate the difficulties when extending the automatic matting methods to natural images with salient transparent/meticulous foregrounds or non-salient foregrounds by proposing a novel end-to-end matting network, which can predict a generalized trimap for any image of the above types as a unified semantic representation and simultaneously guide the matting network to focus on the transition areas via an attention mechanism. We also construct a test set AIM-500 that contains 500 diverse natural images covering all types along with manually labeled alpha mattes, making it feasible to benchmark the generalization ability of AIM models.