https://github.com/vita-group/usaid

[Preprint] "Segmentation-Aware Image Denoising without Knowing True Segmentation"
https://github.com/vita-group/usaid

deep-neural-networks denoising pytorch segmentation-aware unsupervised-learning

Last synced: about 23 hours ago
JSON representation

[Preprint] "Segmentation-Aware Image Denoising without Knowing True Segmentation"

• Host: GitHub
• URL: https://github.com/vita-group/usaid
• Owner: VITA-Group
• Created: 2019-06-09T18:22:17.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2021-12-31T07:07:00.000Z (almost 3 years ago)
• Last Synced: 2023-10-20T23:41:57.826Z (about 1 year ago)
• Topics: deep-neural-networks, denoising, pytorch, segmentation-aware, unsupervised-learning
• Language: Python
• Homepage:
• Size: 8.47 MB
• Stars: 34
• Watchers: 3
• Forks: 10
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Segmentation-aware Image Denoising Without Knowing True Segmentation

Implement of the paper: 
 


[Segmentation-aware Image Denoising Without Knowing True Segmentation](https://arxiv.org/abs/1905.08965) 
 


Sicheng Wang, Bihan Wen, Junru Wu, Dacheng Tao, Zhangyang Wang 


## Overview

we propose a segmentation-aware image denoising model dubbed **U-SAID**, which does not need any ground-truth segmentation map in training, and thus can be applied to any image dataset directly. 

We demonstrate the U-SAID generates denoised image has:

* better visual quality; 


* stronger robustness for subsequent semantic segmentation tasks. 


We also manifest U-SAID's superior generalizability in three folds: 

* denoising unseen types of images; 


* pre-processing unseen noisy images for segmentation; 


* pre-processing unseen images for unseen high-level tasks. 


## Methods

![](https://github.com/sharonwang1/seg_denoising/blob/master/docs/images/FlowChart.png)



U-SAID: Network architecture. The USA module is composed of a feature embedding sub-network for transforming the denoised image to a feature space, followed by an unsupervised segmentation sub-network that projects the feature to a segmentation map and calculates its pixel-wise uncertainty.



## Visual Examples

### Visual comparison on [Kodak](http://r0k.us/graphics/kodak/) Images

![](https://github.com/sharonwang1/seg_denoising/blob/master/docs/images/kodak_ship.jpg)

### Semantic segmentation from [Pascal VOC 2012 validation set](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/index.html)

![](https://github.com/sharonwang1/seg_denoising/blob/master/docs/images/VOC_segmentation.jpg)

## How to run

### Dependences

* [PyTorch](http://pytorch.org/)

* [torchvision](https://github.com/pytorch/vision)

* OpenCV for Python

* [tensorboardX](https://github.com/lanpa/tensorboard-pytorch) (TensorBoard for PyTorch)

### Train

```

USAID_train.py

```

### Saved Models

```

Saved_Models/USAID.pth

```

## Citation

If you use this code for your research, please cite our paper.

```

@misc{1905.08965,

Author = {Sicheng Wang and Bihan Wen and Junru Wu and Dacheng Tao and Zhangyang Wang},

Title = {Segmentation-Aware Image Denoising without Knowing True Segmentation},

Year = {2019},

Eprint = {arXiv:1905.08965},

}

```