Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/huochaitiantang/pytorch-deep-image-matting
Pytorch implementation of deep image matting
https://github.com/huochaitiantang/pytorch-deep-image-matting
deep-learning matting
Last synced: 29 days ago
JSON representation
Pytorch implementation of deep image matting
- Host: GitHub
- URL: https://github.com/huochaitiantang/pytorch-deep-image-matting
- Owner: huochaitiantang
- Created: 2018-08-13T04:40:25.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2022-11-22T02:54:27.000Z (about 2 years ago)
- Last Synced: 2024-08-04T03:02:12.902Z (4 months ago)
- Topics: deep-learning, matting
- Language: Python
- Homepage:
- Size: 34.4 MB
- Stars: 294
- Watchers: 17
- Forks: 71
- Open Issues: 32
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-neural-art - Another PyTorch implementation
README
# pytorch-deep-image-matting
This repository includes the non-official pytorch implementation of [deep image matting](http://openaccess.thecvf.com/content_cvpr_2017/papers/Xu_Deep_Image_Matting_CVPR_2017_paper.pdf).
## Performance
|model |SAD ↓|MSE ↓ |Grad ↓ |Conn ↓ | link |
|------------|---------|---------|---------|---------| ---- |
|stage0-paper| 59.6 | 0.019 | 40.5 | 59.3 | |
|stage1-paper| 54.6 |**0.017**| 36.7 | 55.3 | |
|stage0-our | 56.01 | 0.0173 | 33.71 | 57.57 | |
|stage1-our | 54.42 | 0.0175 | 35.01 | 54.85 |[download](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/v1.4/stage1_sad_54.4.pth)|
|**stage1-our-skip** |**52.99**| 0.0171 | **31.56** | **53.24** |[download](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/V1.5/stage1_skip_sad_52.9.pth)
* Lower metrics show better performance.
* Training batch=1, images=43100, epochs=12 for stage1-our-skip model, it takes about 1 days.
* Test maxSize=1600.
* GPU memory >= 10GB
## Updates
* 2020.09.09: run the demo.py with the latest model (stage1-skip-sad-52.9.pth), and update the visualization results.
* 2020.09.09: **adopt VGG-16 backbone with skip connection**, and we **get better performance with less training costs** (12 epochs), Get [Stage1-Skip-SAD=52.9](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/V1.5/stage1_skip_sad_52.9.pth)
* 2019.10.29: conduct stage0 experiment using current code. Get Stage0-SAD=56.0.
* 2019.09.09: conv6 kernel size from 1x1 to 3x3. Get [Stage1-SAD=54.4](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/v1.4/stage1_sad_54.4.pth). **The performance of stage1 is as good as paper**. While using model released before this day, please change the kernel_size=1 and padding=0 of conv6 in file core/net.py.
* 2019.08.24: Fix cv2.dilate and cv2.erode iterations is set default = 1 and set triamp dilate and erode as the test 1k tirmap (k_size:2-5, iterations:5-15). Get [Stage1-SAD=57.1](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/v1.3/stage1_sad_57.1.pth).
* 2019.07.05: Training with refine stage, fixed encoder-decoder. Get [Stage2-SAD=57.7](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/v1.2/stage2_norm_balance_sad_57.9.pth).
* 2019.06.23: Training with alpha loss and composite loss. Get [Stage1-SAD=58.7](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/v1.2/stage1_norm_balance_sad_58.7.pth).
* 2019.06.17: Training trimap generated by erode as well as dialte to balance the 0 and 1 value. Get [Stage0-SAD=62.0](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/v1.2/stage0_norm_balance_sad_62.0.pth).
* 2019.04.22: Input image is normalized by mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225] and fix crop error. Get [Stage0-SAD=69.1](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/v1.1/stage0_norm_e12_sad_69.1.pth).
* 2018.12.14: Initial. Get [Stage0-SAD=72.9](https://github.com/huochaitiantang/pytorch-deep-image-matting/releases/download/v1.0/my_stage0_sad_72.9.pth).
## Installation
* Python 2.7.12 or 3.6.5
* Pytorch 0.4.0 or 1.0.0
* OpenCV 3.4.3
## Demo
Download our model to the `./model` and run the following command. Then the predict alpha mattes will locate in the folder `./result/example/pred`.
python core/demo.py
## Training
### Adobe-Deep-Image-Matting-Dataset
Please concat author for available.
### MSCOCO-2017-Train-Dataset
[Download](http://images.cocodataset.org/zips/train2017.zip)
### PASCAL-VOC-2012
[Download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar)
### Composite-Dataset
Run the following command and the composite training and test dataset will locate in `Combined_Dataset/Training_set/comp` and `Combined_Dataset/Test_set/comp`, `Combined_Dataset` is the extracted folder of Adobe-Deep-Image-Matting-Dataset
python tools/composite.py
### Pretrained-Model
Run the following command and the pretrained model will locate in `./model/vgg_state_dict.pth`
python tools/chg_model.py
### Start Training
Run the following command and start the training
bash train.sh
## Test
Run the following command and start the test of Adobe-1k-Composite-Dataset
bash deploy.sh
## Evaluation
Please eval with [official Matlab Code](https://docs.google.com/uc?export=download&id=1euP9WmWve3c7EgOwRqgHfnp2H8NXH3OM). and get the SAD, MSE, Grad Conn.
## Visualization
Running model is Stage1-Skip-SAD=52.9, please click to view whole images.
| Image | Trimap | Pred-Alpha | GT-Alpha |
|---|---|---|---|
| | | |
| | | |
| | | |
| | | |
| | | |
## Disclaimer
As covered by the ADOBE IMAGE DATASET LICENSE AGREEMENT, the pre-trained models included in this repository can only be used and distributed for non-commercial purposes.