https://github.com/zeroact/scenetextremover-pytorch

This is a minimal implementation of Scene text removal via cascaded text stroke detection and erasing. This github repository is for studying on image in-painting for scene text erasing. Thank you :)
https://github.com/zeroact/scenetextremover-pytorch

detection erasing inpainting stroke text

Last synced: 27 days ago
JSON representation

This is a minimal implementation of Scene text removal via cascaded text stroke detection and erasing. This github repository is for studying on image in-painting for scene text erasing. Thank you :)

• Host: GitHub
• URL: https://github.com/zeroact/scenetextremover-pytorch
• Owner: ZeroAct
• License: mit
• Created: 2021-05-22T08:31:32.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2021-05-24T12:59:37.000Z (over 3 years ago)
• Last Synced: 2023-03-14T07:20:41.438Z (almost 2 years ago)
• Topics: detection, erasing, inpainting, stroke, text
• Language: Python
• Homepage:
• Size: 1.69 MB
• Stars: 34
• Watchers: 3
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Scene Text Remover Pytorch Implementation

This is a minimal implementation of [Scene text removal via cascaded text stroke detection and erasing](https://arxiv.org/pdf/2011.09768.pdf). This   github repository is for studying on image in-painting for scene text erasing. Thank you :)

## Requirements

Python 3.7 or later with all [requirements.txt](./requirements.txt) dependencies installed, including `torch>=1.6`. To install run:

```

$ pip install -r requirements.txt

```

## Model Summary

![model architecture](./doc/model.png)

 This model has u-net sub modules. 

`Gd` detects text stroke image `Ms` with `I` and `M`. `G'd` detects more precise text stroke `M's`.

Similarly,  `Gr` generates text erased image `Ite`, and `G'r` generates more precise output `I'te`.

## Custom Dictionary

Not to be confused, I renamed the names.

`I` : Input Image (with text)


`Mm` : Text area mask (`M` in the model)


`Ms` : Text stroke mask; output of `Gd`


`Ms_` : Text stroke mask; output of `G'd`


`Msgt` : Text stroke mask ; ground truth


`Ite` : Text erased image; output of `Gr`


`Ite_` : Text erased image; output of `G'r`


`Itegt`: Text erased image; ground truth


## Prepare Dataset

You need to prepare background images in `backs` directory and text binary images in `font_mask` directory. 

![background image, text image example](./doc/back.png)

[part of background image sample, text binary image sample]

Executing `python create_dataset.py` will automatically generate `I`, `Itegt`, `Mm`, `Msgt` data.

(If you already have `I`, `Itegt`, `Mm`, `Msgt`, you can skip this section)

```

├─dataset

│  ├─backs

│  │  # background images

│  └─font_mask

│  │  # text binary images

│  └─train

│  │  └─I

│  │  └─Itegt

│  │  └─Mm

│  │  └─Msgt  

│  └─val

│     └─I

│     └─Itegt

│     └─Mm

│     └─Msgt

```

I generated my dataset with 709 background images and 2410 font mask.

I used 17040 pairs for training and 4260 pairs for validation.

![](./doc/dataset_example.png)

Thanks for helping me gathering background images [sina-Kim]([sina-Kim (github.com)](https://github.com/sina-Kim)).

## Train

All you need to do is:

``` python

python train.py

```

## Result

From the left

`I`, `Itegt`, `Ite`, `Ite_`, `Msgt`, `Ms`, `Ms_`

* Epoch 2


  ![](./doc/epoch1.PNG)

* Epoch 5


  ![](./doc/epoch5.PNG)

* Epoch 10


  ![](./doc/epoch10.PNG)

* Epoch 30


  ![](./doc/epoch30.PNG)

* Epoch 50


  ![](./doc/epoch50.PNG)

* Epoch 120


  ![](./doc/epoch120.PNG)

These are not good enough for real task. I think the reason is lack of dataset and simplicity. 

But, it was a good experience for me to implement the paper.

## Issue

If you are having a trouble to run this code, please use issue tab. Thank you.