https://github.com/prasunroy/stefann

:fire: [CVPR 2020] STEFANN: Scene Text Editor using Font Adaptive Neural Network (official code).
https://github.com/prasunroy/stefann

color-transfer colornet computer-vision cvpr cvpr2020 deep-learning fannet font-generation scene-text-editor stefann

Last synced: 16 days ago
JSON representation

:fire: [CVPR 2020] STEFANN: Scene Text Editor using Font Adaptive Neural Network (official code).

Host: GitHub
URL: https://github.com/prasunroy/stefann
Owner: prasunroy
License: apache-2.0
Created: 2020-04-15T13:30:46.000Z (about 5 years ago)
Default Branch: master
Last Pushed: 2024-04-30T15:14:56.000Z (12 months ago)
Last Synced: 2025-03-31T14:12:56.004Z (23 days ago)
Topics: color-transfer, colornet, computer-vision, cvpr, cvpr2020, deep-learning, fannet, font-generation, scene-text-editor, stefann
Language: Python
Homepage: https://prasunroy.github.io/stefann
Size: 15.4 MB
Stars: 268
Watchers: 10
Forks: 41
Open Issues: 7
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

Getting Started  •
Training Networks  •
External Links  •
Citation  •
License

The official GitHub repository for the paper on STEFANN: Scene Text Editor using Font Adaptive Neural Network.

### :boom: Quick installation
#### Step 1: Install [Git](https://git-scm.com/) and [Conda](https://docs.conda.io/) package manager (Miniconda / Anaconda)
#### Step 2: Update and configure Conda
```bash
conda update conda
conda config --set env_prompt "({name}) "
```
#### Step 3: Clone this repository and change directory to repository root
```bash
git clone https://github.com/prasunroy/stefann.git
cd stefann
```
#### Step 4: Create an environment and install depenpencies

#### On Linux and Windows
* To create **CPU** environment: `conda env create -f release/env_cpu.yml`
* To create **GPU** environment: `conda env create -f release/env_gpu.yml`

#### On macOS
* To create **CPU** environment: `conda env create -f release/env_osx.yml`

### :boom: Quick test
#### Step 1: [Download](https://drive.google.com/open?id=16-mq3MOR1zmOsxNgegRmGDeVRyeyQ0_H) models and pretrained checkpoints into `release/models` directory
#### Step 2: [Download](https://drive.google.com/uc?export=download&id=1Gzb-VYeQJNXwDnkoEI4iAskOGYmWR6Rk) sample images and extract into `release/sample_images` directory
```
stefann/
├── ...
├── release/
│ ├── models/
│ │ ├── colornet.json
│ │ ├── colornet_weights.h5
│ │ ├── fannet.json
│ │ └── fannet_weights.h5
│ ├── sample_images/
│ │ ├── 01.jpg
│ │ ├── 02.jpg
│ │ └── ...
│ └── ...
└── ...
```
#### Step 3: Activate environment
To activate **CPU** environment: `conda activate stefann-cpu`

To activate **GPU** environment: `conda activate stefann-gpu`
#### Step 4: Change directory to `release` and run STEFANN
```bash
cd release
python stefann.py
```

### 2. Editing Results :satisfied:

Each image pair consists of the original image (Left) and the edited image (Right).

## Training Networks
### 1. Downloading Datasets
#### [Download](https://drive.google.com/open?id=1dOl4_yk2x-LTHwgKBykxHQpmqDvqlkab) datasets and extract the archives into `datasets` directory under repository root.
```
stefann/
├── ...
├── datasets/
│ ├── fannet/
│ │ ├── pairs/
│ │ ├── train/
│ │ └── valid/
│ └── colornet/
│ ├── test/
│ ├── train/
│ └── valid/
└── ...
```

#### :pushpin: Description of `datasets/fannet`

This dataset is used to train FANnet and it consists of 3 directories: fannet/pairs, fannet/train and fannet/valid. The directories fannet/train and fannet/valid consist of 1015 and 300 sub-directories respectively, each corresponding to one specific font. Each font directory contains 64x64 grayscale images of 62 English alphanumeric characters (10 numerals + 26 upper-case letters + 26 lower-case letters). The filename format is xx.jpg where xx is the ASCII value of the corresponding character (e.g. "48.jpg" implies an image of character "0"). The directory fannet/pairs contains 50 image pairs, each corresponding to a random font from fannet/valid. Each image pair is horizontally concatenated to a dimension of 128x64. The filename format is id_xx_yy.jpg where id is the image identifier, xx and yy are the ASCII values of source and target characters respectively (e.g. "00_65_66.jpg" implies a transformation from source character "A" to target character "B" for the image with identifier "00").

#### :pushpin: Description of `datasets/colornet`

This dataset is used to train Colornet and it consists of 3 directories: colornet/test, colornet/train and colornet/valid. Each directory consists of 5 sub-directories: _color_filters, _mask_pairs, input_color, input_mask and output_color. The directory _color_filters contains synthetically generated color filters of dimension 64x64 including both solid and gradient colors. The directory _mask_pairs contains a set of 64x64 grayscale image pairs selected at random from 1315 available fonts in datasets/fannet. Each image pair is horizontally concatenated to a dimension of 128x64. For colornet/train and colornet/valid each color filter is applied on each mask pair. This results in 64x64 image triplets of color source image, binary target image and color target image in input_color, input_mask and output_color directories respectively. For colornet/test one color filter is applied only on one mask pair to generate similar image triplets. With a fixed set of 100 mask pairs, 80000 colornet/train and 20000 colornet/valid samples are generated from 800 and 200 color filters respectively. With another set of 50 mask pairs, 50 colornet/test samples are generated from 50 color filters.

### 2. Training FANnet and Colornet
#### Step 1: Activate environment
To activate **CPU** environment: `conda activate stefann-cpu`

To activate **GPU** environment: `conda activate stefann-gpu`
#### Step 2: Change directory to project root
```bash
cd stefann
```
#### Step 3: Configure and train FANnet
To configure training options edit `configurations` section `(line 40-72)` of `fannet.py`

To start training: `python fannet.py`
###### :cloud: Check [this notebook](https://www.kaggle.com/prasunroy/starter-1-font-generation-stefann-cvpr-2020) hosted at Kaggle for an interactive demonstration of FANnet.
#### Step 4: Configure and train Colornet
To configure training options edit `configurations` section `(line 38-65)` of `colornet.py`

To start training: `python colornet.py`
###### :cloud: Check [this notebook](https://www.kaggle.com/prasunroy/starter-2-color-transfer-stefann-cvpr-2020) hosted at Kaggle for an interactive demonstration of Colornet.

## External Links

Project  •
Paper  •
Supplementary Materials  •
Datasets  •
Models  •
Sample Images

## Citation
```
@InProceedings{Roy_2020_CVPR,
title = {STEFANN: Scene Text Editor using Font Adaptive Neural Network},
author = {Roy, Prasun and Bhattacharya, Saumik and Ghosh, Subhankar and Pal, Umapada},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}
```

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
```

##### Made with :heart: and :pizza: on Earth.

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/prasunroy/stefann

Awesome Lists containing this project

README

Project  •
Paper  •
Supplementary Materials  •
Datasets  •
Models  •
Sample Images

https://github.com/prasunroy/stefann

Awesome Lists containing this project

README

Project • Paper • Supplementary Materials • Datasets • Models • Sample Images

Project •
Paper •
Supplementary Materials •
Datasets •
Models •
Sample Images