Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/justusschock/shapenet
PyTorch implementation of "Super-Realtime Facial Landmark Detection and Shape Fitting by Deep Regression of Shape Model Parameters" predicting facial landmarks with up to 400 FPS
https://github.com/justusschock/shapenet
deep-learning delira facial-keypoints facial-keypoints-cnn facial-landmarks landmark-detection pytorch shape shape-models shape-prediction shape-predictor
Last synced: about 2 months ago
JSON representation
PyTorch implementation of "Super-Realtime Facial Landmark Detection and Shape Fitting by Deep Regression of Shape Model Parameters" predicting facial landmarks with up to 400 FPS
- Host: GitHub
- URL: https://github.com/justusschock/shapenet
- Owner: justusschock
- License: agpl-3.0
- Created: 2019-01-31T15:08:38.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2020-05-24T20:28:09.000Z (over 4 years ago)
- Last Synced: 2024-06-09T01:36:24.221Z (3 months ago)
- Topics: deep-learning, delira, facial-keypoints, facial-keypoints-cnn, facial-landmarks, landmark-detection, pytorch, shape, shape-models, shape-prediction, shape-predictor
- Language: Python
- Homepage: https://shapenet.rtfd.io
- Size: 146 KB
- Stars: 340
- Watchers: 14
- Forks: 62
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# shapenet
[](https://travis-ci.com/justusschock/shapenet) [](https://shapenet.readthedocs.io/en/latest/?badge=latest) [](https://badge.fury.io/py/shapenet) [](https://codecov.io/gh/justusschock/shapenet) 
This repository contains the [PyTorch](https://pytorch.org) implementation of [our Paper "SUPER-REALTIME FACIAL LANDMARK DETECTION AND SHAPE FITTING BY DEEP REGRESSION OF SHAPE MODEL PARAMETERS"](#our-paper).
## Contents
* [Installation](#installation)
* [Demo](#demo)
* [Usage](#usage)
* [By Scripts](#by-scripts)
* [From Python](#from-python)
* [Pretrained Weights](#pretrained-weights)
* [Our Paper](#our-paper)
## Installation
### From Binary:
`pip install shapenet`
### From Source:
`pip install git+https://github.com/justusschock/shapenet`
## Demo
Demonstration Videos comparing our method to [`dlib`](https://dlib.net) can be found [here as overlay](https://drive.google.com/open?id=1hLaNuWy8eC3xs9qcTRzVZfBjdlB7xZ8c) and [here as side-by-side view](https://drive.google.com/file/d/128ZnFSOAhKnhN7xpgi6FR5KircnwIxca/view?usp=sharing)
## Usage
### By Scripts
For simplicity we provide several scripts to preprocess the data, train networks, predict from networks and export the network via [`torch.jit`](https://pytorch.org/docs/stable/jit.html).
To get a list of the necessary and accepted arguments, run the script with the `-h` flag.
#### Data Preprocessing
* `prepare_all_data`: prepares multiple datasets (you can select the datasets to preprocess via arguments passed to this script)
* `prepare_cat_dset`: Download and preprocesses the [Cat-Dataset](https://www.kaggle.com/crawford/cat-dataset)
* `prepare_helen_dset`: Preprocesses an already downloaded ZIP file of the [HELEN Dataset](http://www.ifp.illinois.edu/~vuongle2/helen/) (Download is recommended from [here](https://ibug.doc.ic.ac.uk/download/annotations/helen.zip) since this already contains the landmarks)
* `prepare_lfpw_dset`: Preprocesses an already downloaded ZIP file of the [LFPW Dataset](https://neerajkumar.org/databases/lfpw/) (Download is recommended from [here](https://ibug.doc.ic.ac.uk/download/annotations/lfpw.zip) since this already contains the landmarks)
#### Training
* `train_shapenet`: Trains the shapenet with the configuration specified in an extra configuration file (exemplaric configuration for all available datasets are provided in the [example_configs](example_configs) folder)
#### Prediction
* `predict_from_net`: Predicts all images in a given directory (assumes existing groundtruths for cropping, otherwise the cropping to groundtruth could be replaced by a detector)
#### JIT-Export
* `export_to_jit`: Traces the given model and saves it as jit-ScriptModule, which can be accessed via Python and C++
### From Python
This implementation uses the [`delira`-Framework](https://github.com/justusschock/delira) for training and validation handling. It supports mixed precision training and inference via [NVIDIA/APEX](https://github.com/NVIDIA/apex) (must be installed separately). The data-handling is outsourced to [shapedata](https://github.com/justusschock/shapedata).
The following gives a short overview about the packages and classes.
#### `shapenet.networks`
The `networks` subpackage contains the actual implementation of the shapenet with bindings to integrate the `ShapeLayer` and other feature extractors (currently the ones registered in `torchvision.models`).
#### `shapenet.layer`
The `layer` subpackage contains the Python and C++ Implementations of the ShapeLayer and the Affine Transformations. It is supposed to use these Layers as layers in `shapenet.networks`
#### `shapenet.jit`
The `jit` subpackage is a less flexible reimplementation of the subpackages `shapenet.networks` and `shapenet.layer` to export trained weights as jit-ScriptModule
#### `shapenet.utils`
The `utils` subpackage contains everything that did not suit into the scope of any other package. Currently it is mainly responsible for parsing of configuration files.
#### `shapenet.scripts`
The `scripts` subpackage contains all scipts described in [Scripts](#by-scripts) and their helper functions.
### Pretrained Weights
Currently Pretrained Weights are available for [grayscale faces](https://drive.google.com/file/d/1QS2GUZK9xKWvpbDYgUCc-m0qI60TMnLj/view?usp=sharing) and [cats](https://drive.google.com/file/d/13S-4vLmmUBNy2XKJl_yR1u7Z283Iu1zB/view?usp=sharing).
For these Networks the image size is fixed to 224 and the pretrained weights can be loaded via `torch.jit.load("PATH/TO/NETWORK/FILE.ptj")`. The inputs have to be of type `torch.Tensor` with dtype `torch.float` in shape `(BATCH_SIZE, 1, 224, 224)` and normalized in a range between (0, 1).
## Our Paper
If you use our Code for your own research, please cite our paper:
```
@article{Kopaczka2019,
title = {Super-Realtime Facial Landmark Detection and Shape Fitting by Deep Regression of Shape Model Parameters},
author = {Marcin Kopaczka and Justus Schock and Dorit Merhof},
year = {2019},
journal = {arXiV preprint}
}
```
The Paper is available as [PDF on arXiv](https://arxiv.org/abs/1902.03459).