https://github.com/strvcom/strv-ml-mask2face
Virtually remove a face mask to see what a person looks like underneath
https://github.com/strvcom/strv-ml-mask2face
autoencoder computer-vision convolutional-neural-network deep-learning face-mask inpainting machine-learning python resnet tensorflow2 unet
Last synced: about 2 months ago
JSON representation
Virtually remove a face mask to see what a person looks like underneath
- Host: GitHub
- URL: https://github.com/strvcom/strv-ml-mask2face
- Owner: strvcom
- License: mit
- Created: 2020-09-01T15:32:26.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2021-05-15T13:28:36.000Z (about 4 years ago)
- Last Synced: 2025-03-23T20:22:23.522Z (3 months ago)
- Topics: autoencoder, computer-vision, convolutional-neural-network, deep-learning, face-mask, inpainting, machine-learning, python, resnet, tensorflow2, unet
- Language: Jupyter Notebook
- Homepage:
- Size: 24.4 MB
- Stars: 64
- Watchers: 6
- Forks: 21
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Mask2face
Can you virtually remove a face mask to see what a person looks like underneath? Our Machine Learning team proves it’s possible via an image inpainting-based ML solution. [Here](https://www.strv.com/blog/mask2face-how-we-built-ai-that-shows-face-beneath-mask-engineering) is exactly how our engineers approached the problem — from the preconditions to the implementation, results and future improvements.
**Check out [the article](https://www.strv.com/blog/mask2face-how-we-built-ai-that-shows-face-beneath-mask-engineering) for a more in-depth explanation**
**Examples of results** (input | expected output | actual output)
## How to run this project
Results can be replicated by following those steps:
### How to setup ENV
- If your **system does not have an Nvidia CUDA device available**, please comment `tensorflow-gpu==2.2.0` in the _environment.yml_ file.
- If you are running MacOS, change `tensorflow==2.2.0` to `tensorflow==2.0.0` in the _environment.yml_ file.
- Use [Conda ENV Manager](https://towardsdatascience.com/a-guide-to-conda-environments-bc6180fc533) to create new ENV: `conda env create -f environment.yml`
- Activate the ENV: `conda activate mask2face`
### Get Data
- Download [Labeled Faces in the Wild data](http://vis-www.cs.umass.edu/lfw/lfw-deepfunneled.tgz) and unzip its content into _data_ folder or use `mask2face.ipynb` that will download it automatically.
- You can get better results using larger dataset or dataset with higher quality images. For example [CelebA dataset](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html) contains over 200 000 high quality images.
### Configuration
You can configure the project using the `configuration.json`. Some of the items are set up and should not be changed. However, changing some of the following items can be useful.
- `input_images_path`: Path to dataset with images that are input for the DataGenerator. If you want to use a different dataset than the default one, set the path to it here.
- `train_data_path`: Path where are training images generated and where training algorithm is looking for training data.
- `test_data_path`: Path where are testing images generated.
- `train_image_count`: Number of training image pairs generated by DataGenerator.
- `test_image_count`: Number of testing image pairs generated by DataGenerator.
- `train_data_limit`: Number of training image pairs used for model training.
- `test_data_limit`: Number of testing image pairs used for model testing.
### Train the model
- Run Jupyter server in the ENV: `jupyter notebook`
- Follow instructions in the _mask2face_ notebook
### Known Issues
- There might be a memory leak when training the model
- Performance on real-world data might not be optimal - follow our [tips](https://www.strv.com/blog/mask2face-how-we-built-ai-that-shows-face-beneath-mask-engineering#ideas-for-additional-improvements) to get possible results on real-world data
- **If you encounter any other issues, please, let us know; we are happy to help**
## Final Notes
If you’re considering our help, you may be interested in our other past work—like the [custom AI solution we built for Cinnamon in just four months](https://www.strv.com/blog/strv-cinnamon-building-custom-ai-solutions-in-4-months). And if you’re a fellow engineer, please feel free to reach out to us with any questions or share your work results. We’re always happy to start up a discussion.