https://github.com/sshh12/inverse-style-gan
Looking up a generative latent vectors from (face) reference images.
https://github.com/sshh12/inverse-style-gan
deepfake face-recognition gan inverse-gan stylegan
Last synced: 7 months ago
JSON representation
Looking up a generative latent vectors from (face) reference images.
- Host: GitHub
- URL: https://github.com/sshh12/inverse-style-gan
- Owner: sshh12
- License: mit
- Created: 2019-09-05T00:09:34.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2019-10-25T17:08:15.000Z (almost 6 years ago)
- Last Synced: 2025-02-28T21:41:38.736Z (8 months ago)
- Topics: deepfake, face-recognition, gan, inverse-gan, stylegan
- Language: Python
- Homepage:
- Size: 18.6 KB
- Stars: 32
- Watchers: 6
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Inverse Style GAN
> Looking up a generative latent vectors from reference images.
### Usage
```shell
$ git clone https://github.com/NVlabs/stylegan.git
$ git clone https://github.com/sshh12/Inverse-Style-GAN.git
$ cd Inverse-Style-GAN
$ echo Install Tensorflow For GPUs
$ pip install -r requirements.txt
```
#### Gradient Descent Algo
```shell
$ python grad-lookup.py --img_path face.jpg --max_iter 10000 --lr 0.1 --keras_verbose 1
```
or resume training given `best.npy` with
```shell
$ python grad-lookup.py --img_path face.jpg --max_iter 10000 --input_init_vec best.npy
```
This will iteratively output `best.jpg`/`best.npy` which correspond to the best matching generated image and its latent vector.
##### How?
The script creates a model (and sort of a pipeline) that takes a latent vector, converts it to a face (StyleGan), and then finds the facial features of that face (FaceNet). Since GANs don't normally work backward, this script leverages the fact that both StyleGAN and VGGFace are differentiable to find the latent vector that would produce a given target face. When trained, the model freezes its StyleGAN and VGGFace weights so the only update on each iteration of gradient descent is the input latent vector. The model's loss is a function of the L2 difference between the target face's embeddings and the generated face's embeddings.
#### Brute Force Algo
```shell
$ python brute-lookup.py --img_path face.jpg --max_iter 100
```
This will also iteratively output `best.jpg`/`best.npy` which correspond to the best matching generated image and its latent vector.
### Related Research
* [A Style-Based Generator Architecture for Generative Adversarial Networks](https://arxiv.org/abs/1812.04948)
* [Inverting The Generator Of A Generative Adversarial Network](https://arxiv.org/abs/1802.05701)
* [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://arxiv.org/abs/1503.03832)