Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/socaity/face2face

Swap faces in images and videos. Create face embeddings. Enhance face image quality. Deploy as a web api.
https://github.com/socaity/face2face

artificial-intelligence celebrity deep-fake deep-learning face-detection face-enhancement face-recognition face-restoration face-swapping gan insightface roop video-processing webservice

Last synced: 7 days ago
JSON representation

Swap faces in images and videos. Create face embeddings. Enhance face image quality. Deploy as a web api.

Awesome Lists containing this project

README 

        
  
Face2Face





  



  
Instantly swap faces in images and videos





Face2Face is a generative AI technology to swap faces (aka Deep Fake) in images from one to another. 

For example, you can swap your face with Mona Lisa or your favorite celebrity.



With this repository you can:



- [Swap faces from one image to another](#swap-faces-from-one-image-to-another). 

- [Swap faces in images and videos](#face-swapping-with-face-embeddings).

- [Face embeddings](#face-swapping-with-face-embeddings): Create face embeddings. With these embeddings you can later swap faces just by using the name.

- [With face recognition](#face-swapping-with-face-embeddings): Swap faces with face recognition.

- [Face restoration](#face-enhancing): Enhance image quality of a portrait with a face enhancer model.

- Identify faces with face-recognition

- [Run face swapping as a service](docs/WebService.md).



This is a one shot face-swap model; for this reason only one face is needed to swap. It should work for all kinds of content, also for anime.

The face swapping model itself was created by [Insightface](https://github.com/deepinsight/insightface)



We provide the face swapping functionality as SDK and as a convenient [web (openAPI) API](docs/WebService.md) with [FastTaskAPI](https://github.com/SocAIty/FastTaskAPI).

The endpoint allows you to easily deploy face swapping, recognition and restoration as a service.



## Example swaps

| [Face-swap](#swap-faces-from-one-image-to-another)    | [Multi-face Swap](#swap-faces-from-one-image-to-another ) | 

|-------------------------------------------------------|-----------------------------------------------------------|

|  |   |



| [Video-swapping](#face-swapping-with-face-embeddings)                                                                 | [Video-Swapping with face-recognition](#swap-faces-in-videos)                                                                              | 

|---------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------|

|  |  |   



| [Face restoration](#face-enhancing)                        | [Face-Swap with face-recognition](#face-swapping-with-face-embeddings) |

|------------------------------------------------------------|------------------------------------------------------------------------------------|

|  |                          |



# Setup



### Install via pip

Depending on your use case you can install the package with or without the service.

```bash

# face2face without service (only for inference from script)

pip install socaity-face2face 

# full package with service

pip install socaity-face2face[full]

```

Additional dependencies:

- For VideoFile support in the webservice you also need to install [ffmpeg](https://ffmpeg.org/download.html) 



Requirements:

- Python 3.7 or higher

- Minimum 5GB of RAM (you'll get a "Killed" message without further information if you run out of memory)

- Recommended: GPU with at least 8GB of VRAM for fast-inference. Runs also on CPU though.



Note: Models are downloaded automatically



# Usage



We provide two ways to use the face swapping functionality.

1. [Direct module import and inference](#Inference-from-script) 

2. [By deploying and calling the web service](#docs/WebService.md)



# Inference from script

Use the Face2Face class to swap faces from one image to another.

First create an instance of the class.



```python

from face2face import Face2Face

f2f = Face2Face(device_id=0) 

```

With the device_id setting you can set the GPU device id. This also allows to run face2face in multiple processes on

different GPUs.



## Swap faces from one image to another

```python

swapped_img = f2f.swap_img_to_img("path/to/src.jpg", "path/to/target.jpg")

```

## Face swapping with face embeddings



Create a face embedding with the add_face function reuse those embeddings later.

```python

# create a face embedding and save it to disk

embedding = f2f.add_face("my_new_face", "path/to/my_portrait_image.jpg", save=True)

# Swap all faces in the image or video with the face(s) in the face embedding

swapped = f2f.swap(media="path/to/my_img_or_video.jpg", faces="my_new_face")

```

If argument save=true is set, the face embedding is persisted and the f2f.swap function can be used later with the same face_name, even after restarting the project.



## Face swapping with face recognition (swap pairs)



After an embedding was created, we can recognize / identify those persons.

Then the identified persons can specifically be swapped with defined swap pairs.

If the faces argument is provided as dict, the swap function recognizes and swaps the face-pairs correspondingly.



```python

# Swap faces with defined swap pairs

# This function will swap the faces of trump with hagrid and biden with ron.

# assumption the faces [trump, hagrid, biden, ron] are already added with f2f.add_face

swapped = f2f.swap(

  media="path/to/my_img_or_video.mp4", 

  faces={

      "trump": "hagrid",

      "biden": "ron"

  }

)

```



## Face swapping with a generator

Iteratively swapping from a list of images



```python

def my_image_generator():

  for i in range(100):

    yield cv2.imread(f"image_{i}.jpg")



# for swapping to always the same face

for swapped_img in f2f.swap_to_face_generator(faces="my_embedding", target_img_generator=my_image_generator()):

  cv2.imshow("swapped", swapped_img)

  cv2.waitKey(1)



# including face recognition

for swapped_img in f2f.swap_pairs_generator(target_img_generator=my_image_generator(), swap_pairs={"trump": "hagrid"}):

  cv2.imshow("swapped", swapped_img)

  cv2.waitKey(1)



```



## Face enhancing



The roop (inswapper) model operates on low resolution - what can harm the result face quality. 

However, there exist AI models, which can enhance the face quality by upscaling the image.

We provide different models for face enhancement: [gfpgan_1.4](https://github.com/TencentARC/GFPGAN), 

and the [gpen](https://github.com/yangxy/GPEN) family.

Check model_definitions.py for the available models.

You can upscale up to 2048 with the GPEN model --> higher quality + higher runtime.

```python

swapped_img = f2f.swap(media="path/to/my_img_or_video.mp4", enhance_face_model='gpen_bfr_512')

```

The corresponding model is automatically downloaded and used when enhance_faces is set to True.



### Face-enhancing without face-swapping



Alternatively you can enhance faces directly without applying a swap. 

```python

# enhance all faces in the image

enhanced_img = f2f.enhance_faces(image="path/to/my_img.jpg", enhance_face_model='gpen_bfr_512')

# enhance a specific face.

target_img = cv2.imread("path/to/my_img.jpg")

detected_face = f2f.detect_faces(target_img)[0]  # In this case we simply take the first one

enhanced_img = f2f.enhance_single_face(target_img, detected_face, enhance_face_model='gpen_bfr_512')

```



# Disclaimer



The author is not responsible for any misuse of the repository. Face swapping is a powerful technology that can be used for good and bad purposes.

Please use it responsibly and do not harm others. Do not publish any images without the consent of the people in the images.

The credits for face swapping technology go to the great Insightface Team thank you [insightface.ai](https://insightface.ai/). 

This project uses their pretrained models and parts of their code. Special thanks goes to their work around [ROOP](https://github.com/s0md3v/sd-webui-roop).

The author does not claim authorship for this repository. The authors contribution was to provide a convenient API and service around the face swapping.

A big thank you also goes to the authors of [GPEN](https://github.com/yangxy/GPEN) and [GFPGAN](https://github.com/TencentARC/GFPGAN),

who developed the models for face restoration.



# Contribute



Any help with maintaining and extending the package is welcome. Feel free to open an issue or a pull request.



ToDo:

- Improve face swap quality

  - Implement strength factor for applied face

- Improve inference times

  - by implementing batching.

  - by using multi-threading in image_generators

- remove insightface dependency and update onnx version

- streaming for the webserver