{"id":13578511,"url":"https://github.com/YuvalNirkin/face_segmentation","last_synced_at":"2025-04-05T19:33:11.420Z","repository":{"id":19950978,"uuid":"88372066","full_name":"YuvalNirkin/face_segmentation","owner":"YuvalNirkin","description":"Deep face segmentation in extremely hard conditions","archived":false,"fork":false,"pushed_at":"2023-05-09T10:28:57.000Z","size":86,"stargazers_count":728,"open_issues_count":9,"forks_count":151,"subscribers_count":44,"default_branch":"master","last_synced_at":"2024-11-05T16:46:00.745Z","etag":null,"topics":["face","segmentation"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/YuvalNirkin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-04-15T20:20:22.000Z","updated_at":"2024-11-05T13:52:25.000Z","dependencies_parsed_at":"2024-01-16T20:26:08.009Z","dependency_job_id":"a9f386af-27e6-4110-b810-83532691e516","html_url":"https://github.com/YuvalNirkin/face_segmentation","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuvalNirkin%2Fface_segmentation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuvalNirkin%2Fface_segmentation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuvalNirkin%2Fface_segmentation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuvalNirkin%2Fface_segmentation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/YuvalNirkin","download_url":"https://codeload.github.com/YuvalNirkin/face_segmentation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393013,"owners_count":20931802,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["face","segmentation"],"created_at":"2024-08-01T15:01:31.247Z","updated_at":"2025-04-05T19:33:11.066Z","avatar_url":"https://github.com/YuvalNirkin.png","language":"C++","funding_links":[],"categories":["\u003ca name=\"cpp\"\u003e\u003c/a\u003eC++"],"sub_categories":[],"readme":"# Deep face segmentation in extremely hard conditions\n![alt text](https://yuvalnirkin.github.io/assets/img/projects/face_segmentation/face_segmentation_teaser.jpg \"Samples\")  \nCOFW sample images segmented using our method.\n\n[Yuval Nirkin](http://www.nirkin.com/), [Iacopo Masi](http://www-bcf.usc.edu/~iacopoma/), [Anh Tuan Tran](https://sites.google.com/site/anhttranusc/), [Tal Hassner](http://www.openu.ac.il/home/hassner/), and [Gerard Medioni](http://iris.usc.edu/people/medioni/index.html).\n\n## News (10/07/18)\n- [New FCN model](https://github.com/YuvalNirkin/face_segmentation/releases/download/1.1/face_seg_fcn8s_300_no_aug.zip) released for lower resolution images (300X300), trained without augmentations. Useful if you have limited GPU memory.\n- A better performing and more efficient U-Net model will be released soon, including training and inference scripts using PyTorch.  \n\n## Overview\nThis project provides an interface for face segmentation using Caffe with a fully convolutional neural network.\nThe network was trained on IARPA Janus CS2 dataset (excluding subjects that are also in [LFW](http://vis-www.cs.umass.edu/lfw/)) using a novel process for collecting ground truth face segmentations, involving our tool for [semi-supervised Face video segmentation](https://github.com/YuvalNirkin/face_video_segment). Additional synthetic images were generated by augmenting hands from the [EgoHands dataset](http://vision.soic.indiana.edu/projects/egohands/), and augmenting 3D models of glasses and microphones.\n\nIf you find this code useful, please make sure to cite our paper in your work:\n\nYuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, Gerard Medioni, \"[On Face Segmentation, Face Swapping, and Face Perception](https://arxiv.org/abs/1704.06729)\", IEEE Conference on Automatic Face and Gesture Recognition (FG), Xi'an, China on May 2018\n\nPlease see [project page](http://www.openu.ac.il/home/hassner/projects/faceswap/) for more details, more resources and updates on this project.\n\n## Dependencies\n| Library                                                            | Minimum Version | Notes                                    |\n|--------------------------------------------------------------------|-----------------|------------------------------------------|\n| [Boost](http://www.boost.org/)                                     | 1.47            |Optional - For command line tools         |\n| [OpenCV](http://opencv.org/)                                       | 3.0             |                                          |\n| [Caffe](https://github.com/BVLC/caffe)                             | 1.0             |☕️                                        |\n\n## Installation\n- Use CMake and your favorite compiler to build and install the library.\n- Download the [face_seg_fcn8s.zip](https://github.com/YuvalNirkin/face_segmentation/releases/download/1.0/face_seg_fcn8s.zip) or [face_seg_fcn8s_300_no_aug.zip](https://github.com/YuvalNirkin/face_segmentation/releases/download/1.1/face_seg_fcn8s_300_no_aug.zip) and extract to \"data\" in the installation directory.\n- Add \"bin\" in the installation directory to path.\n\n## Usage\n- For using the library's C++ interface, please take a look at the [Doxygen generated documentation](https://yuvalnirkin.github.io/docs/face_segmentation/).\n- For python go to \"interfaces/python\" in the installation directory and run:\n```BASH\npython face_seg.py\n```\n- For running the segmentation on a single image:\n```BASH\ncd path/to/face_segmentation/bin\nface_seg_image ../data/images/Alison_Lohman_0001.jpg -o . -m ../data/face_seg_fcn8s.caffemodel -d ../data/face_seg_fcn8s_deploy.prototxt\n```\n- For running the segmentation on all the images in a directory:\n```BASH\ncd path/to/face_segmentation/bin\nface_seg_batch ../data/images -o . -m ../data/face_seg_fcn8s.caffemodel -d ../data/face_seg_fcn8s_deploy.prototxt\n```\n- For running the segmentation on a list of images, first prepare a file \"img_list.txt\", in which each line is a path to an image and call the following command:\n```BASH\ncd path/to/face_segmentation/bin\nface_seg_batch img_list.txt -o . -m ../data/face_seg_fcn8s.caffemodel -d ../data/face_seg_fcn8s_deploy.prototxt\n```\n\nNote: The segmentation model was trained by cropping the training images using [find_face_landmarks](https://github.com/YuvalNirkin/find_face_landmarks). For best results crop the input images the same way, with crop resolution below 350 X 350. A Matlab function is available [here](https://github.com/YuvalNirkin/find_face_landmarks/blob/master/interfaces/matlab/bbox_from_landmarks.m).\n\n## Important note\nIn our paper we used a different network for our face segmentation. In the process of converting it to the Caffe model used in our [end-to-end face swap distribution](https://github.com/YuvalNirkin/face_swap) we notices some performance drop. We are working to fix this. We therefore ask that you please check here soon for updated on this Caffe model. \n\n## Citation\n\nPlease cite our paper with the following bibtex if you use our face segmentation network:\n\n``` latex\n@inproceedings{nirkin2018_faceswap,\n      title={On Face Segmentation, Face Swapping, and Face Perception},\n      booktitle = {IEEE Conference on Automatic Face and Gesture Recognition},\n      author={Nirkin, Yuval and Masi, Iacopo and Tran, Anh Tuan and Hassner, Tal and Medioni, and G\\'{e}rard Medioni},\n      year={2018},\n    }\n```\n\n## Related projects\n- [End-to-end, automatic face swapping pipeline](https://github.com/YuvalNirkin/face_swap), example application using out face segmentation method.\n- [Interactive system for fast face segmentation ground truth labeling](https://github.com/YuvalNirkin/face_video_segment), used to produce the training set for our deep face segmentation.\n- [CNN3DMM](http://www.openu.ac.il/home/hassner/projects/CNN3DMM/), estimation of 3D face shapes from single images.\n- [ResFace101](http://www.openu.ac.il/home/hassner/projects/augmented_faces/), deep face recognition used in the paper to test face swapping capabilities. \n\n## Copyright\nCopyright 2017, Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, and Gerard Medioni \n\nThe SOFTWARE provided in this page is provided \"as is\", without any guarantee made as to its suitability or fitness for any particular use. It may contain bugs, so use of this tool is at your own risk. We take no responsibility for any damage of any sort that may unintentionally be caused through its use.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYuvalNirkin%2Fface_segmentation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FYuvalNirkin%2Fface_segmentation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYuvalNirkin%2Fface_segmentation/lists"}