{"id":14036572,"url":"https://github.com/iacopomasi/face_specific_augm","last_synced_at":"2025-07-27T03:32:29.163Z","repository":{"id":83945318,"uuid":"70083409","full_name":"iacopomasi/face_specific_augm","owner":"iacopomasi","description":"Face Renderer to perform Domain (Face) Specific Data Augmentation ","archived":false,"fork":false,"pushed_at":"2019-04-22T05:11:56.000Z","size":41771,"stargazers_count":417,"open_issues_count":1,"forks_count":124,"subscribers_count":26,"default_branch":"master","last_synced_at":"2025-01-06T17:30:31.881Z","etag":null,"topics":["3d-shapes","face-recognition","face-renderer","frontalization","landmark","python"],"latest_commit_sha":null,"homepage":"http://www.openu.ac.il/home/hassner/projects/augmented_faces/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iacopomasi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-10-05T17:36:33.000Z","updated_at":"2025-01-06T16:57:22.000Z","dependencies_parsed_at":"2023-03-01T07:00:33.897Z","dependency_job_id":null,"html_url":"https://github.com/iacopomasi/face_specific_augm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/iacopomasi/face_specific_augm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iacopomasi%2Fface_specific_augm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iacopomasi%2Fface_specific_augm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iacopomasi%2Fface_specific_augm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iacopomasi%2Fface_specific_augm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iacopomasi","download_url":"https://codeload.github.com/iacopomasi/face_specific_augm/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iacopomasi%2Fface_specific_augm/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267294180,"owners_count":24065343,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-27T02:00:11.917Z","response_time":82,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-shapes","face-recognition","face-renderer","frontalization","landmark","python"],"created_at":"2024-08-12T03:02:03.408Z","updated_at":"2025-07-27T03:32:24.149Z","avatar_url":"https://github.com/iacopomasi.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"Face Renderer\n===========\n\nThis page contains an improved version of the face renderer from the paper _I. Masi\\*, A. Tran\\*, T. Hassner\\*, J. Leksut, G. Medioni, \"[Do We Really Need to Collect Million of Faces for Effective Face Recognition?](http://www.openu.ac.il/home/hassner/projects/augmented_faces/Masietal2016really.pdf) \", in Proc. of ECCV 2016 [1]_.\n\nThis release is part of an on-going face recognition project [4]. Please, check [this project page](http://www.openu.ac.il/home/hassner/projects/augmented_faces/) for updates and more data.\n\n**New!** We released an [end-to-end pipeline](https://github.com/fengju514/Face-Pose-Net) with Python code and deep models for direct 6DoF, 3D head pose estimation and face rendering (e.g., _frontalization_). The new code removes the need to run external facial landmark detection methods for alignment. Instead, it uses our extremely fast and robust, deep face pose estimator, described in [this paper](https://arxiv.org/abs/1708.07517) [5].\n\n __New!__ The code now supports adding novel views if you want for other yaw angles or even new title angles. You have to modify the code to incorporate the logic of using the novel views.\nFor creating the new precomputed rendered views, [please see this page](render_mat_demo)\n\n\n![Teaser](images/collect2.png)\n\n\nThe code has been ported and extended from the re-implementation provided by [Douglas Souza](https://github.com/dougsouza/face-frontalization) of the MATLAB frontalization code [3] \n\n## Features\n* **Highly customizable** through configuration file.\n* **Highly portable** few dependencies.\n* Render a face **with the head and background**.\n* The code uses pre-computed 3D projection. It is therefore **as fast as interpolating a single image!**\n* The code currently supports rendering of **multiple poses** {0°, 40°, 75°} and multiple **3D shapes** [1...10]\n\n## Dependencies\n\n* [Dlib Python Wrapper](http://dlib.net/)\n* [OpenCV Python Wrapper](http://opencv.org/)\n* [SciPy](http://www.scipy.org/install.html)\n* [Matplotlib](http://matplotlib.org/)\n* [Numpy](http://www.numpy.org/)\n* [Scikit-Learn](http://scikit-learn.org/)\n* [Python2.7](https://www.python.org/download/releases/2.7/)\n\nThe code has been tested on Linux only. On Linux you can rely on the default version of python, installing all the packages needed from the package manager or on Anaconda Python and install required packages through `conda`. \n\n**Importantly:** OpenGL or other 3D rendering libraries are **not** required to run this code.\n\n## Usage\n\n### Run it\n\nThe renderer can be used from the command line in the following, different ways.\n\nTo run it directly on a single image (software will try to detected landmarks using the DLIB facial landmark detector):\n\n```bash\n$ python demo.py \u003cimage-path\u003e\n```\nTo run it on a single image with provided landmarks (landmarks are assumed to correspond to the 68 detected by DLIB):\n```bash\n$ python demo.py \u003cimage-path\u003e \u003clandmark-path\u003e\n```\n\nThe code can be executed even in batch on a list of files as follows:\n```bash\n$ python demo.py --batch \u003cfile-list-path\u003e\n```\nwhere `\u003cfile-list-path\u003e` is a csv file where each line contains the following:\n\n`\u003csubj_key\u003e,\u003cimage-path\u003e,\u003clandmark-path\u003e` (lines that contain # are skipped)\n\nExample:\n\u003cpre\u003e\n#key,img-path,land-path\niacopo_1,img/iacopo1.jpg,landmarks/iacopo1.pts\niacopo_2,img/iacopo2.jpg,landmarks/iacopo2.pts\niacopo_3,img/iacopo3.jpg,landmarks/iacopo3.pts\n\u003c/pre\u003e\n\nGiven an image_key `\u003csubj_key\u003e` the code will create a folder with the subject name `subj` into its `output` folder. \nSo the `image_key` must be something along the lines of `subject_instance`. If not, the code will use the entire key as a folder name.\n\nYou can run our demo with one of the following:\n\n```bash\n$ python demo.py input/input_1.jpg\n$ python demo.py input/input_1.jpg input/input_1.pts\n$ python demo.py --batch input/input.list\n```\nThe result is saved in the provided `output` folder.\n\n### Customize it\nThe renderer reads the configuration file that is specified by a `config.ini` file.\n\nThe following options are available:\n```ini\n[general]\n## Activate this to make rendered images suitable for ResNet-101\n## for Face Recognition available here www.openu.ac.il/home/hassner/projects/augmented_faces/\n## Note: This will automatically disable some of the other options specified here\n## Moreover, you may have to code yourself in-plane alignment which is not provided here\nresnetON = no\n\n## Activate plotting\nplotON = no\n\n## Resize image to be fed into the CNN\nresizeCNN = yes\n\n## ConvNet imag size (used with resizeCNN)\ncnnSize = 160\n\n## Activate saving of rendered images\nsaveON = yes\n\n## Number of total subjects for the 3D models \nnTotSub = 10\n\n[renderer]\n## Activate rendering of the background\nbackground = yes \n\n## Activate soft-symmetry\nsymmetry = yes\n\n## Parameters to get a bit far from the face when sampling the background\nscaleFaceX = 0.5\n\n## Activate Near View Rendering [2]\nnearView = yes\n\n## (used if symmetry is applied)\n[symmetry]\n\n## If we want to flip the background in symmetry or no\nflipBackground = no\n```\n\nA few explanations on some not-so-obvious options:\n* `resnetON` if activated, set some parameters to produce rendered images that best fit [ResNet101 for Face Recognition](http://www.openu.ac.il/home/hassner/projects/augmented_faces/). Note you have still to code in-plane alignment by yourself. You can use the produced images with `resnetON=yes` as reference coordinate systems to do in-plane alignment.\n* `nTotSub` controls how many generic 3D shapes should be used to render the faces. If 10 is specified, all ten 3D generic shapes provide with this distribution are used. If 1, only the first generic shape is used, if 2, the first two are used, and so forth. Note that you need to edit the code if you want to render e.g. _only_ subject 7.\n* `background` if yes the code will try to render the full background (head+background). Otherwise only the face region of the head is renderer and the background is left black.\n* `scaleFaceX` this parameter control how much you want to sample the background when you render a profile faces and the background falls outside of the image. Basically it controls the distance of the projected points on the X-axis to the face part; If you change this param, you can control this behavior.\n* `nearView` if activated, render a faces with a similar strategy of [2]: avoid frontalizing the faces which are near profile. Otherwise render faces to all poses {0,40,75}.\n* `flipBackground` Determines if the background is flipped or not when applying symmetry (you will filp the head and the background as well).\n\n## Sample Results\n![Input](images/input.jpg) ![Pose_0_mulitShape](images/pose1.gif) ![Pose_40_mulitShape](images/pose2.gif) ![Pose_75_mulitShape](images/pose3.gif)\n\n\u003csub\u003eRespectively: Input image; then frontalization (0) with multi-shapes; render to (40) with multi-shapes; render to (75) with multi-shapes.\u003c/sub\u003e\n\n## Current Limitations\nThe renderer currently assumes reasonable landmark detector responses. It will fail if landmarks are not accurately localized. This is partly mitigated by rendering faces to a local pose and exploiting only one side of the face, similarly to [2].\n\n## Citation\n\nPlease cite our paper with the following bibtex if you use our face renderer:\n\n``` latex\n@inproceedings{masi16dowe,\n      title={Do {W}e {R}eally {N}eed to {C}ollect {M}illions of {F}aces \n      for {E}ffective {F}ace {R}ecognition?},\n      booktitle = {European Conference on Computer Vision},\n      author={Iacopo Masi \n      and Anh Tran \n      and Tal Hassner \n      and Jatuporn Toy Leksut \n      and G\\'{e}rard Medioni},\n      year={2016},\n    }\n```\n\n## References\n\n[1] I. Masi\\*, A. Tran\\*, T. Hassner\\*, J. Leksut, G. Medioni, \"Do We Really Need to Collect Million of Faces for Effective Face Recognition? \", ECCV 2016, \n    \\* denotes equal authorship\n\n[2] I. Masi, S. Rawls, G. Medioni, P. Natarajan \"Pose-Aware Face Recognition in the Wild\", CVPR 2016\n\n[3] T. Hassner, S. Harel, E. Paz and R. Enbar \"Effective Face Frontalization in Unconstrained Images\", CVPR 2015\n\n[4] Brendan F. Klare, Ben Klein, Emma Taborsky, Austin Blanton, Jordan Cheney, Kristen Allen, Patrick Grother, Alan Mah, Anil K. Jain, \"Pushing the Frontiers of Unconstrained Face Detection and Recognition: IARPA Janus Benchmark A\", CVPR 2015\n\n[5] F. Chan, A. Tran, T. Hassner, I. Masi, R. Nevatia, G. Medioni, \"FacePoseNet: Making a Case for Landmark-Free Face Alignment,\" ICCVw, 2017\n\n## Changelog\n- September 2016, First  Release \n\n## Disclaimer\n\n_The SOFTWARE PACKAGE provided in this page is provided \"as is\", without any guarantee made as to its suitability or fitness for any particular use. It may contain bugs, so use of this tool is at your own risk. We take no responsibility for any damage of any sort that may unintentionally be caused through its use._\n\n## Contacts\n\nIf you have any questions, drop an email to _iacopo.masi@usc.edu_ and _hassner@isi.edu_ or leave a message below with GitHub (log-in is needed).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiacopomasi%2Fface_specific_augm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiacopomasi%2Fface_specific_augm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiacopomasi%2Fface_specific_augm/lists"}