{"id":22011502,"url":"https://github.com/seasonsh/probabilistic-face-embeddings","last_synced_at":"2025-04-09T08:09:06.550Z","repository":{"id":42197550,"uuid":"182166548","full_name":"seasonSH/Probabilistic-Face-Embeddings","owner":"seasonSH","description":"(ICCV 2019) Uncertainty-aware Face Representation and Recognition","archived":false,"fork":false,"pushed_at":"2019-08-08T03:59:38.000Z","size":37010,"stargazers_count":345,"open_issues_count":17,"forks_count":59,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-02T05:08:22.925Z","etag":null,"topics":["deep-learning","deep-neural-networks","face-recognition","tensorflow","uncertainty"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seasonSH.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-18T22:39:09.000Z","updated_at":"2025-04-02T01:39:42.000Z","dependencies_parsed_at":"2022-09-12T15:31:04.596Z","dependency_job_id":null,"html_url":"https://github.com/seasonSH/Probabilistic-Face-Embeddings","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seasonSH%2FProbabilistic-Face-Embeddings","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seasonSH%2FProbabilistic-Face-Embeddings/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seasonSH%2FProbabilistic-Face-Embeddings/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seasonSH%2FProbabilistic-Face-Embeddings/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seasonSH","download_url":"https://codeload.github.com/seasonSH/Probabilistic-Face-Embeddings/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247999860,"owners_count":21031046,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-neural-networks","face-recognition","tensorflow","uncertainty"],"created_at":"2024-11-30T02:17:10.441Z","updated_at":"2025-04-09T08:09:06.527Z","avatar_url":"https://github.com/seasonSH.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Probabilistic Face Embeddings\r\n\r\n\u003cimg src=\"https://raw.githubusercontent.com/seasonSH/Probabilistic-Face-Embeddings/master/assets/PFE.png\" width=\"600px\"\u003e\r\n\r\n#### News: Our paper has been accepted to ICCV 2019.\r\nThis is a demo code of training and testing [Probabilistic Face Embeddings](https://arxiv.org/abs/1904.09658) using Tensorflow. Probabilistic Face Embeddging (PFE) is a method that converts conventional CNN-based face embeddings into probabilistic embeddings by calibrating each feature value with an uncertainty value. The representation of each face will be an Guassian distribution parametrized by (mu, sigma), where mu is the original embedding and sigma is the learned uncertainty. Experiments show that PFE could\r\n+ improve the performance of face recognition models by taking uncertainty into account.\r\n+ give more insight into the models' understanding of faces and more controll over potential risks.\r\n\r\n## \u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/2/2d/Tensorflow_logo.svg/1000px-Tensorflow_logo.svg.png\" width=\"25\"/\u003e Compatability\r\nCurrently this repo is compatible with Python 3 and Tensorflow r1.9.\r\n\r\n## \u003cimg src=\"https://image.flaticon.com/icons/svg/149/149366.svg\" width=\"25\"/\u003e Update Notes\r\n+ 07/08/2019: Correct the CASIA file list.\r\n+ 30/04/2019: Upload more pre-trained models.\r\n+ 19/04/2019: Initial Upload.\r\n\r\n## \u003cimg src=\"https://image.flaticon.com/icons/svg/182/182321.svg\" width=\"25\"/\u003e Citation\r\n\r\n    @article{shi2019PFE,\r\n      title = {Probabilistic Face Embeddings},\r\n      author = {Shi, Yichun and Jain, Anil K.},\r\n      booktitle = {arXiv:1904.09658},\r\n      year = {2019}\r\n    }\r\n\r\n\r\n## \u003cimg src=\"https://image.flaticon.com/icons/svg/1/1383.svg\" width=\"25\"/\u003e Usage\r\n**Note:** In this section, we assume that you are always in the directory **`$PROJECT_ROOT/`**\r\n### Preprocessing\r\nIn this demo, we will use CASIA-WebFace, LFW and IJB-A as examples for training and testing PFEs. In this section, we will align these datasets with the landmarks I pre-extracted.\r\n1. Download the original images of CASIA-WebFace dataset and align the images with the following command:\r\n    ``` Shell\r\n    python align/align_dataset.py data/ldmark_casia_mtcnncaffe.txt \\\r\n    data/casia_mtcnncaffe_aligned \\\r\n    --prefix /path/to/CASIA-Webface/images \\\r\n    --transpose_input --image_size 96 112\r\n    ```\r\n    If you want to train the Ms-ArcFace model, you can download the dataset [here](https://github.com/deepinsight/insightface/wiki/Dataset-Zoo) and decode it using [this code](https://github.com/deepinsight/insightface/blob/master/recognition/data/rec2image.py).\r\n2. Download the original images of LFW dataset and align the images with the following command:\r\n    ``` Shell\r\n    python align/align_dataset.py data/ldmark_lfw_mtcnncaffe.txt \\\r\n    data/lfw_mtcnncaffe_aligned \\\r\n    --prefix /path/to/LFW/images \\\r\n    --transpose_input --image_size 96 112\r\n    ```\r\n3. Download the IJB-A dataset and crop the faces with the following command:\r\n    ``` Shell\r\n    python align/crop_ijba.py proto/IJB-A/metadata.csv \\\r\n    /path/to/IJB-A/images/ \\\r\n    data/ijba_cropped/\r\n    ```\r\n    To crop the images, you need to make sure that there are two folders under the given dataset folder: ```img``` and ```frame```. After cropping, align the images with the following command:\r\n    ``` Shell\r\n    python align/align_dataset.py data/ldmark_ijba_mtcnncaffe.txt \\\r\n    data/ijba_mtcnncaffe_aligned \\\r\n    --prefix data/ijba_cropped \\\r\n    --transpose_input --image_size 96 112\r\n    ```\r\n\r\n### Training\r\n1. Before training, you need to prepare a base embedding network. To use the example [base model](https://drive.google.com/open?id=1MiC_qCj5GFidWLtON9ekClOCJu6dPHT4), download zip file and unzip the files under ```pretrained/sphere64_caisa_am/```.\r\n\r\n2. The configuration files for training are saved under ```config/``` folder, where you can define the training data, pre-trained model, network definition and other hyper-parameters. \r\n3. The uncertainty module that we are going to train is in ```models/uncertainty_module.py```.\r\n4. Use the following command to run the default training configuration for the example base model:\r\n    ``` Shell\r\n    python train.py config/sphere64_casia.py\r\n    ```\r\n    The command will create an folder under ```log/sphere64_casia_am_PFE/```, which saves all the checkpoints and summaries. The model directory is named as the time you start training.\r\n\r\n### Testing\r\n+ **Single Image Comparison**\r\n    We use LFW dataset as an example for single image comparison. Make sure you have aligned LFW images using the previous commands. Then you can test it on the LFW dataset with the following command:\r\n    ```Shell\r\n    python evaluation/eval_lfw.py --model_dir /path/to/your/model/directory \\\r\n    --dataset_path data/lfw_mtcnncaffe_aligned\r\n    ```\r\n\r\n+ **Template Fusion and Comparison**\r\n    We use IJB-A dataset as an example for template face comparison. Make sure you have aligned IJB-A images using the previous commands. Then you can test it on the IJB-A dataset with the following command:\r\n    ```Shell\r\n    python evaluation/eval_ijb.py --model_dir /path/to/your/model/directory \\\r\n    --dataset_path data/ijba_mtcnncaffe_aligned\r\n    ```\r\n+ Note that in the original paper, I used Matlab to normalize the images, but this demo uses pure python implementation. So the performance could be slightly different. \r\n\r\n### Visualization of Uncertainty\r\nTODO\r\n\r\n\r\n## \u003cimg src=\"https://image.flaticon.com/icons/svg/48/48541.svg\" width=\"25\"/\u003e Pre-trained Model\r\n#### 64-CNN on CASIA-WebFace: \r\nBase Mode: [Google Drive](https://drive.google.com/open?id=1MiC_qCj5GFidWLtON9ekClOCJu6dPHT4)  \r\nPFE: [Google Drive](https://drive.google.com/open?id=1BeTUYnc__u1_cYEKoXqfGDQjdk2TChoD)\r\n\r\n#### 64-CNN on MS-ArcFace: \r\nBase Mode: [Google Drive](https://drive.google.com/open?id=15CMJ4vh2_fdX3M05CIJY7V2B0ydoSr2Q)  \r\nPFE: [Google Drive](https://drive.google.com/open?id=1R-sl5vaxWheyQBpgtONiSH5Qt4153Tl3)\r\n\r\nNote: In the paper we used a different version of Ms-Celeb-1M. According to the authors of ArcFace, this dataset (MS-ArcFace) has already been cleaned and has no overlap with the test data.\r\n\r\n#### Test Results: \r\n| Model | Method | LFW | IJB-A (FAR=0.1%) |\r\n| ----- | ------ |--- | ----- |\r\n| 64-CNN CASIA-WebFace | Baseline | 99.20 | 83.21 |\r\n| 64-CNN CASIA-WebFace | PFE | 99.47 | 87.53 |\r\n| 64-CNN Ms-ArcFace | Baseline | 99.72 | 91.93 |\r\n| 64-CNN Ms-ArcFace | PFE | 99.83 | 94.82 |\r\n\r\n(The PFE models and test results are obtained using exactly this demo code)\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseasonsh%2Fprobabilistic-face-embeddings","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseasonsh%2Fprobabilistic-face-embeddings","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseasonsh%2Fprobabilistic-face-embeddings/lists"}