{"id":15176716,"url":"https://github.com/modanesh/gofacerec","last_synced_at":"2025-10-26T13:31:25.171Z","repository":{"id":173026348,"uuid":"607130424","full_name":"modanesh/GoFaceRec","owner":"modanesh","description":"Face recognition in Go using MTCNN and QMagFace","archived":false,"fork":false,"pushed_at":"2023-06-12T07:50:28.000Z","size":385,"stargazers_count":30,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-11T04:41:44.468Z","etag":null,"topics":["face-detection","face-recognition","golang","tensorflow","tfgo"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/modanesh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-27T11:30:16.000Z","updated_at":"2024-09-04T14:49:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"eff0be96-fa26-4872-954b-21e1e611d69d","html_url":"https://github.com/modanesh/GoFaceRec","commit_stats":{"total_commits":50,"total_committers":1,"mean_commits":50.0,"dds":0.0,"last_synced_commit":"76d8ca0954c480f7c23a5a21e311e6ce249921c9"},"previous_names":["modanesh/gofacerec"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modanesh%2FGoFaceRec","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modanesh%2FGoFaceRec/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modanesh%2FGoFaceRec/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modanesh%2FGoFaceRec/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/modanesh","download_url":"https://codeload.github.com/modanesh/GoFaceRec/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219862885,"owners_count":16555951,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["face-detection","face-recognition","golang","tensorflow","tfgo"],"created_at":"2024-09-27T13:40:41.771Z","updated_at":"2025-10-26T13:31:19.724Z","avatar_url":"https://github.com/modanesh.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"## GoFaceRec\n\nThis repository uses [tfgo](https://github.com/galeone/tfgo) to perform face recognition on an image from file. After \na lot of efforts, I came to the conclusion that if one wants to load a deep learning model in PyTorch or Jax in Go, they better think twice before committing a good amount of effort into it. Instead, they are better to first convert their models to TensorFlow and then work with tfgo.\n\nIn this repo, the input image is first processed, and then its embeddings are compared against the ones already computed from our dataset. In order to compute and save embeddings from an arbitrary dataset, one can use the [QMagFace's repo](https://github.com/pterhoer/QMagFace). Once the embeddings are ready, this repo uses Go in order to do face recognition. If the distance between embeddings falls bellow a specific threshold, then the face is considered as unknown. Otherwise, the proper label will be printed.   \n\n### Requirements\n\nThis project is tested using `Go 1.17` on Ubuntu 20.04. Except for `tfgo`, latest version of other packages have been used and installed.\n\nFor `gocv`, the version of `OpenCV` installed is `4.7`. And for `tfgo`, I installed [this version](https://github.com/galeone/tfgo) instead of the official one.\n\n\n### Installing\nJust run the following command in you project in order to install this package:\n```shell\ngo get github.com/modanesh/GoFaceRec@v0.1.1\n```\n\n### Converting models\nThere are many ways to convert a non-TF model to a TF one. For that purpose, I used ONNX as an intermediary to convert \nthe [QMagFace's model](https://github.com/pterhoer/QMagFace) from PyTorch to TF. \n\nUse the [model_converter.py](model_converter.py) script to convert the PyTorch model to ONNX first, and then the ONNX\nmodel to TF. \n\nSome of the code in the [model_converter.py](model_converter.py) is taken from the official [QMagFace's implementation](https://github.com/pterhoer/QMagFace).\n\nFor this project, you may download the `MTCNN` and `MagFace` tensorflow models from the following URL:\n\n| Model   | URL            |\n|---------|----------------|\n| MTCNN   | [Google Drive](https://drive.google.com/drive/folders/1_H_2qAdUfyKT8yyP9cxn0m20mMZe_yjg?usp=sharing)|\n| MagFace | [Google Drive](https://drive.google.com/drive/folders/136zEa3-6ve31HFse2sVgTi-eWkiLtfpk?usp=sharing)|\n\n### Extracting layers\n\nIn order to run the model using tfgo, you should know the input and output layers' names. In order to extract such \ninformation, the `saved_model_cli` command could be useful. A model exported with `tf.saved_model.save()` automatically\ncomes with the \"serve\" tag because the SavedModel file format is designed for serving. This tag contains the various \nfunctions exported. Among these, there is always present the \"serving_default\" `signature_def`. This signature def\nworks exactly like the TF 1.x graph. Get the input tensor and the output tensor, and use them as placeholder to feed \nand output to get, respectively. \n\nTo get info inside a SavedModel the best tool is `saved_model_cli` that comes with the TensorFlow Python package, for\nexample:\n```\nsaved_model_cli show --all --dir output/keras\ngives, among the others, this info:\n\nsignature_def['serving_default']:\nThe given SavedModel SignatureDef contains the following input(s):\n  inputs['inputs_input'] tensor_info:\n      dtype: DT_FLOAT\n      shape: (-1, 28, 28, 1)\n      name: serving_default_inputs_input:0\nThe given SavedModel SignatureDef contains the following output(s):\n  outputs['logits'] tensor_info:\n      dtype: DT_FLOAT\n      shape: (-1, 10)\n      name: StatefulPartitionedCall:0\nMethod name is: tensorflow/serving/predict\n```\n\nKnowing the input and output layers' names, `serving_default_inputs_input:0` and `StatefulPartitionedCall:0`, is \nessential to run the model in tfgo.\n\n\n### Running the model\n\nThis project uses MTCNN for face detection and QMagFace for face recognition. For MTCNN, three stages (PNet, RNet, ONet) have been used in a close fashion similar to [FaceNet](https://github.com/davidsandberg/facenet). Each stage is done in its corresponding function:\n- First stage (PNet): [`totalBoxes := firstStage(scales, img, pnetModel)`](https://github.com/modanesh/GoFaceRec/blob/main/main.go?plain=1#L1658)\n- Second stage (RNet): [`squaredBoxes := secondStage(totalBoxes, width, height, img, rnetModel)`](https://github.com/modanesh/GoFaceRec/blob/main/main.go?plain=1#L1667)\n- Third stage (ONet): [`thirdPickedBoxes, pickedPoints := thirdStage(squaredBoxes, width, height, img, onetModel)`](https://github.com/modanesh/GoFaceRec/blob/main/main.go?plain=1#L1676)\n\nYou may download the models from the available [Google Drive URLs](#converting-models).\n\nAfter the face detection stage, there is a face alignment. The function to perform face alignment is [`pImgs := alignFace(thirdPickedBoxes, pickedPoints, img)`](https://github.com/modanesh/GoFaceRec/blob/main/main.go?plain=1#L1685) which imitates the steps from [here](https://github.com/pterhoer/QMagFace/blob/main/preprocessing/insightface/src/face_preprocess.py#L195).\n\nFinally, once the face is detected and aligned, the recognition phase can start. It happens at this line: [`recognizeFace(pImgs, qmfModel, regEmbeddings, bSize, regFiles)`](https://github.com/modanesh/GoFaceRec/blob/main/main.go?plain=1#L1694).\n\nUse the bellow command to run the code:\n```shell\ngo run main.go IMAGE.jpg path/to/REGISTERED_IMAGES path/to/EMBEDDINGS.npy path/to/MTCNN_MODELS_DIR path/to/MAGFACE_MODEL_DIR \n```\nwhere:\n- `IMAGE.jpg`: path to the given image\n- `path/to/REGISTERED_IMAGES`: directory containing register images\n- `path/to/EMBEDDINGS.npy`: the embeddings extracted from the register images using the [Python's QMagFace implementation](https://github.com/pterhoer/QMagFace)\n- `path/to/MTCNN_MODELS_DIR`: directory containing tensorflow models for MTCNN\n- `path/to/MAGFACE_MODEL_DIR`: directory containing tensorflow model for MagFace\n\n### Challenges\n\nThe main challenge thus far was the conversion between [`gocv.Mat`](https://github.com/hybridgroup/gocv), [`tfgo.Tensor`](https://github.com/galeone/tfgo), [`gonum`](https://github.com/gonum/gonum/), and Go's native slice. The conversion is required as some matrix transformations are only available in `gocv` and some in `tfgo`. Also, the input to the `tfgo` model should be of type `tfgo.Tensor`, so inevitably one needs to convert the image read by `gocv` to `tfgo`. Also, some matrix operations are not available in any of these packages, so I had to implement them myself from scratch. To do so, I had to use Go's native slice. So inevitable conversions between these types are frequent throughout the code.\n\nFor example, the function [`adjustInput()`](https://github.com/modanesh/GoFaceRec/blob/main/main.go?plain=1#L502) besides doing some scaling, it also converts a `gocv.Mat` to Go's `[][][][]float32`. In addition, at this line: [`inputBufTensor, _ := tf.NewTensor(inputBuf)`](https://github.com/modanesh/GoFaceRec/blob/main/main.go?plain=1#L402) a `[][][][]float32` slice is converted to a `tfgo.Tensor`.\n\nIn contrast, these type conversions are done pretty easy and fast in Python.\n\n### ToDo\n- [X] Check why recognition model takes so long for a forward pass. In Python, it takes about 0.5 milliseconds while in Go it takes about 5500 milliseconds. For the first run, in Go, the session instantiation takes a long time. For next runs, Go runs pretty fast. [Take a look at this issue](https://github.com/galeone/tfgo/issues/4). The [`fakeRun()`](https://github.com/modanesh/GoFaceRec/blob/main/main.go?plain=1#L1582) function is for that purpose.\n- [X] Upload the models\n- [ ] Create a Go package\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodanesh%2Fgofacerec","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmodanesh%2Fgofacerec","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodanesh%2Fgofacerec/lists"}