{"id":15642844,"url":"https://github.com/pinto0309/dmhead","last_synced_at":"2025-04-30T11:14:55.197Z","repository":{"id":37264212,"uuid":"505243376","full_name":"PINTO0309/DMHead","owner":"PINTO0309","description":"Dual model head pose estimation. Fusion of SOTA models. 360° 6D HeadPose detection. All pre-processing and post-processing are fused together, allowing end-to-end processing in a single inference.","archived":false,"fork":false,"pushed_at":"2023-06-04T23:29:26.000Z","size":102,"stargazers_count":70,"open_issues_count":0,"forks_count":7,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-30T15:36:45.173Z","etag":null,"topics":["6d","cuda","head-pose-estimation","headpose-detection","headpose-estimation","models","onnx","tensorrt"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PINTO0309.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-19T23:47:27.000Z","updated_at":"2025-03-17T08:22:04.000Z","dependencies_parsed_at":"2024-10-23T06:02:44.346Z","dependency_job_id":null,"html_url":"https://github.com/PINTO0309/DMHead","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PINTO0309%2FDMHead","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PINTO0309%2FDMHead/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PINTO0309%2FDMHead/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PINTO0309%2FDMHead/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PINTO0309","download_url":"https://codeload.github.com/PINTO0309/DMHead/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249499832,"owners_count":21282312,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["6d","cuda","head-pose-estimation","headpose-detection","headpose-estimation","models","onnx","tensorrt"],"created_at":"2024-10-03T11:57:45.911Z","updated_at":"2025-04-18T13:32:09.599Z","avatar_url":"https://github.com/PINTO0309.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DMHead\nDual model head pose estimation. Fusion of SOTA models. 360° 6D HeadPose detection. All pre-processing and post-processing are fused together, allowing end-to-end processing in a single inference.\n\n## 1. Summary\n![icon_design drawio (14)](https://user-images.githubusercontent.com/33194443/175760025-b359e1d2-ac16-456e-8cf6-2c58514fbc7c.png)\n- **`[Front side]`** Wearing a mask mode - 6DRepNet (RepVGG-B1g2)\n\n  - Paper\n  \n    ![image](https://user-images.githubusercontent.com/33194443/175760351-bd8d2e61-bb49-48f3-8023-c45c12cbd800.png)\n\n  - Fine tune (My own training)\n    ```\n    Yaw: 3.3193, Pitch: 4.9063, Roll: 3.3687, MAE: 3.8648\n    ```\n\n- **`[Front side]`** Not wearing a mask mode - SynergyNet (MobileNetV2)\n\n  - Paper\n\n    ![image](https://user-images.githubusercontent.com/33194443/174690800-272e5a06-c932-414f-8397-861d7d6284d0.png)\n\n- **`[Rear side]`** WHENet\n\n  - Paper\n\n    ![image](https://user-images.githubusercontent.com/33194443/175760218-4e61da30-71b6-4d2a-8ca4-ddc4c2ec5df0.png)\n\n## 2. Inference Test\n\n```bash\nwget https://github.com/PINTO0309/DMHead/releases/download/1.1.2/yolov4_headdetection_480x640_post.onnx\nwget https://github.com/PINTO0309/DMHead/releases/download/1.1.2/dmhead_mask_Nx3x224x224.onnx\nwget https://github.com/PINTO0309/DMHead/releases/download/1.1.2/dmhead_nomask_Nx3x224x224.onnx\n\npython demo_video.py\n```\n\n```bash\npython demo_video.py \\\n[-h] \\\n[--device DEVICE] \\\n[--height_width HEIGHT_WIDTH] \\\n[--mask_or_nomask {mask,nomask}]\n\noptional arguments:\n  -h, --help\n    Show this help message and exit.\n\n  --device DEVICE\n    Path of the mp4 file or device number of the USB camera.\n    Default: 0\n\n  --height_width HEIGHT_WIDTH\n    {H}x{W}.\n    Default: 480x640\n\n  --mask_or_nomask {mask,nomask}\n    Select either a model that provides high accuracy when wearing a mask or\n    a model that provides high accuracy when not wearing a mask.\n    Default: mask\n```\n\n## 3. Atmosphere\n- August 15, 2022 - MAE: 3.8648\n\n  https://user-images.githubusercontent.com/33194443/184782685-52aa9fe3-d086-4104-8ea1-00c4a7418142.mp4\n\n  https://user-images.githubusercontent.com/33194443/184784102-089a82b9-765a-4431-bf33-43370b5c8174.mp4\n\n## 4. Benchmark\n- 6DRepNet\n- Official Paper FineTuned\n    ```\n    Yaw: 3.6266, Pitch: 4.9066, Roll: 3.3734, MAE: 3.9688\n    ```\n- Trained on 300W-LP (Custom, Mask-wearing face image augmentation)\n- Test on AFLW2000\n  - June 20, 2022\n    ```\n    Yaw: 3.6129, Pitch: 5.5801, Roll: 3.8468, MAE: 4.3466\n    ```\n  - July 3, 2022 `_epoch_321.pth`\n    ```\n    Yaw: 3.3346, Pitch: 5.0004, Roll: 3.5381, MAE: 3.9577\n    ```\n  - August 15, 2022\n    ```\n    Yaw: 3.3193, Pitch: 4.9063, Roll: 3.3687, MAE: 3.8648\n    ```\n\n## 5. Model Structure\n- INPUTS: `Float32 [N,3,224,224]`\n- OUTPUTS: `Float32 [N,3]`, `[Yaw,Roll,Pitch]`\n\n\u003cdetails\u003e\u003csummary\u003eClick to expand\u003c/summary\u003e\u003cdiv\u003e\n\n  ![pinheadpose_1x3x224x224 onnx](https://user-images.githubusercontent.com/33194443/174504855-bf03e294-c9c9-477d-9faf-07b3d0393463.png)\n\n\u003c/div\u003e\u003c/details\u003e\n  \n## 6. References\n1. https://github.com/choyingw/SynergyNet\n2. https://github.com/thohemp/6DRepNet\n3. https://github.com/Ascend-Research/HeadPoseEstimation-WHENet\n4. https://github.com/PINTO0309/Face_Mask_Augmentation\n5. https://github.com/PINTO0309/PINTO_model_zoo/tree/main/383_DirectMHP/post_process_gen_tools\n6. https://github.com/PINTO0309/PINTO_model_zoo/tree/main/383_DirectMHP\n\n## 7. Citation\n```\n@misc{https://doi.org/10.48550/arxiv.2005.10353,\n    doi = {10.48550/ARXIV.2005.10353},\n    url = {https://arxiv.org/abs/2005.10353},\n    author = {Zhou, Yijun and Gregson, James},\n    title = {WHENet: Real-time Fine-Grained Estimation for Wide Range Head Pose},\n    publisher = {arXiv},\n    year = {2020},\n}\n```\n```\n@misc{hempel20226d,\n    title={6D Rotation Representation For Unconstrained Head Pose Estimation},\n    author={Thorsten Hempel and Ahmed A. Abdelrahman and Ayoub Al-Hamadi},\n    year={2022},\n    eprint={2202.12555},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV}\n}\n```\n```\n@INPROCEEDINGS{wu2021synergy,\n  author={Wu, Cho-Ying and Xu, Qiangeng and Neumann, Ulrich},\n  booktitle={2021 International Conference on 3D Vision (3DV)},\n  title={Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry},\n  year={2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpinto0309%2Fdmhead","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpinto0309%2Fdmhead","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpinto0309%2Fdmhead/lists"}