{"id":22967288,"url":"https://github.com/thohemp/6drepnet","last_synced_at":"2025-05-15T18:00:18.215Z","repository":{"id":37403506,"uuid":"461865969","full_name":"thohemp/6DRepNet","owner":"thohemp","description":"Official Pytorch implementation of  6DRepNet: 6D Rotation representation for unconstrained head pose estimation.","archived":false,"fork":false,"pushed_at":"2024-07-02T14:10:30.000Z","size":104,"stargazers_count":583,"open_issues_count":20,"forks_count":82,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-05-07T23:39:17.093Z","etag":null,"topics":["6d","aflw2000","analysis","biwi","estimation","facial","head","head-pose","head-pose-estimation","orientation","pose","pytorch","pytorch-implementation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thohemp.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-21T13:07:08.000Z","updated_at":"2025-05-07T09:02:26.000Z","dependencies_parsed_at":"2024-12-21T17:19:26.962Z","dependency_job_id":"129b2678-028e-4ce8-8cce-c6f5f014fcc7","html_url":"https://github.com/thohemp/6DRepNet","commit_stats":{"total_commits":61,"total_committers":8,"mean_commits":7.625,"dds":0.6229508196721312,"last_synced_commit":"6dd8cf297c258dc80795756f3f38e38f19770b69"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thohemp%2F6DRepNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thohemp%2F6DRepNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thohemp%2F6DRepNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thohemp%2F6DRepNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thohemp","download_url":"https://codeload.github.com/thohemp/6DRepNet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254394718,"owners_count":22063984,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["6d","aflw2000","analysis","biwi","estimation","facial","head","head-pose","head-pose-estimation","orientation","pose","pytorch","pytorch-implementation"],"created_at":"2024-12-14T21:12:31.557Z","updated_at":"2025-05-15T18:00:18.124Z","avatar_url":"https://github.com/thohemp.png","language":"Python","readme":"[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/6d-rotation-representation-for-unconstrained/head-pose-estimation-on-biwi)](https://paperswithcode.com/sota/head-pose-estimation-on-biwi?p=6d-rotation-representation-for-unconstrained)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/6d-rotation-representation-for-unconstrained/head-pose-estimation-on-aflw2000)](https://paperswithcode.com/sota/head-pose-estimation-on-aflw2000?p=6d-rotation-representation-for-unconstrained)\n[![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/osanseviero/6DRepNet)\n\n# \u003cdiv align=\"center\"\u003e **6D Rotation Representation for Unconstrained Head Pose Estimation (Pytorch)** \u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/thohemp/archive/blob/main/6DRepNet2.gif\" alt=\"animated\" /\u003e\n\u003c/p\u003e\n\n## **Citing**\n\nIf you find our work useful, please cite the paper:\n```BibTeX\n@ARTICLE{10477888,\n  author={Hempel, Thorsten and Abdelrahman, Ahmed A. and Al-Hamadi, Ayoub},\n  journal={IEEE Transactions on Image Processing}, \n  title={Toward Robust and Unconstrained Full Range of Rotation Head Pose Estimation}, \n  year={2024},\n  volume={33},\n  number={},\n  pages={2377-2387},\n  keywords={Head;Training;Predictive models;Pose estimation;Quaternions;Three-dimensional displays;Training data;Head pose estimation;full range of rotation;rotation matrix;6D representation;geodesic loss},\n  doi={10.1109/TIP.2024.3378180}}\n```\n```BibTeX\n@INPROCEEDINGS{9897219,\n  author={Hempel, Thorsten and Abdelrahman, Ahmed A. and Al-Hamadi, Ayoub},\n  booktitle={2022 IEEE International Conference on Image Processing (ICIP)}, \n  title={6d Rotation Representation For Unconstrained Head Pose Estimation}, \n  year={2022},\n  volume={},\n  number={},\n  pages={2496-2500},\n  doi={10.1109/ICIP46576.2022.9897219}}\n```\n## Updates\n### 18.09.2023\n* We present **6DRepNet360**! Checkout our **new version** of 6DRepNet tackling the prediction of the entire range of head pose orientations: https://github.com/thohemp/6DRepNet360\n### 13.09.2022\n* 6DRepNet is now avaiable as pip package for even more accessable usage: *pip3 install SixDRepNet*\n### 20.06.2022\n* 6DRepNet has been accepted to ICIP 2022.\n### 29.05.2022\n* Simplified training script\n* Updated default training configuration for more robust results\n\n## \u003cdiv align=\"center\"\u003e **Paper**\u003c/div\u003e\n\u003e Thorsten Hempel and Ahmed A. Abdelrahman and Ayoub Al-Hamadi, \"6D Rotation Representation for Unconstrained Head Pose Estimation\", *accepted to ICIP 2022*. [[ResearchGate]](https://www.researchgate.net/publication/358898627_6D_Rotation_Representation_For_Unconstrained_Head_Pose_Estimation)[[Arxiv]](https://arxiv.org/abs/2202.12555)\n\n\n### \u003cdiv align=\"center\"\u003e **Abstract**\u003c/div\u003e\n\u003e In this paper, we present a method for unconstrained end-to-end head pose estimation. We address the problem of ambiguous rotation labels by introducing the rotation matrix formalism for our ground truth data and propose a continuous 6D rotation matrix representation for efficient and robust direct regression. This way, our method can learn the full rotation appearance which is contrary to previous approaches that restrict the pose prediction to a narrow-angle for satisfactory results. In addition, we propose a geodesic distance-based loss to penalize our network with respect to the \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\textit{SO}(3)\"\u003e manifold geometry. Experiments on the public AFLW2000 and BIWI datasets demonstrate that our proposed method significantly outperforms other state-of-the-art methods by up to 20\\%.\n___\n\n\u003cdiv align=\"left\"\u003e\n\n## **Trained on 300W-LP, Test on AFLW2000 and BIWI**\n|                        |         |   ||             |          |          |          |          |          |          |\n| --------------------- | -------------- |--- | ------- | ------ | ------ | ------ | ------ | ------ | ------ | ------ |\n|                        | **Full Range** |        **Yaw**        |  **Pitch**   |   **Roll**   |   **MAE**    | |  **Yaw**    |  **Pitch**   |   **Roll**   |   **MAE**    |\n| HopeNet (\u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\alpha\"\u003e =2) |           N       |       6.47        |   6.56   |   5.44   |   6.16   ||   5.17   |   6.98   |   3.39   |   5.18   |\n| HopeNet  (\u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\alpha\"\u003e =1)|           N       |       6.92        |   6.64   |   5.67   |   6.41   ||   4.81   |   6.61   |   3.27   |   4.90   |\n| FSA-Net                |           N       |       4.50        |   6.08   |   4.64   |   5.07   ||   4.27   |   4.96   |   2.76   |   4.00   |\n| HPE                    |           N       |       4.80        |   6.18   |   4.87   |   5.28   ||   3.12   |   5.18   |   4.57   |   4.29   |\n| QuatNet                |          N        |       3.97        |   5.62   |   3.92   |   4.50   || **2.94** |   5.49   |   4.01   |   4.15   |\n| WHENet-V               |         N         |       4.44        |   5.75   |   4.31   |   4.83   ||   3.60   | **4.10** |   2.73   |   3.48   |\n| WHENet                 |         Y/N         |       5.11        |   6.24   |   4.92   |   5.42   ||   3.99   |   4.39   |   3.06   |   3.81   |\n| TriNet                 |         Y         |       4.04        |   5.77   |   4.20   |   4.67   ||   4.11   |   4.76   |   3.05   |   3.97   |\n| FDN                    |         N         |       3.78        |   5.61   |   3.88   |   4.42  | |   4.52   |   4.70   | **2.56** |   3.93   |\n|                        |                  |                   |          |          |          |          |          |          |          |\n| **6DRepNet**               |         Y        |     **3.63**      | **4.91** | **3.37** | \u003cins\u003e**3.97**\u003c/ins\u003e ||   3.24   |   4.48   |   2.68   |\u003cins\u003e **3.47**\u003c/ins\u003e |\n|                        |                  |                   |          |          |          |          |          |          |          | |\n\n\u003c/div\u003e\n\n\u003cdiv align=\"left\"\u003e\n\n## **BIWI 70/30**\n\n|                         |          |          |          |          |\n| :---------------------- | :------: | :------: | :------: | :------: |\n|                         |   **Yaw**    |  **Pitch**   |   **Roll**   |   **MAE**    |\n| HopeNet (\u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\alpha\"\u003e =1) |   3.29   |   3.39   |   3.00   |   3.23   |\n| FSA-Net                 |   2.89   |   4.29   |   3.60   |   3.60   |\n| TriNet                  |   2.93   |   3.04   |   2.44   |   2.80   |\n| FDN                     |   3.00   |   3.98   |   2.88   |   3.29   |\n|                         |          |          |          |          |\n| **6DRepNet**                | **2.69** | **2.92** | **2.36** | \u003cins\u003e**2.66** \u003c/ins\u003e|\n|                         |          |          |          |          |\n\u003c/div\u003e\n\n## **Fine-tuned Models**\n\nFine-tuned models can be download from here: https://drive.google.com/drive/folders/1V1pCV0BEW3mD-B9MogGrz_P91UhTtuE_?usp=sharing\n\n\n\n# \u003cdiv align=\"center\"\u003e **Quick Start**: \u003c/div\u003e\n## Pip install:\n\n```sh\npip3 install sixdrepnet\n```\nExample usage:\n```py\n# Import SixDRepNet\nfrom sixdrepnet import SixDRepNet\nimport cv2\n\n\n# Create model\n# Weights are automatically downloaded\nmodel = SixDRepNet()\n\nimg = cv2.imread('/path/to/image.jpg')\n\npitch, yaw, roll = model.predict(img)\n\nmodel.draw_axis(img, yaw, pitch, roll)\n\ncv2.imshow(\"test_window\", img)\ncv2.waitKey(0)\n\n```\n\n## Setting it up your own:\n\n\n```sh\ngit clone https://github.com/thohemp/6DRepNet\ncd 6DRepNet\n```\n### Set up a virtual environment:\n```sh\npython3 -m venv venv\nsource venv/bin/activate\npip install -r requirements.txt  # Install required packages\n```\nIn order to run the demo scripts you need to install the face detector\n\n```sh\npip install git+https://github.com/elliottzheng/face-detection.git@master\n```\n\n##  **Camera Demo**:\n\n```sh\npython ./sixdrepnet/demo.py  --snapshot 6DRepNet_300W_LP_AFLW2000.pth \\\n                             --cam 0\n```\n\n\n___\n\n# \u003cdiv align=\"center\"\u003e **Test/Train 3DRepNet** \u003c/div\u003e\n##  **Preparing datasets**\nDownload datasets:\n\n* **300W-LP**, **AFLW2000** from [here](http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/3DDFA/main.htm).\n\n* **BIWI** (Biwi Kinect Head Pose Database) from [here](https://icu.ee.ethz.ch/research/datsets.html)\n\nStore them in the *datasets* directory.\n\nFor 300W-LP and AFLW2000 we need to create a *filenamelist*.\n```\npython create_filename_list.py --root_dir datasets/300W_LP\n```\nThe BIWI datasets needs be preprocessed by a face detector to cut out the faces from the images. You can use the script provided [here](https://github.com/shamangary/FSA-Net/blob/master/data/TYY_create_db_biwi.py). For 7:3 splitting of the BIWI dataset you can use the equivalent script [here](https://github.com/shamangary/FSA-Net/blob/master/data/TYY_create_db_biwi_70_30.py). We set the cropped image size to *256*.\n\n\n\n## **Testing**:\n\n```sh\npython test.py  --batch_size 64 \\\n                --dataset AFLW2000 \\\n                --data_dir datasets/AFLW2000 \\\n                --filename_list datasets/AFLW2000/files.txt \\\n                --snapshot output/snapshots/1.pth \\\n                --show_viz False\n```\n\n## **Training**\n\nDownload pre-trained RepVGG model '**RepVGG-B1g2-train.pth**' from [here](https://drive.google.com/drive/folders/1Avome4KvNp0Lqh2QwhXO6L5URQjzCjUq) and save it in the root directory.\n\n```sh\npython sixdrepnet/train.py\n```\n\n\n##  **Deploy models**\n\nFor reparameterization the trained models into inference-models use the convert script.\n\n```\npython convert.py input-model.tar output-model.pth\n```\n\nInference-models are loaded with the flag ```deploy=True```.\n\n```python\nmodel = SixDRepNet(backbone_name='RepVGG-B1g2',\n                    backbone_file='',\n                    deploy=True,\n                    pretrained=False)\n```\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthohemp%2F6drepnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthohemp%2F6drepnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthohemp%2F6drepnet/lists"}