{"id":17948946,"url":"https://github.com/hysts/pytorch_mpiigaze","last_synced_at":"2025-04-05T22:06:12.686Z","repository":{"id":43921655,"uuid":"117894502","full_name":"hysts/pytorch_mpiigaze","owner":"hysts","description":"An unofficial PyTorch implementation of MPIIGaze and MPIIFaceGaze","archived":false,"fork":false,"pushed_at":"2024-03-15T06:43:03.000Z","size":203,"stargazers_count":364,"open_issues_count":2,"forks_count":88,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-29T20:08:51.320Z","etag":null,"topics":["computer-vision","gaze","gaze-estimation","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hysts.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-17T21:24:47.000Z","updated_at":"2025-03-28T09:12:49.000Z","dependencies_parsed_at":"2025-02-11T09:29:30.971Z","dependency_job_id":"9216280e-49a3-469a-9b73-7b3330753058","html_url":"https://github.com/hysts/pytorch_mpiigaze","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hysts%2Fpytorch_mpiigaze","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hysts%2Fpytorch_mpiigaze/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hysts%2Fpytorch_mpiigaze/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hysts%2Fpytorch_mpiigaze/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hysts","download_url":"https://codeload.github.com/hysts/pytorch_mpiigaze/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247406087,"owners_count":20933803,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","gaze","gaze-estimation","pytorch"],"created_at":"2024-10-29T09:10:47.072Z","updated_at":"2025-04-05T22:06:12.642Z","avatar_url":"https://github.com/hysts.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# An unofficial PyTorch implementation of MPIIGaze and MPIIFaceGaze\n\n[![MIT License](https://img.shields.io/badge/license-MIT-green)](https://opensource.org/licenses/MIT)\n[![GitHub stars](https://img.shields.io/github/stars/hysts/pytorch_mpiigaze.svg?style=flat-square\u0026logo=github\u0026label=Stars\u0026logoColor=white)](https://github.com/hysts/pytorch_mpiigaze)\n\n[Here](https://github.com/hysts/pytorch_mpiigaze_demo) is a demo program.\nSee also [this repo](https://github.com/hysts/pl_gaze_estimation).\n\n## Requirements\n\n* Linux (Tested on Ubuntu only)\n* Python \u003e= 3.7\n\n```bash\npip install -r requirements.txt\n```\n\n\n## Download the dataset and preprocess it\n\n### MPIIGaze\n\n```bash\nbash scripts/download_mpiigaze_dataset.sh\npython tools/preprocess_mpiigaze.py --dataset datasets/MPIIGaze -o datasets/\n```\n\n### MPIIFaceGaze\n\n```bash\nbash scripts/download_mpiifacegaze_dataset.sh\npython tools/preprocess_mpiifacegaze.py --dataset datasets/MPIIFaceGaze_normalized -o datasets/\n```\n\n\n## Usage\n\nThis repository uses [YACS](https://github.com/rbgirshick/yacs) for\nconfiguration management.\nDefault parameters are specified in\n[`gaze_estimation/config/defaults.py`](gaze_estimation/config/defaults.py)\n(which is not supposed to be modified directly).\nYou can overwrite those default parameters using a YAML file like\n[`configs/mpiigaze/lenet_train.yaml`](configs/mpiigaze/lenet_train.yaml).\n\n\n### Training and Evaluation\n\nBy running the following code, you can train a model using all the\ndata except the person with ID 0, and run test on that person.\n\n```bash\npython train.py --config configs/mpiigaze/lenet_train.yaml\npython evaluate.py --config configs/mpiigaze/lenet_eval.yaml\n```\n\nUsing [`scripts/run_all_mpiigaze_lenet.sh`](scripts/run_all_mpiigaze_lenet.sh) and\n[`scripts/run_all_mpiigaze_resnet_preact.sh`](scripts/run_all_mpiigaze_resnet_preact.sh),\nyou can run all training and evaluation for LeNet and ResNet-8 with\ndefault parameters.\n\n\n## Results\n\n### MPIIGaze\n\n| Model           | Mean Test Angle Error [degree] | Training Time |\n|:----------------|:------------------------------:|--------------:|\n| LeNet           |              6.52              |  3.5 s/epoch  |\n| ResNet-preact-8 |              5.73              |   7 s/epoch   |\n\nThe training time is the value when using GTX 1080Ti.\n\n![](figures/mpiigaze/lenet.png)\n\n![](figures/mpiigaze/resnet_preact_8.png)\n\n### MPIIFaceGaze\n\n| Model     | Mean Test Angle Error [degree] | Training Time |\n|:----------|:------------------------------:|--------------:|\n| AlexNet   |              5.06              |  135 s/epoch  |\n| ResNet-14 |              4.83              |   62 s/epoch  |\n\nThe training time is the value when using GTX 1080Ti.\n\n![](figures/mpiifacegaze/alexnet.png)\n\n![](figures/mpiifacegaze/resnet_simple.png)\n\n\n\n\n### Demo\n\nThis demo program runs gaze estimation on the video from a webcam.\n\n1. Download the dlib pretrained model for landmark detection.\n\n    ```bash\n    bash scripts/download_dlib_model.sh\n    ```\n\n2. Calibrate the camera.\n\n    Save the calibration result in the same format as the sample\n    file [`data/calib/sample_params.yaml`](data/calib/sample_params.yaml).\n\n4. Run demo.\n\n    Specify the model path and the path of the camera calibration results\n    in the configuration file as in\n    [`configs/demo_mpiigaze_resnet.yaml`](configs/demo_mpiigaze_resnet.yaml).\n\n    ```bash\n    python demo.py --config configs/demo_mpiigaze_resnet.yaml\n    ```\n\n## Related repos\n\n- https://github.com/hysts/pl_gaze_estimation\n- https://github.com/hysts/pytorch_mpiigaze_demo\n\n## References\n\n* Zhang, Xucong, Yusuke Sugano, Mario Fritz, and Andreas Bulling. \"Appearance-based Gaze Estimation in the Wild.\" Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. [arXiv:1504.02863](https://arxiv.org/abs/1504.02863), [Project Page](https://www.mpi-inf.mpg.de/departments/computer-vision-and-multimodal-computing/research/gaze-based-human-computer-interaction/appearance-based-gaze-estimation-in-the-wild/)\n* Zhang, Xucong, Yusuke Sugano, Mario Fritz, and Andreas Bulling. \"It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation.\" Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW), 2017. [arXiv:1611.08860](https://arxiv.org/abs/1611.08860), [Project Page](https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/gaze-based-human-computer-interaction/its-written-all-over-your-face-full-face-appearance-based-gaze-estimation/)\n* Zhang, Xucong, Yusuke Sugano, Mario Fritz, and Andreas Bulling. \"MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation.\" IEEE transactions on pattern analysis and machine intelligence 41 (2017). [arXiv:1711.09017](https://arxiv.org/abs/1711.09017)\n* Zhang, Xucong, Yusuke Sugano, and Andreas Bulling. \"Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications.\" Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), 2019. [arXiv](https://arxiv.org/abs/1901.10906), [code](https://git.hcics.simtech.uni-stuttgart.de/public-projects/opengaze)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhysts%2Fpytorch_mpiigaze","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhysts%2Fpytorch_mpiigaze","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhysts%2Fpytorch_mpiigaze/lists"}