{"id":47958634,"url":"https://github.com/david-wb/gaze-estimation","last_synced_at":"2026-04-04T09:38:28.627Z","repository":{"id":41064951,"uuid":"239582924","full_name":"david-wb/gaze-estimation","owner":"david-wb","description":"A deep learning based gaze estimation framework implemented with PyTorch","archived":false,"fork":false,"pushed_at":"2020-02-26T19:42:47.000Z","size":2387,"stargazers_count":171,"open_issues_count":1,"forks_count":32,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-30T07:01:57.594Z","etag":null,"topics":["artificial-intelligence","artificial-neural-networks","cnn","computer-vision","deep-learning","eye-tracking","gaze-estimation","machine-learning","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/david-wb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-10T18:28:36.000Z","updated_at":"2025-03-26T12:29:45.000Z","dependencies_parsed_at":"2022-07-16T05:30:37.234Z","dependency_job_id":null,"html_url":"https://github.com/david-wb/gaze-estimation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/david-wb/gaze-estimation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/david-wb%2Fgaze-estimation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/david-wb%2Fgaze-estimation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/david-wb%2Fgaze-estimation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/david-wb%2Fgaze-estimation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/david-wb","download_url":"https://codeload.github.com/david-wb/gaze-estimation/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/david-wb%2Fgaze-estimation/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31395344,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T09:13:02.600Z","status":"ssl_error","status_checked_at":"2026-04-04T09:13:01.683Z","response_time":60,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","artificial-neural-networks","cnn","computer-vision","deep-learning","eye-tracking","gaze-estimation","machine-learning","pytorch"],"created_at":"2026-04-04T09:38:28.528Z","updated_at":"2026-04-04T09:38:28.617Z","avatar_url":"https://github.com/david-wb.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gaze Estimation with Deep Learning\n\nThis project implements a deep learning model to predict eye region landmarks and gaze direction.\nThe model is trained on a set of computer generated eye images synthesized with UnityEyes [1]. This work is heavily based on [2] but with some key modifications. \nThis model achieves ~14% mean angular error on the MPIIGaze evaluation set after training on UnityEyes alone.\n\n### Setup\n\nNOTE: This repo has been tested only on Ubuntu 16.04 and MacOS. \n\nFirst, create a conda env for your system and activate it:\n```bash\nconda env create -f env-linux.yml\nconda activate ge-linux\n```\n\nThen download the pretrained model files. One is for detecting face landmarks. The other is the main pytorch model.\n\n```bash\n./scripts/fetch_models.sh\n```\n\nFinally, run the webcam demo. You will likely need a GPU and have cuda 10.1 installed in order to get acceptable performance. \n\n```bash\npython run_with_webcam.py\n```\n\nIf you'd like to train the model yourself, please see the readme under `datasets/UnityEyes`.\n\n### Materials and Methods\n\nOver 100k training images were generated using UnityEyes [1]. These images are each labeled\n with a json metadata file. The labels provide eye region landmark positions in screenspace,\n  the direction the eye is looking in camera space, and other pieces of information. A rectangular region around the eye was extracted from each raw traing image and normalized to have a width equal to the eye width (1.5 times the distance between eye corners). \n  For each preprocessed image, a set of heatmaps corresponding\n  to 34 eye region landmarks was created. The model was trained to regress directly on the landmark locations and gaze direction in (pitch, yaw) form. The model was implemented in pytorch. The overall method is summarized in the following figure.\n![alt text](static/fig1.png \"Logo Title Text 1\")\n\nThe model architecture is based on the stacked hourglass model [3]. The main modification was to add a separate pre-hourglass layer for predicting the gaze direction. The output of the additional layer is concatenated with the predicted eye-region landmarks before being passed to two fully connected layers. This way, the model can make use of the high-level landmark features for predicting the gaze direction. \n\n### Demo Video\n\n[![Watch the video](static/ge_screenshot.png)](https://drive.google.com/open?id=1WUUmd4quXq_YA5ANWDoUxqFGgguE_QJi)\n\n\n### References\n\n1. https://www.cl.cam.ac.uk/research/rainbow/projects/unityeyes/\n2. https://github.com/swook/GazeML\n3. https://github.com/princeton-vl/pytorch_stacked_hourglass\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavid-wb%2Fgaze-estimation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavid-wb%2Fgaze-estimation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavid-wb%2Fgaze-estimation/lists"}