{"id":13553241,"url":"https://github.com/charliegerard/gaze-detection","last_synced_at":"2025-04-04T07:05:40.753Z","repository":{"id":37528466,"uuid":"334417379","full_name":"charliegerard/gaze-detection","owner":"charliegerard","description":"👀 Use machine learning in JavaScript to detect eye movements and build gaze-controlled experiences.","archived":false,"fork":false,"pushed_at":"2021-06-24T17:01:37.000Z","size":6631,"stargazers_count":579,"open_issues_count":4,"forks_count":40,"subscribers_count":19,"default_branch":"main","last_synced_at":"2024-04-14T04:35:30.215Z","etag":null,"topics":["creative-coding","eye-detection","eye-tracking","frontend","human-computer-interaction","javascript","machine-learning","tensorflow","tensorflowjs","tfjs"],"latest_commit_sha":null,"homepage":"https://gaze-keyboard.netlify.app/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/charliegerard.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":"charliegerard","ko_fi":"devdevcharlie","custom":["https://buymeacoffee.com/devdevcharlie"]}},"created_at":"2021-01-30T13:20:15.000Z","updated_at":"2024-04-13T14:38:50.000Z","dependencies_parsed_at":"2022-07-14T00:50:29.788Z","dependency_job_id":null,"html_url":"https://github.com/charliegerard/gaze-detection","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charliegerard%2Fgaze-detection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charliegerard%2Fgaze-detection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charliegerard%2Fgaze-detection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charliegerard%2Fgaze-detection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/charliegerard","download_url":"https://codeload.github.com/charliegerard/gaze-detection/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247135142,"owners_count":20889420,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["creative-coding","eye-detection","eye-tracking","frontend","human-computer-interaction","javascript","machine-learning","tensorflow","tensorflowjs","tfjs"],"created_at":"2024-08-01T12:02:20.500Z","updated_at":"2025-04-04T07:05:40.730Z","avatar_url":"https://github.com/charliegerard.png","language":"JavaScript","funding_links":["https://github.com/sponsors/charliegerard","https://ko-fi.com/devdevcharlie","https://buymeacoffee.com/devdevcharlie"],"categories":["JavaScript","frontend","Projects"],"sub_categories":[],"readme":"# Gaze-detection\n\nUse machine learning in JavaScript to detect eye movements and build gaze-controlled experiences!\n\n## Demo\n\nVisit [https://gaze-keyboard.netlify.app/](https://gaze-keyboard.netlify.app/) _(Works well on mobile too!!)_ 😃\n\n![](gaze-demo.gif)\n\n_Inspired by the Android application [\"Look to speak\"](https://play.google.com/store/apps/details?id=com.androidexperiments.looktospeak)._\n\nUses Tensorflow.js's [face landmark detection model](https://www.npmjs.com/package/@tensorflow-models/face-landmarks-detection).\n\n## Detection\n\nThis tool detects when the user looks right, left, up and straight forward.\n\n## How to use\n\n### Install\n\nAs a module:\n\n```bash\nnpm install gaze-detection --save\n```\n\n### Code sample\n\nStart by importing it:\n\n```js\nimport gaze from \"gaze-detection\";\n```\n\nLoad the machine learning model:\n\n```js\nawait gaze.loadModel();\n```\n\nThen, set up the camera feed needed for the detection. The `setUpCamera` method needs a `video` HTML element and, optionally, a camera device ID if you are using more than the default webcam.\n\n```js\nconst videoElement = document.querySelector(\"video\");\n\nconst init = async () =\u003e {\n  // Using the default webcam\n  await gaze.setUpCamera(videoElement);\n\n  // Or, using more camera input devices\n  const mediaDevices = await navigator.mediaDevices.enumerateDevices();\n  const camera = mediaDevices.find(\n    (device) =\u003e\n      device.kind === \"videoinput\" \u0026\u0026\n      device.label.includes(/* The label from the list of available devices*/)\n  );\n\n  await gaze.setUpCamera(videoElement, camera.deviceId);\n};\n```\n\nRun the predictions:\n\n```js\nconst predict = async () =\u003e {\n  const gazePrediction = await gaze.getGazePrediction();\n  console.log(\"Gaze direction: \", gazePrediction); //will return 'RIGHT', 'LEFT', 'STRAIGHT' or 'TOP'\n  if (gazePrediction === \"RIGHT\") {\n    // do something when the user looks to the right\n  }\n  let raf = requestAnimationFrame(predict);\n};\npredict();\n```\n\nStop the detection:\n\n```js\ncancelAnimationFrame(raf);\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharliegerard%2Fgaze-detection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcharliegerard%2Fgaze-detection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharliegerard%2Fgaze-detection/lists"}