{"id":26446592,"url":"https://github.com/itskashfur/eye_blink_detection_system","last_synced_at":"2025-08-02T14:07:23.134Z","repository":{"id":274183235,"uuid":"922163992","full_name":"itskashfur/Eye_Blink_Detection_System","owner":"itskashfur","description":null,"archived":false,"fork":false,"pushed_at":"2025-04-03T14:20:01.000Z","size":15440,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T15:28:27.120Z","etag":null,"topics":["dlib","jupyter","opencv","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/itskashfur.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-25T13:46:39.000Z","updated_at":"2025-04-03T14:20:04.000Z","dependencies_parsed_at":"2025-02-12T13:43:58.403Z","dependency_job_id":"9a2fba42-5683-427f-b99e-4e69c134bc53","html_url":"https://github.com/itskashfur/Eye_Blink_Detection_System","commit_stats":null,"previous_names":["itskashfur/eye-blink-detection-system","itskashfur/eye_blink_detection_system"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/itskashfur/Eye_Blink_Detection_System","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itskashfur%2FEye_Blink_Detection_System","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itskashfur%2FEye_Blink_Detection_System/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itskashfur%2FEye_Blink_Detection_System/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itskashfur%2FEye_Blink_Detection_System/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/itskashfur","download_url":"https://codeload.github.com/itskashfur/Eye_Blink_Detection_System/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itskashfur%2FEye_Blink_Detection_System/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268401594,"owners_count":24244464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-02T02:00:12.353Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dlib","jupyter","opencv","python"],"created_at":"2025-03-18T12:19:18.248Z","updated_at":"2025-08-02T14:07:23.124Z","avatar_url":"https://github.com/itskashfur.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Real-Time Eye Blink Detection using Facial Landmarks\n \n* Abstract.\n  A real-time algorithm to detect eye blinks in a video sequence from a standard camera is proposed. Recent landmark detectors, trained on in-the wild datasets exhibit excellent robustness against ahead orientation with respect to a camera, varying illumination and facial expressions. We show that the landmarks are detected precisely enough to reliably estimate the level of the eye opening. The proposed algorithm therefore estimates the landmark positions, extracts a single scalar quantity– eye aspect ratio (EAR)– characterizing the eye opening in each frame. Finally, an SVM classifier detects eye blinks as a pattern of EAR values in a short temporal window. The simple algorithm outperforms the state-of-the-art results on two standard datasets\n  \n![495123_1_En_77_Fig2_HTML](https://github.com/user-attachments/assets/9a57141c-50d3-4611-90a4-ae7d0ab7e1b8)\nFigure 1: Open and closed eyes with landmarks pi automatically detected by [1]. The eye aspect ratio EAR in Eq. (1) plotted for several frames of a video sequence. A single blink is present.\n\n1. Introduction\n   -----------\n Detecting eye blinks is important for instance in systems that monitor a human operator vigilance,e.g. driver drowsiness [5, 13], in systems that warn a computer user staring at the screen without blink\ning for a long time to prevent the dry eye and the computer vision syndromes [17, 7, 8], in human computer interfaces that ease communication for disabled people [15], or for anti-spoofing protection in face recognition systems [11].Existing methods are either active or passive. Active methods are reliable but use special hardware,often expensive and intrusive, e.g. infrared cameras and illuminators [2], wearable devices, glasses with a special close-up cameras observing the eyes [10].While the passive systems rely on a standard remote camera only.\n  Many methods have been proposed to automatically detect eye blinks in a video sequence. Several methods are based on a motion estimation in the eye region. Typically, the face and eyes are detected by a Viola-Jones type detector. Next, motion in the eye area is estimated from optical flow, by sparse tracking [7, 8],\nor by frame-to-frame intensity differencing and adaptive thresholding. Finally, a decision is made whether the eyes are or are not covered by eyelids [9, 15]. A different approach is to infer the state of the eye opening from a single image, as e.g. by correlation matching with open and closed eye tem plates [4], a heuristic horizontal or vertical image intensity projection over the eye region [5, 6], a para metric model fitting to find the eyelids [18], or active shape models [14].\n  A major drawback of the previous approaches is that they usually implicitly impose too strong requirements on the setup, in the sense of a relative face-camera pose (head orientation), image resolution, illumination, motion dynamics,etc. Especially the heuristic methods that use raw image intensity are likely to be very sensitive despite their real-time performance.\n  \n  However nowadays, robust real-time facial landmark detectors that capture most of the characteristic points on a human face image, including eye corners and eyelids, are available, see Fig. 1. Mostof the state-of-the-art landmark detectors formulate a regression problem, where a mapping from an image into landmark positions [16] or into other landmark parametrization [1] is learned. These modern landmark detectors are trained on “in-the-wild datasets” and they are thus robust to varying illumination, various facial expressions, and moderatenon-frontal head rotations. An average error of the landmark localization of a state-of-the-art detector is usually below five percent of the inter-ocular distance. Recent methods run even significantly super real-time [12].\n  Therefore, we propose a simple but efficient algorithm to detect eye blinks by using a recent facial landmark detector. A single scalar quantity that reflects a level of the eye opening is derived from the landmarks. Finally, having a per-frame sequence of the eye opening estimates, the eye blinks are found by an SVM classifier that is trained on examples of blinking and non-blinking patterns.\n  Facial segmentation model presented in [14] is similar to the proposed method. However, their system is based on active shape models with reported processing time of about 5 seconds per frame for the segmentation, and the eye opening signal is normalized by statistics estimated by observing a longer sequence. The system is thus usable for offline processing only. The proposed algorithm runs real-time, since the extra costs of the eye opening from land marks and the linear SVM are negligible.\n \n The contributions of the paper are:\n   1. Ability of two state-of-the-art landmark detectors [1, 16] to reliably distinguish between the open and closed eye states is quantitatively demonstrated on a challenging in-the wild dataset and for various face image resolutions.\n   2. Anovel real-time eye blink detection algorithm which integrates a landmark detector and a classifier is proposed. The evaluation is done on two standard datasets [11, 8] achieving state-of-the art results.\n  The rest of the paper is structured as follows: The algorithm is detailed in Sec. 2, experimental validation and evaluation is presented in Sec. 3. Finally,Sec. 4 concludes the paper.\n ![2-Figure2-1](https://github.com/user-attachments/assets/77504133-e1bd-4ebe-bca9-695eb1965fc1)\nFigure 2: Example of detected blinks. The plots of the eye aspect ratio EAR in Eq. (1), results of the EAR thresholding (threshold set to 0.2), the blinks detected by EAR SVM and the ground-truth labels over the video sequence. Input image with detected landmarks (depicted frame is marked by a red line).\n \n2. Proposed method\n   --------------\n  The eye blink is a fast closing and reopening of a human eye. Each individual has a little bit different pattern of blinks. The pattern differs in the speed of closing and opening, a degree of squeezing the eye and in a blink duration. The eye blink lasts approximately 100-400 ms.\n  We propose to exploit state-of-the-art facial landmark detectors to localize the eyes and eyelid contours. From the landmarks detected in the image,we derive the eye aspect ratio (EAR) that is used as an estimate of the eye opening state. Since the per frame EAR may not necessarily recognize the eye blinks correctly, a classifier that takes a larger temporal window of a frame into account is trained.\n 2.1. Description of features\n For every video frame, the eye landmarks are detected. The eye aspect ratio (EAR) between height and width of the eye is computed.\n \n where,p1,....,p6 are the 2D landmark locations, de-picted in Fig. 1.\n The EAR is mostly constant when an eye is open and is getting close to zero while closing an eye. It is partially person and head pose insensitive. Aspect ratio of the open eye has asmallvariance among individuals and it is fully invariant to a uniform scaling of the image and in-plane rotation of the face. Since eye blinking is performed by both eyes synchronously,the EAR of both eyes is averaged. An example of an EAR signal over the video sequence is shown in Fig. 1, 2, 7.\n Asimilar feature to measure the eye opening wassuggested in [9], but it was derived from the eye segmentation in a binary image.\n 2.2. Classification\n It generally does not hold that low value of the EAR means that a person is blinking. A low value of the EAR may occur when a subject closes his/her eyes intentionally for a longer time or performs a facial expression, yawning, etc., or the EAR captures a short random fluctuation of the landmarks.\n Therefore, we propose a classifier that takes a larger temporal window of a frame as an input. For the 30fps videos, we experimentally found that 6 frames can have a significant impact on a blink detection for a frame where an eye is the most closed when blinking. Thus, for each frame, a 13-dimensional feature is gathered by concatenating the EARs of its +-6 neighboring frames.\n This is implemented by a linear SVM classifier(called EAR SVM) trained from manually annotated sequences. Positive examples are collected as  ground-truth blinks, while the negatives are those that are sampled from parts of the videos where no blink occurs, with 5 frames spacing and 7 frames margin from the ground-truth blinks. While testing, a classifier is executed in a scanning-window fashion. A13-dimensional feature is computed and classified by EAR SVM for each frame except the beginning and ending of a video sequence.\n\n3. Experiments\n   -----------\n Two types of experiments were carried out: The experiments that measure accuracy of the landmark detectors, see Sec. 3.1, and the experiments that evaluate performance of the whole eye blink detection algorithm, see Sec 3.2.\n   3.1. Accuracy of landmark detectors\n   To evaluate accuracy of tested landmark detectors,weused the 300-VW dataset [19]. It is a dataset containing 50 videos where each frame has associated a precise annotation of facial landmarks. The videos are “in-the-wild”, mostly recorded from a TV.\n The purpose of the following tests is to demonstrate that recent landmark detectors are particularly robust and precise in detecting eyes, i.e. the eye corners and contour of the eyelids. Therefore we prepared a dataset, a subset of the 300-VW, containing sample images with both open and closed eyes. More precisely, having the ground-truth landmark annotation, we sorted the frames for each subject by the eye aspect ratio (EAR in Eq. (1)) and took 10 frames of the highest ratio (eyes wide open), 10 frames of the lowest ratio (mostly eyes tightly shut) and 10 frames sampled randomly. Thus we collected 1500 images.Moreover, all the images were later subsampled (successively 10 times by factor 0.75) in order to evaluate accuracy of tested detectors on small face images.\n Two state-of-the-art landmark detectors were tested: Chehra [1] and Intraface [16]. Both run in real-time1. Samples from the dataset are shown in Fig. 3. Notice that faces are not always frontal to the camera, the expression is not always neutral, people are often emotionally speaking or smiling, etc.Sometimes people wear glasses, hair may occasionally partially occlude one of the eyes. Both detectors perform generally well, but the Intraface is more robust to very small face images, sometimes at impressive extent as shown in Fig. 3.\n![28-Figure5 1-1](https://github.com/user-attachments/assets/b56bac77-79e3-4894-9063-684975a61d77)\nFigure 3: Example images from the 300-VW dataset with landmarks obtained by Chehra [1] and Intraface [16]. Original images (left) with inter-ocular distance (IOD) equal to 63 (top) and 53 (bottom) pixels. Images subsampled (right) to IOD equal to 6.3(top) and 17 (bottom).\n Quantitatively, the accuracy of the landmark detection for a face image is measured by the average relative landmark localization error, defined as usually\n where xi is the ground-truth location of landmark i in the image, xi is an estimated landmark location by a detector, N is a number of landmarks and normalization factor is the inter-ocular distance (IOD), i.e.Euclidean distance between eye centers in the image.\n First, a standard cumulative histogram of the aver age relative landmark localization error was calculated, see Fig. 4, for a complete set of 49 landmarks and also for a subset of 12landmarksoftheeyesonly,since these landmarks are used in the proposed eye blink detector. The results are calculated for all the original images that have average IOD around 80 px,and also for all “small” face images (including sub Sampled ones) having IOD 50 px. For all landmarks, Chehra has more occurrences of very small errors (up to 5 percent of the IOD), but Intraface is more robust having more occurrences of errors below 10 percent of the IOD. For eye landmarks only, the Intraface is always more precise than Chehra. As already mentioned, the Intraface is much more robust to small images than Chehra. This behaviour is further observed in the following experiment.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitskashfur%2Feye_blink_detection_system","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fitskashfur%2Feye_blink_detection_system","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitskashfur%2Feye_blink_detection_system/lists"}