{"id":18584086,"url":"https://github.com/e-candeloro/driver-state-detection","last_synced_at":"2025-08-13T13:31:12.888Z","repository":{"id":38393904,"uuid":"375127906","full_name":"e-candeloro/Driver-State-Detection","owner":"e-candeloro","description":"A real time, webcam based, driver attention state detection/monitoring system in Python3 using OpenCV and Mediapipe","archived":false,"fork":false,"pushed_at":"2024-06-16T22:28:38.000Z","size":87516,"stargazers_count":123,"open_issues_count":0,"forks_count":28,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-11-30T17:41:12.638Z","etag":null,"topics":["automotive","collaborate","computer-vision","distracted-driving-detection","dlib","driver-safety","human-attention","machine-learning","mediapipe","opencv-python","perclos","pose-estimation","python3","real-time"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/e-candeloro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-08T19:50:48.000Z","updated_at":"2024-11-21T09:28:06.000Z","dependencies_parsed_at":"2024-06-16T21:40:01.158Z","dependency_job_id":null,"html_url":"https://github.com/e-candeloro/Driver-State-Detection","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e-candeloro%2FDriver-State-Detection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e-candeloro%2FDriver-State-Detection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e-candeloro%2FDriver-State-Detection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e-candeloro%2FDriver-State-Detection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/e-candeloro","download_url":"https://codeload.github.com/e-candeloro/Driver-State-Detection/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229763586,"owners_count":18120482,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automotive","collaborate","computer-vision","distracted-driving-detection","dlib","driver-safety","human-attention","machine-learning","mediapipe","opencv-python","perclos","pose-estimation","python3","real-time"],"created_at":"2024-11-07T00:26:07.017Z","updated_at":"2025-08-13T13:31:12.861Z","avatar_url":"https://github.com/e-candeloro.png","language":"Python","readme":"# Real Time Driver State Detection\n![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge\u0026logo=python\u0026logoColor=ffdd54)  ![OpenCV](https://img.shields.io/badge/opencv-%23white.svg?style=for-the-badge\u0026logo=opencv\u0026logoColor=white) \n\nReal time, webcam based, driver attention state detection and monitoring using Python with the OpenCV and Mediapipe libraries.\n\n![driver state detection demo](./demo/new_mediapipe_dsd_demo.gif)\n\n**Note**:\nThis work is partially based on [this paper](https://www.researchgate.net/publication/327942674_Vision-Based_Driver%27s_Attention_Monitoring_System_for_Smart_Vehicles) for the scores and methods used.\n\n## Mediapipe Update\n\nThanks to the awesome contribution of [MustafaLotfi](https://github.com/MustafaLotfi), now the script uses the better performing and accurate face keypoints detection model from the [Google Mediapipe library](https://github.com/google/mediapipe).\n\n## Last Features added:\n\n- Fast 478 face keypoints detection with Mediapipe\n- Direct iris keypoint detection with Mediapipe for gaze score estimation\n- Improved head pose estimation using the dynamical canonical face model\n- Fixed euler angles function and wrong returned values\n- Using time variables to make the code more modular and machine agnostic\n- Added rolling PERCLOS estimation and smoother driver state detection with decay factor\n- Added new demo video\n\n**NOTE**: the old mediapipe version can still be found in the \"dlib-based\" repository branch.\n\n## How Does It Work?\n\nThis script searches for the driver face, then use the mediapipe library to predict 478 face and iris keypoints.\nThe enumeration and location of all the face keypoints/landmarks can be seen [here](./demo/face_keypoints.jpg).\n\nWith those keypoints, the following scores are computed:\n\n- **EAR**: Eye Aspect Ratio, it's the normalized average eyes aperture, and it's used to see how much the eyes are opened or closed\n- **Gaze Score**: L2 Norm (Euclidean distance) between the center of the eye and the pupil, it's used to see if the driver is looking away or not\n- **Head Pose**: Roll, Pitch and Yaw of the head of the driver. The angles are used to see if the driver is not looking straight ahead or doesn't have a straight head pose (is probably unconscious)\n- **PERCLOS**: PERcentage of CLOSure eye time, used to see how much time the eyes are closed in a minute. A threshold of 0.2 is used in this case (20% of a minute) and the EAR score is used to estimate when the eyes are closed.\n\nThe driver states can be classified as:\n\n- **Normal**: no messages are printed\n- **Tired**: when the PERCLOS score is \u003e 0.2, a warning message is printed on screen\n- **Asleep**: when the eyes are closed (EAR \u003c closure_threshold) for a certain amount of time, a warning message is printed on screen\n- **Looking Away**: when the gaze score is higher than a certain threshold for a certain amount of time, a warning message is printed on screen\n- **Distracted**: when the head pose score is higher than a certain threshold for a certain amount of time, a warning message is printed on screen\n\n## Demo\n\n\u003cvideo src=\"https://github.com/user-attachments/assets/94fd76f3-d298-4226-a684-52076f40fe7d\" controls=\"controls\" style=\"max-width: 100%; height: auto;\"\u003e\n    Your browser does not support the video tag.\n\u003c/video\u003e\n\n## The Scores Explained\n\n### EAR\n\n**Eye Aspect Ratio** is a normalized score that is useful to understand the rate of aperture of the eyes.\nUsing the mediapipe face mesh keypoints for each eye (six for each), the eye lenght and width are estimated and using this data the EAR score is computed as explained in the image below:\n![EAR](https://user-images.githubusercontent.com/67196406/121489162-18210900-c9d4-11eb-9d2e-765f5ac42286.png)\n\n**NOTE:** the average of the two eyes EAR score is computed\n\n### Gaze Score Estimation\n\nThe gaze score gives information about how much the driver is looking away without turning his head.\n\nTo understand this, the distance between the eye center and the position of the pupil is computed. The result is then normalized by the eye width that can be different depending on the driver physionomy and distance from the camera.\n\nThe below image explains graphically how the Gaze Score for a single eye is computed:\n![Gaze Score](https://user-images.githubusercontent.com/67196406/121489746-ab5a3e80-c9d4-11eb-8f33-d34afd0947b4.png)\n**NOTE:** the average of the two eyes Gaze Score is computed\n\n### Head Pose Estimation\n\nFor the head pose estimation, a standard 3d head model in world coordinates was considered, in combination of the respective face mesh keypoints in the image plane. \nIn this way, using the solvePnP function of OpenCV, estimating the rotation and translation vector of the head in respect to the camera is possible.\nThen the 3 Euler angles are computed.\n\nThe partial snippets of code used for this task can be found in [this article](https://learnopencv.com/head-pose-estimation-using-opencv-and-dlib/).\n\n## Installation\n\nThis projects runs on Python with the following libraries:\n\n- numpy\n- OpenCV (opencv-python)\n- mediapipe\n\nOr you can use poetry to automatically create a virtualenv with all the required packages:\n\n```\npip install poetry #a global install of poetry is required\n```\n\nThen inside the repo directory:\n\n```\npoetry install\n```\n\nTo activate the env to execute command lines:\n\n```\npoetry shell\n```\n\nAlternatively (not recommended), you can use the requirements.txt file provided in the repository using:\n    \n    pip install -r requirements.txt\n    \n\n## Usage\n\nFirst navigate inside the driver state detection folder:\n    \n    cd driver_state_detection\n\nThe scripts can be used with all default options and parameters by calling it via command line:\n\n    python main.py\n\nFor the list of possible arguments, write:\n\n    python main.py --help\n\nExample of a possible use with parameters:\n\n    python main.py --ear_time_tresh 5\n\nThis will sets to 5 seconds the eye closure time before a warning  message is shown on screen\n\n## Why this project\n\nThis project was developed as part for a final group project for the course of [Computer Vision and Cognitive Systems](https://international.unimore.it/singleins.html?ID=295) done at the [University of Modena and Reggio Emilia](https://international.unimore.it/) in the second semester of the academic year 2020/2021.\nGiven the possible applications of Computer Vision, we wanted to focus mainly on the automotive field, developing a useful and potential life saving proof of concept project.\nIn fact, sadly, many fatal accidents happens [because of the driver distraction](https://www.nhtsa.gov/risky-driving/distracted-driving).\n\n## License and Contacts\n\nThis project is freely available under the MIT license. You can use/modify this code as long as you include the original license present in this repository in it.\n\nFor any question or if you want to contribute to this project, feel free to contact me or open a pull request.\n\n## Improvements to make\n\n- [x] Reformat code in packages\n- [x] Add argparser to run the script with various settings using the command line\n- [x] Improve robustness of gaze detection (using mediapipe)\n- [x] Add argparser option for importing and using the camera matrix and dist. coefficients\n- [x] Reformat classes to follow design patterns and Python conventions\n- [ ] Debug new mediapipe methods and classes and adjust thresholds\n- [ ] Improve perfomances of the script by minimizing image processing steps\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fe-candeloro%2Fdriver-state-detection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fe-candeloro%2Fdriver-state-detection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fe-candeloro%2Fdriver-state-detection/lists"}