{"id":24128159,"url":"https://github.com/rignitc/finger-reader","last_synced_at":"2025-10-11T07:16:22.929Z","repository":{"id":220823299,"uuid":"517317109","full_name":"rignitc/Finger-Reader","owner":"rignitc","description":"Finger-worn device to assist children and individuals with Dyslexia in Reading and Object Recognition.","archived":false,"fork":false,"pushed_at":"2024-10-20T14:00:38.000Z","size":5164,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-03-01T03:44:33.507Z","etag":null,"topics":["object-recognition","opencv","reading-assistant","tesseract-ocr","yolo-algorithm","yolov4-tiny"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rignitc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-24T12:25:21.000Z","updated_at":"2024-10-20T16:25:05.000Z","dependencies_parsed_at":"2024-02-04T16:29:28.634Z","dependency_job_id":"0ae44e79-50ba-4c2f-9246-618c046e0d91","html_url":"https://github.com/rignitc/Finger-Reader","commit_stats":null,"previous_names":["rignitc/finger-reader"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rignitc/Finger-Reader","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rignitc%2FFinger-Reader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rignitc%2FFinger-Reader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rignitc%2FFinger-Reader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rignitc%2FFinger-Reader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rignitc","download_url":"https://codeload.github.com/rignitc/Finger-Reader/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rignitc%2FFinger-Reader/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006596,"owners_count":26084130,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["object-recognition","opencv","reading-assistant","tesseract-ocr","yolo-algorithm","yolov4-tiny"],"created_at":"2025-01-11T18:38:00.391Z","updated_at":"2025-10-11T07:16:22.892Z","avatar_url":"https://github.com/rignitc.png","language":"Python","readme":"# Finger-Reader\n\n## Project Description\n\nDesigned and implemented a portable, finger-worn device to assist children and individuals with dyslexia in reading. Assists the user by reading out the words to which the user points. Also has an added mode to to detect objects to which the used points. The following work is presented is this project:\n1. Designing and implementing the hardware of the device.\n2. Developing the Python codes for reading assistance, utilizing the `OpenCV` library for image processing and the `Tesseract OCR engine` to extract text from images.\n3. Developing the Python codes for object recognition using the `YOLO algorithm`.\n\n## Hardware\n\n![alt text](Images/Screenshot%20from%202024-02-06%2021-31-54-modified.png)\n\n* Raspberry Pi (3 B+ model)\n* Raspberry Pi Camera Module v2\n* Camera mount (3D printed)\n* Wrist casing for Raspberry Pi (3D printed)\n* Speaker\n\n## Setting up the Reading Assistance feature\n\n__Code__: [reader.py](reader.py)\n\nThe Python source code is run on the Raspberry Pi. OpenCV library is used to enhance the text in the‬\nimage and the Pytesseract library (OCR engine wrapper) extracts text from images. The text is converted to‬\nspeech by the gTTS library (Google Text-to-Speech). Represented pictorially as:\n\n![alt text](Images/Screenshot%20from%202024-02-06%2021-32-05-modified.png)\n\n### Defining function to Deskew the image\n\n__Code__: [deskew.py](deskew.py)\n\nTo install OpenCV and numpy, run:\n```python\npip install opencv-python\n```\n```python\npip install numpy\n```\nCertain image processing techniques need to be done, to obtain good results from OCR. Deskewing is the process of correcting the twists (skew) of the text in an image,  rotating the image such that text is vertically aligned. It includes the following process.\n1. Image Loading \u0026 Grayscale Conversion: The image is loaded and converted to grayscale, simplifying processing by reducing it to a single channel.\n2. Inversion \u0026 Thresholding: The grayscale image is inverted to enhance contrast, and Otsu's thresholding is applied to create a binary image, clearly separating the foreground from the background.\n3. Contour Extraction \u0026 Rectangle Calculation: Coordinates of the foreground pixels are extracted, and the minimum area rectangle that encloses these points is determined, providing information about the image's tilt.\n4. Angle Adjustment \u0026 Rotation: The angle of the rectangle is adjusted for manageable rotation, and a rotation matrix is created to deskew the image back to its straight position.\n\n### Defining Text to Speech function \n\nCode: [sound.py](sound.py)\n\nThe text output by the OCR engine is converted to‬ speech by the gTTS library (Google Text-to-Speech). Install packages,\n```python\npip install gtts\npip install playsound\n```\n### Reading the text \n\nCode: [reader.py](reader.py) \n\n1. __Color Detection__: Converts each frame to the HSV color space and creates a mask to isolate a specific color range using cv2.inRange(). This mask helps in filtering the image to highlight the finger or other objects of interest.\n2. __Contour Detection__: After thresholding the mask to create a binary image, the code identifies contours using cv2.findContours(). This helps in detecting the finger in the image.\n3. __Finding the Tip of the Finger__: The code calculates the lowest point of the largest contour, which represents the tip of the finger. It iterates through the contour points to determine the point with the minimum y-coordinate.\n4. __Tesseract OCR engine__:\n   __Code__: [ocr.py](ocr.py)\n   \n   It is used to extract the text from th image. Pytesseract or Python-tesseract is an OCR tool for python that also serves as a wrapper for the Tesseract-OCR Engine. The distance between each of the word and finger tip is measured, and the one closest is read.\n\nTo install Tesseract OCR Engine:\n\n__For Windows__: Download the Tesseract installer from the [Github Link](https://github.com/tesseract-ocr/tesseract)\n\n__For macOS__: \n```\nbrew install tesseract\n```\n__For Ubuntu/Debian__:\n```\nsudo apt-get update\nsudo apt-get install tesseract-ocr\n```\nTo install the python packages to use OCR engine, run:\n```python\npip install pytesseract\n```\n## Object Detection\n\n__Code__: [Object_recognition.py](Object_recognition.py) \u0026 [Object_detection.py](Object_detection.py)\n\n 1. The device has an added mode that helps the user in object recognition. Assists the user by identifying and playing the sound of the object's name. \n 2. The object detection is done using the `Yolo algorithm`. To understand more about the Yolo algorithm refer: [AlexeyAB/darknet](https://github.com/AlexeyAB/darknet)\n\nTo use the codes [Object_recognition.py](Object_recognition.py) \u0026 [Object_detection.py](Object_detection.py) implemented using the Yolo algorithm download the [coco.names](https://github.com/AlexeyAB/darknet/blob/master/data/coco.names), [yolov4-tiny.cfg](https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov4-tiny.cfg) and the [yolov4-tiny.weights](yolov4-tiny.weights) files from [AlexeyAB/darknet](https://github.com/AlexeyAB/darknet)\n\nThe code [Object_detection.py](Object_detection.py) detects all the objects in the image. But for assisting the user, the object to which the user points must be read. This is done in the code [Object_recognition.py](Object_recognition.py).\n1. __Initialization and Model Setup__: The code initializes the YOLOv4 model using the yolov4.weights and yolov4.cfg files. It reads class names from coco.names and sets up the webcam to capture live video frames.\n2. __Object Detection and Preprocessing__: Each frame from the webcam is preprocessed using cv2.dnn.blobFromImage(). The model performs a forward pass, generating bounding box coordinates, confidence scores, and class IDs for detected objects.\n3. __Finding the Closest Object__: The code calculates the distance from the center of the frame (at (width // 2, height // 2)) to the centers of all detected objects. It identifies the closest object by tracking the index with the minimum distance.\n4. Displaying Results and Triggering Sound: If a close object is detected, a bounding box is drawn around it, its label is displayed and the corresponding label sound is played. \n\n\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frignitc%2Ffinger-reader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frignitc%2Ffinger-reader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frignitc%2Ffinger-reader/lists"}