{"id":27645985,"url":"https://github.com/gbr-rl/visionsort-rpi","last_synced_at":"2025-10-17T09:46:03.799Z","repository":{"id":269604004,"uuid":"907946338","full_name":"GBR-RL/VisionSort-RPi","owner":"GBR-RL","description":"A conveyor belt sorting system powered by Raspberry Pi and YOLOv8 for real-time object detection.","archived":false,"fork":false,"pushed_at":"2025-04-09T09:44:09.000Z","size":36255,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-24T01:14:17.275Z","etag":null,"topics":["automation","computer-vision","diy-project","hailo","hailo-ai","object-detection","raspberry-pi","real-time-inference","yolov8"],"latest_commit_sha":null,"homepage":"https://gbr-rl.github.io/VisionSort-RPi/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GBR-RL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-24T17:09:14.000Z","updated_at":"2025-04-17T15:51:26.000Z","dependencies_parsed_at":null,"dependency_job_id":"9027622c-c7df-4f89-ae9d-f00ccebd79f7","html_url":"https://github.com/GBR-RL/VisionSort-RPi","commit_stats":null,"previous_names":["gbr-rl/visionsort-rpi"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GBR-RL%2FVisionSort-RPi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GBR-RL%2FVisionSort-RPi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GBR-RL%2FVisionSort-RPi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GBR-RL%2FVisionSort-RPi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GBR-RL","download_url":"https://codeload.github.com/GBR-RL/VisionSort-RPi/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250540917,"owners_count":21447427,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","computer-vision","diy-project","hailo","hailo-ai","object-detection","raspberry-pi","real-time-inference","yolov8"],"created_at":"2025-04-24T01:14:52.644Z","updated_at":"2025-10-17T09:46:03.787Z","avatar_url":"https://github.com/GBR-RL.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![GitHub Pages](https://img.shields.io/badge/Live%20Docs-GitHub%20Pages-blue?logo=github)](https://gbr-rl.github.io/VisionSort-RPi/)\n\n![Computer Vision](https://img.shields.io/badge/Project-Computer%20Vision-blue)\n![Embedded Systems](https://img.shields.io/badge/Project-Embedded%20Systems-green)\n![Image Processing](https://img.shields.io/badge/Project-Image%20Processing-yellow)\n![Python](https://img.shields.io/badge/Python-3.10-blue)\n![OpenCV](https://img.shields.io/badge/OpenCV-4.5-green)\n![ONNX](https://img.shields.io/badge/ONNX-Inference-purple)\n![YOLOv8](https://img.shields.io/badge/YOLO-v8-orange)\n![Raspberry Pi](https://img.shields.io/badge/Hardware-Raspberry%20Pi-red)\n![Status](https://img.shields.io/badge/Project%20Status-Completed-success)\n\n# VisionSort-RPi \n**An AI-Powered Conveyor Belt Sorting System Using Raspberry Pi + YOLOv8 + Hailo**\n\nA real-time bolt-and-nut sorting system powered by computer vision and embedded AI. This project blends hardware engineering with object detection and real-world automation — combines hardware engineering, object detection, and creative problem-solving to efficiently sort Bolts and Nuts.\n\n---\n\n## Overview\n\n**VisionSort-RPi** is an edge-deployed conveyor belt system that showcases real-world applications of embedded AI.  \nBuilt using **Raspberry Pi 5**, **YOLOv8**, and the **Hailo AI accelerator**, it automates object classification and sorting tasks with high accuracy and speed.\n\n---\n\n## 🖼️ 3D Model \u0026 Real Conveyor Sorting System\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cstrong\u003e3D CAD Model\u003c/strong\u003e\u003cbr\u003e\n      \u003cimg src=\"Images/View1.jpg\" alt=\"3D CAD Model\" width=\"400\"/\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cstrong\u003eReal Conveyor System\u003c/strong\u003e\u003cbr\u003e\n      \u003cimg src=\"Images/Real_System1.jpg\" alt=\"Conveyor Sorting System\" width=\"400\"/\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003e _Side-by-side view of the designed CAD model and the actual built system with camera and flapper mounted._\n\n\n## Key Features\n\n- ✅ **Real-Time Object Detection** with YOLOv8 + Hailo for bolts and nuts\n- ✅ **Stepper + Servo Motor Integration** for precise movement and sorting\n- ✅ **Edge Deployment** on Raspberry Pi 5 with hardware-accelerated inference\n- ✅ **Custom Dataset** with over 9,000 annotated training images\n- ✅ **Optimized for speed** using neural network acceleration (Hailo RT)\n- ✅ **Fully Autonomous Sorting** with no human-in-the-loop\n\n---\n\n## Tech Stack\n\n| Category         | Tools \u0026 Hardware                               |\n|------------------|-------------------------------------------------|\n| 🧠 CV/AI          | YOLOv8, Roboflow, Python, OpenCV               |\n| ⚙️ Embedded HW    | Raspberry Pi 5, Hailo AI Kit, NEMA 17, MG996R Servo  |\n| 📷 Camera         | Raspberry Pi HQ Camera + 16mm 10MP Telephoto   |\n| 🔌 Communication  | GPIO, PWM, I2C                                 |\n| 📦 Dataset        | Roboflow custom-labeled (9000+ images)         |\n\n---\n\n## Mechanical Components \u0026 CAD\n\nThis project includes custom-designed mechanical components modeled in Creo Parametric and fabricated using 3D printing. Key components include:\n\n- Flapper mechanism for nut/bolt sorting\n- Servo mounting bracket\n- RPi HQ camera holder\n\n🔗 View CAD designs \u0026 prints in [`CAD/`](./CAD)  \n📄 Download full BOM: [`BOM.xlsx`](./CAD/BOM.xlsx)  \n📤 External CAD repo: [GrabCAD – VisionSort-RPi](https://grabcad.com/library/visionsort-rpi-custom-cad-parts-1)\n\n📦 Download 3D printable CAD models:\n➡️ [GitHub Release: CAD v1.0](https://github.com/GBR-RL/VisionSort-RPi/releases)\n\n---\n\n## Project Architecture\n\n### Hardware Design\n- Conveyor belt driven by **NEMA 17 stepper motor**\n- **Servo-controlled deflector arm** for object redirection\n- HQ Camera with a **16mm telephoto lens** for accurate detection\n- All powered and controlled via **Raspberry Pi 5**\n\n### Data + AI\n- Pre-annotated dataset from **Roboflow**\n- Bounding box labels with `nut` and `bolt` classes\n- Trained on HPC using YOLOv8 custom config\n- Inference optimized with **Hailo RT SDK**\n\n---\n\n### Software Pipeline\n\n1. **Camera Frame Capture** → \n2. **Image Preprocessing (OpenCV)** →\n3. **YOLOv8 Inference (Hailo AI)** →\n4. **Object Classification** →\n5. **Motor Control Signal** →\n6. **Stepper + Servo Movement**\n\n---\n\n##  Setup Instructions\n\n### Hailo AI Environment Setup\n\nThis project utilizes **Hailo’s AI acceleration platform** for deploying YOLOv8 models on the Raspberry Pi 5.  \nYou can find the environment setup in their [official repo](https://github.com/hailo-ai/hailo-rpi5-examples).\n\n#### Installation Steps\n\n```bash\ngit clone https://github.com/hailo-ai/hailo-rpi5-examples.git\ncd hailo-rpi5-examples\n./install.sh\n```\n\nEvery time you open a new terminal session, activate the environment:\n\n```bash\nsource setup_env.sh\n```\n\n---\n\n### Running the Conveyor Sorting System\n\nRun the real-time detection and sorting application using:\n\n```bash\npython detection/detection.py \\\n  --labels-json resources/nut_bolt-labels.json \\\n  --hef-path model/nut_bolt_model.hef \\\n  --input rpi\n```\n\n\u003e Press `Ctrl+C` to stop the system gracefully.\n\n---\n\n## Hardware Wiring Diagram\n\nThe wiring for the stepper and servo motor system was meticulously crafted for precise control and safety.  \nKey components include the **TMC2208 stepper driver**, **MG996R servo motor**, **buck converter**, and **external 12V power supply**.\n\n### Wiring Summary\n\n| Component             | Raspberry Pi GPIO           | Notes                              |\n|----------------------|-----------------------------|-------------------------------------|\n| **TMC2208 - STEP**    | GPIO17 (Pin 11)             | Step signal                         |\n| **TMC2208 - DIR**     | GPIO27 (Pin 13)             | Direction signal                    |\n| **TMC2208 - EN**      | GND                         | Must be LOW to enable               |\n| **TMC2208 - VIO**     | 3.3V                        | Logic level                         |\n| **TMC2208 - GND**     | GND                         |                                     |\n| **MG996R - SIGNAL**   | GPIO12 (PWM)                | Servo control via PWM               |\n| **Servo Vcc**         | Buck Converter 5V           | Ensure stable 5V using multimeter   |\n| **Buck Converter Input** | 12V external supply      | Powers motors + regulated 5V output |\n\n📎 Full schematic is detailed in [`docs/Wiring_Setup.pdf`](docs/Wiring_Setup.pdf)\n\n---\n\n## Training the YOLOv8 Model\n\nThe model used in this project is a **fine-tuned YOLOv8n** trained on a **custom Roboflow dataset** (9000+ images).\n\nTraining script used: [`training.py`](training.py)\n\n- Based on YOLOv8n.pt pretrained weights\n- Trained to detect only two classes: `bolt` and `nut`\n- Handled augmentation, batch balancing, and early stopping\n\n---\n\n## Model Conversion for Hailo Deployment\n\nYOLOv8 `.pt` model was converted to Hailo-compatible `.hef` format using the **Hailo Dataflow Compiler (DFC)**.\n\n Conversion steps are documented in [`Convert_YoloV8_to_HEF.ipynb`](Convert_YoloV8_to_HEF.ipynb)\n\n### Pipeline\n- `.pt` → `.onnx` via Ultralytics export\n- `.onnx` → `.hef` using Hailo’s `hef-generator` script\n\n Guide followed: [Guide to Using Hailo DFC](https://community.hailo.ai/t/guide-to-using-the-dfc-to-convert-a-modified-yolov11-on-google-colab/7131)\n\n---\n\n## System Demonstration\n\n\u003e  Real-time sorting demo using computer vision and Raspberry Pi:\n\n🎥 [![VisionSort Demo](https://img.youtube.com/vi/cI-RrebKUd8/hqdefault.jpg)](https://www.youtube.com/watch?v=cI-RrebKUd8)\n\n---\n\n## What's Next?\n\n- Add camera calibration + lighting normalization\n- Expand dataset to include more object categories for diverse sorting applications.  \n- Integrate IoT for remote monitoring. \n---\n\n## 📩 Contact\nI’m excited to connect and collaborate!  \n- **Email**: [gbrohiith@gmail.com](mailto:your.email@example.com)  \n- **LinkedIn**: [https://www.linkedin.com/in/rohiithgb/](https://linkedin.com/in/yourprofile)  \n- **GitHub**: [https://github.com/GBR-RL/](https://github.com/yourusername)\n\n---\n\n## 📚 License\nThis project is open-source and available under the [MIT License](LICENSE).  \n\n---\n\n🌟 **If you like this project, please give it a star!** 🌟\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgbr-rl%2Fvisionsort-rpi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgbr-rl%2Fvisionsort-rpi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgbr-rl%2Fvisionsort-rpi/lists"}