{"id":51358372,"url":"https://github.com/youhanasheriff/pocket_ai","last_synced_at":"2026-07-02T21:10:14.357Z","repository":{"id":361638324,"uuid":"1255213073","full_name":"youhanasheriff/pocket_ai","owner":"youhanasheriff","description":"Real-time YOLO11 obstacle detection for edge devices — depth estimation, spatial reasoning, and spoken guidance (\u003c30ms)","archived":false,"fork":false,"pushed_at":"2026-05-31T15:02:32.000Z","size":25652,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-31T17:05:28.657Z","etag":null,"topics":["accessibility","computer-vision","edge-ai","onnx","python","real-time","yolo"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/youhanasheriff.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-31T14:52:06.000Z","updated_at":"2026-05-31T15:02:36.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/youhanasheriff/pocket_ai","commit_stats":null,"previous_names":["youhanasheriff/pocket_ai"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/youhanasheriff/pocket_ai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youhanasheriff%2Fpocket_ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youhanasheriff%2Fpocket_ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youhanasheriff%2Fpocket_ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youhanasheriff%2Fpocket_ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/youhanasheriff","download_url":"https://codeload.github.com/youhanasheriff/pocket_ai/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youhanasheriff%2Fpocket_ai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35062941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-02T02:00:06.368Z","response_time":173,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accessibility","computer-vision","edge-ai","onnx","python","real-time","yolo"],"created_at":"2026-07-02T21:10:13.618Z","updated_at":"2026-07-02T21:10:14.346Z","avatar_url":"https://github.com/youhanasheriff.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pocket AI Guardian 👁️🔊\n\n**Offline, real-time obstacle-detection assistant that turns a camera feed into spoken navigation guidance — built to run on the edge.**\n\nPocket AI Guardian is a modular, fully on-device pipeline that detects navigation hazards (obstacles, doors, stairs, people, …) from a live camera and speaks concise, prioritized guidance. It runs without a network connection and is tuned to the latency and memory budgets of edge hardware (RDK X5 / Qualcomm QCS6490) as well as desktops.\n\n\u003e **Pipeline:** Camera → YOLO11 detection → spatial reasoning → template instructions → TTS / console\n\n## Highlights\n- **Real-time on-device inference** — YOLO11 detector with a **desktop profile** (FP16/FP32, 640px, 20–40 FPS target) and an **edge profile** (INT8, 320px, 15–25 FPS target), selected automatically or via `--profile`.\n- **Spatial reasoning** — turns raw detections into distance/direction context and prioritizes by safety level.\n- **Optional depth estimation** — INT8 ONNX depth model, with a heuristic fallback for lower latency.\n- **Spoken guidance** — offline text-to-speech using a \"latest-wins\" single-slot model with deduplication, priority, and a cooldown so the user isn't overwhelmed.\n- **Fully offline** — no network calls at inference time.\n\n## Detection classes\n`closed_door` · `door` · `elevator` · `escalator` · `footpath` · `obstacle` · `person` · `wall`\n\nEach class maps to a **safety level** (high / medium / low) that drives alert priority — e.g. `obstacle`, `person`, and `escalator` are high-priority.\n\n## Architecture\n```\ncamera → detector (YOLO11) → spatial → validator → instructor → tts / console\n                                ↑\n                          depth (ONNX, optional)\n```\n\n| Module | Role |\n|---|---|\n| `pipeline/detector.py` | YOLO11 object detection |\n| `pipeline/depth.py` | Depth estimation (INT8 ONNX + heuristic fallback) |\n| `pipeline/spatial.py` | Distance / direction reasoning |\n| `pipeline/validator.py` | Safety-rule validation |\n| `pipeline/instructor.py` | Template-based instruction generation |\n| `pipeline/tts.py` | Offline text-to-speech (latest-wins, dedup, cooldown) |\n| `pipeline/orchestrator.py` | Wires the pipeline together |\n| `config.py` | Single source of truth: hardware profiles, classes, thresholds |\n\n## Quick start\n```bash\npip install -r requirements.txt\n\npython main.py                      # webcam, auto-detect hardware\npython main.py --source 0 --show    # webcam with OpenCV preview\npython main.py --source image.jpg   # single image\npython main.py --profile edge       # force edge profile (INT8, 320px)\npython main.py --profile desktop    # force desktop profile\npython main.py --list-models        # show available models\npython main.py --conf 0.35          # custom confidence threshold\npython main.py --cooldown 3.0       # instruction cooldown (seconds)\npython main.py --save-logs          # save JSONL detection logs to logs/\n```\n\n## Tech stack\n- **Vision:** Ultralytics YOLO11, OpenCV, NumPy\n- **Edge inference:** ONNX Runtime (INT8 models)\n- **TTS:** pyttsx3 (offline)\n- **Hardware targets:** Desktop (Mac/PC), RDK X5 (10 TOPS BPU), Qualcomm QCS6490 (12 TOPS NPU)\n- **Bundled models:** `models/vision/obstacle_desktop.pt` (YOLO11), `models/depth/depth_small_int8.onnx`\n\n## Tests\n```bash\npytest tests/\n```\n\n## Docs\nSee [`docs/`](docs/) for the architecture overview, deployment guide, technical execution plan, and training report.\n\n---\n*Part of an edge-AI research line exploring assistive, real-time AI on resource-constrained hardware. See also [models-edge-devices](https://github.com/youhanasheriff/models-edge-devices).*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyouhanasheriff%2Fpocket_ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyouhanasheriff%2Fpocket_ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyouhanasheriff%2Fpocket_ai/lists"}