{"id":28474325,"url":"https://github.com/arkdevuk/cozmo_spatial_cue","last_synced_at":"2025-07-01T10:32:13.701Z","repository":{"id":296679089,"uuid":"993819974","full_name":"arkdevuk/cozmo_spatial_cue","owner":"arkdevuk","description":"A simple top-down camera that is able to follow cozmo, the model was trained on a dataset of 200 images taken top-down of Cozmo. Model's included","archived":false,"fork":false,"pushed_at":"2025-06-01T10:07:26.000Z","size":17465,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-06-01T17:46:46.186Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arkdevuk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-31T15:38:48.000Z","updated_at":"2025-06-01T10:09:16.000Z","dependencies_parsed_at":"2025-06-01T17:46:48.268Z","dependency_job_id":"3d9fbf88-b6e4-415d-9eb1-7609c410e04e","html_url":"https://github.com/arkdevuk/cozmo_spatial_cue","commit_stats":null,"previous_names":["arkdevuk/cozmo_spatial_cue"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/arkdevuk/cozmo_spatial_cue","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arkdevuk%2Fcozmo_spatial_cue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arkdevuk%2Fcozmo_spatial_cue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arkdevuk%2Fcozmo_spatial_cue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arkdevuk%2Fcozmo_spatial_cue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arkdevuk","download_url":"https://codeload.github.com/arkdevuk/cozmo_spatial_cue/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arkdevuk%2Fcozmo_spatial_cue/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262944206,"owners_count":23388714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-07T13:06:47.626Z","updated_at":"2025-07-01T10:32:13.693Z","avatar_url":"https://github.com/arkdevuk.png","language":"Python","readme":"# Cozmo Visual Cue Service (model included)\n\n**A real-time object detection and tracking service for Anki Cozmo using YOLOv11 fine-tuned model and MQTT.**\nDetects and tracks your robot with your webcam, sends live annotated images, and publishes robot activity events to MQTT for seamless robotics integration.\n\n---\n\n![annotated image](https://github.com/arkdevuk/cozmo_spatial_cue/raw/master/Doc/images/cozmo_found.jpeg)\n\n---\n\n## Features\n\n* **Real-time webcam object detection \u0026 tracking** using \\[Ultralytics YOLOv11]\n* **Custom YOLOv11 model included** fine-tuned for Cozmo robot detection from top-down webcam view\n* **MQTT event publishing**:\n  * `robot_found` / `robot_lost` when Cozmo appears/disappears\n  * `motion` events when the robot is moving or stuck\n  * Live annotated camera images\n* **Automatic hardware acceleration** (CUDA if available)\n* **Configurable** via `.env` or `.env.local`\n\n---\n\n## How it Works\n\n* The service runs a YOLOv detection model on webcam frames.\n* When Cozmo is detected, it broadcasts:\n  * Annotated image frames (JPG, base64-encoded) on MQTT topic `cozmo/camera_top/annotated`\n  * Robot presence events on `cozmo/camera_top/events`\n  * Motion status events on `cozmo/camera_top/motion`\n* If Cozmo disappears for a short period (configurable), a `robot_lost` event is sent.\n* If Cozmo is detected again, a `robot_found` event is sent.\n* Motion status (`moving` or not) is determined based on the center of the bounding box, filtered by simple logic to avoid noise (Kalman was too slow).\n\n---\n\n![logs](https://github.com/arkdevuk/cozmo_spatial_cue/raw/master/Doc/images/logs.png)\n\n---\n\n## MQTT Topics\n\n* `cozmo/camera_top/annotated` — **Live annotated image (JPG base64, with timestamp)**\n* `cozmo/camera_top/events` — **Robot presence**\n\n  * `{ \"type\": \"robot_found\" }`\n  * `{ \"type\": \"robot_lost\" }`\n* `cozmo/camera_top/motion` — **Motion event**\n\n  * `{ \"state\": true|false, \"stamp\": \"timestamp\" }`\n\n---\n\n## Example `.env`:\n\n```\nMQTT_BROKER=localhost\nMQTT_PORT=1884\nMQTT_USERNAME=username\nMQTT_PASSWORD=password\n```\n\n---\n\n## Code Overview\n\n* `main.py` — Main script, event loop, detection \u0026 MQTT publishing\n* `Camera/CameraStream.py` — Threaded webcam capture\n* `Logger.py` — Logging abstraction\n* `MQTTClient/MQTTClient.py` — MQTT wrapper for easy publishing\n\n---\n\n## Extending/Integrating\n\n* Use the MQTT topics in your own robot control scripts or dashboards.\n* Tune detection, presence, and motion parameters in `main.py`.\n* Swap the model or retrain on your custom dataset for more objects or different robots.\n\n---\n\n## License\n\n[MIT License](LICENSE)\n\n---\n\n## Credits\n\n* [Ultralytics YOLO](https://github.com/ultralytics/ultralytics)\n* [Anki Cozmo SDK](https://github.com/anki/cozmo-python-sdk)\n* [PyCozmo](https://github.com/zayfod/pycozmo)\n* MQTT with [paho-mqtt](https://www.eclipse.org/paho/)\n\n---\n\n***Pull requests welcome!** For questions or ideas, open an issue or discussion.*","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farkdevuk%2Fcozmo_spatial_cue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farkdevuk%2Fcozmo_spatial_cue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farkdevuk%2Fcozmo_spatial_cue/lists"}