{"id":50154792,"url":"https://github.com/howdoiusekeyboard/ros2_navigation_project","last_synced_at":"2026-05-24T10:06:29.289Z","repository":{"id":298137524,"uuid":"975278936","full_name":"howdoiusekeyboard/ros2_navigation_project","owner":"howdoiusekeyboard","description":"Voice-controlled ROS 2 navigation system using Gemini 2.5 Flash for natural language command parsing. Features a real-time React dashboard and full Nav2 integration.","archived":false,"fork":false,"pushed_at":"2026-04-14T07:28:28.000Z","size":7577,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-04-14T09:28:19.029Z","etag":null,"topics":["artificial-inteligence","fastapi","gemini-api","navigation2","react","robotics","ros2","ros2-humble","turtlebot3","typescript","voice-control"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/howdoiusekeyboard.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-04-30T04:13:40.000Z","updated_at":"2026-04-14T07:28:30.000Z","dependencies_parsed_at":"2025-10-21T03:09:23.872Z","dependency_job_id":"4d312eb0-4bc5-4cee-8a3c-063a74470a31","html_url":"https://github.com/howdoiusekeyboard/ros2_navigation_project","commit_stats":null,"previous_names":["howdoiusekeyboard/ros2_navigation_project"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/howdoiusekeyboard/ros2_navigation_project","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/howdoiusekeyboard%2Fros2_navigation_project","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/howdoiusekeyboard%2Fros2_navigation_project/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/howdoiusekeyboard%2Fros2_navigation_project/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/howdoiusekeyboard%2Fros2_navigation_project/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/howdoiusekeyboard","download_url":"https://codeload.github.com/howdoiusekeyboard/ros2_navigation_project/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/howdoiusekeyboard%2Fros2_navigation_project/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33429223,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-23T22:14:44.296Z","status":"online","status_checked_at":"2026-05-24T02:00:06.296Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-inteligence","fastapi","gemini-api","navigation2","react","robotics","ros2","ros2-humble","turtlebot3","typescript","voice-control"],"created_at":"2026-05-24T10:06:28.646Z","updated_at":"2026-05-24T10:06:29.284Z","avatar_url":"https://github.com/howdoiusekeyboard.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ROS 2 Navigation Interface\n\n![ROS 2 Humble](https://img.shields.io/badge/ROS_2-Humble-34495E?logo=ros\u0026logoColor=white)\n![React](https://img.shields.io/badge/React-18-61DAFB?logo=react\u0026logoColor=black)\n![FastAPI](https://img.shields.io/badge/FastAPI-0.109-009688?logo=fastapi\u0026logoColor=white)\n![Gemini 2.5 Flash](https://img.shields.io/badge/AI-Gemini_2.5_Flash-8E44AD?logo=google-gemini\u0026logoColor=white)\n\nThis repository provides an interface for ROS 2 navigation utilizing voice commands. The system processes natural language input via Google's Gemini 2.5 Flash model and visualizes the robot's state on a React-based dashboard.\n\n## System Architecture\n\nThe interaction pipeline consists of four primary components:\n1. **Frontend**: React application built with Vite, utilizing `MediaRecorder` for audio capture.\n2. **Backend**: FastAPI server that coordinates API requests.\n3. **Speech processing**: OpenAI Speech-to-Text transcribes the audio input.\n4. **Command synthesis**: Gemini 2.5 Flash interpolates the transcript and outputs structured ROS 2 `Twist` messages.\n5. **Execution**: The `rosbridge_server` transmits the parsed commands to the ROS 2 Humble environment (Nav2, Cartographer, AMCL).\n\nThe system includes a regex-based fallback parser to ensure operational continuity during API service interruptions.\n\n```mermaid\ngraph TD\n    User((User)) --\u003e|Voice| Frontend[React Dashboard]\n    Frontend --\u003e|Audio Blob| Backend[FastAPI Server]\n    Backend --\u003e|STT| OpenAI[OpenAI Speech-to-Text]\n    OpenAI --\u003e|Transcript| Backend\n    Backend --\u003e|Gemini 2.5 Flash| AI[Google AI Studio]\n    AI --\u003e|JSON Action| Backend\n    Backend --\u003e|Response| Frontend\n    Frontend --\u003e|WebSocket| Bridge[rosbridge_server]\n    Bridge --\u003e|/cmd_vel| Robot[ROS 2 Robot/Sim]\n```\n\n## Installation and Configuration\n\n### Prerequisites\n- Ubuntu 22.04 LTS (or WSL2)\n- ROS 2 Humble Hawksbill (Desktop Install)\n- Node.js (or Bun)\n- Python 3.10+\n\n### Setup Instructions\n\n1. **Repository mapping**\n   ```bash\n   git clone https://github.com/howdoiusekeyboard/ros2_navigation_project.git\n   cd ros2_navigation_project\n   ```\n\n2. **Environment configuration**\n   Populate the backend environment file with required API keys.\n   ```bash\n   cp backend/.env.example backend/.env\n   # Add OPENAI_API_KEY and GEMINI_API_KEY to the .env file\n   ```\n\n3. **System initialization**\n   Execute the provided bash script to instantiate the simulation, backend server, and frontend dashboard concurrently.\n   ```bash\n   ./start_robot_dashboard.sh\n   ```\n\n4. **Interface access**\n   The dashboard hosts on `http://localhost:5173`. A Chromium-based browser is required for full `MediaRecorder` compatibility.\n\n## Operation Guidelines\n\n1. Verify connection state via the dashboard (\"Connected to ROS 2\").\n2. Initiate voice capture utilizing the interface microphone control.\n3. Issue spatial or directional commands (e.g., \"rotate left 90 degrees\", \"proceed forward 2 meters\", \"halt\").\n4. Alternatively, use the text input field for manual command insertion.\n\n## Project Structure\n\n- `src/`: ROS 2 packages integrating Cartographer and Nav2 configurations.\n- `backend/`: Python backend utilizing FastAPI.\n- `project/`: React-based interactive dashboard.\n- `scripts/`: Operational scripts for initialization and debugging.\n\n## Documentation References\n\n- [SETUP.md](SETUP.md): Comprehensive environment preparation guide.\n- [RECOVERY.md](RECOVERY.md): Guidelines for restoring the system from failure states.\n\n## License\n\nThis project operates under the MIT License. Reference the [LICENSE](LICENSE) file for exact parameters.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhowdoiusekeyboard%2Fros2_navigation_project","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhowdoiusekeyboard%2Fros2_navigation_project","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhowdoiusekeyboard%2Fros2_navigation_project/lists"}