{"id":17938846,"url":"https://github.com/Auromix/ROS-LLM","last_synced_at":"2025-03-24T10:31:43.626Z","repository":{"id":176012453,"uuid":"653893086","full_name":"Auromix/ROS-LLM","owner":"Auromix","description":"ROS-LLM is a framework designed for embodied intelligence applications in ROS. It allows natural language interactions and leverages Large Language Models (LLMs) for decision-making and robot control. With an easy configuration process, this framework allows for swift integration, enabling your robot to operate with it in as little as ten minutes.","archived":false,"fork":false,"pushed_at":"2023-07-10T10:32:57.000Z","size":318,"stargazers_count":506,"open_issues_count":5,"forks_count":65,"subscribers_count":9,"default_branch":"ros2-humble","last_synced_at":"2024-10-29T00:06:35.808Z","etag":null,"topics":["embodied-intelligence","gpt","llm","ros"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Auromix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-15T01:19:11.000Z","updated_at":"2024-10-28T23:02:36.000Z","dependencies_parsed_at":"2023-07-26T07:01:01.510Z","dependency_job_id":null,"html_url":"https://github.com/Auromix/ROS-LLM","commit_stats":null,"previous_names":["auromix/ros-llm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Auromix%2FROS-LLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Auromix%2FROS-LLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Auromix%2FROS-LLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Auromix%2FROS-LLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Auromix","download_url":"https://codeload.github.com/Auromix/ROS-LLM/tar.gz/refs/heads/ros2-humble","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245252302,"owners_count":20585007,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embodied-intelligence","gpt","llm","ros"],"created_at":"2024-10-29T00:06:23.836Z","updated_at":"2025-03-24T10:31:43.617Z","avatar_url":"https://github.com/Auromix.png","language":"Python","funding_links":[],"categories":["Research-Grade Frameworks","ROS2","Embodied AI"],"sub_categories":["ROS2 Packages"],"readme":"[![Official](https://img.shields.io/badge/Official%20-Auromix-blue?style=flat\u0026logo=world\u0026logoColor=white)](https://github.com/Auromix) \u0026nbsp;\n[![ROS2 VERSION](https://img.shields.io/badge/ROS-ROS%202%20Humble-brightgreen)](http://docs.ros.org/en/humble/index.html) \u0026nbsp;\n[![Ubuntu VERSION](https://img.shields.io/badge/Ubuntu-22.04-green)](https://ubuntu.com/) \u0026nbsp; [![LICENSE](https://img.shields.io/badge/license-Apache--2.0-informational)](https://github.com/Auromix/ROS-LLM/blob/ros2-humble/LICENSE) \u0026nbsp;\n[![GitHub Repo stars](https://img.shields.io/github/stars/Auromix/ROS-LLM?style=social)](https://github.com/Auromix/ROS-LLM/stargazers) \u0026nbsp;\n[![Twitter Follow](https://img.shields.io/twitter/follow/Hermanye233?style=social)](https://twitter.com/Hermanye233) \u0026nbsp;\n# ROS-LLM\nThe ROS-LLM project is a ROS framework for embodied intelligence applications. It enables natural language interactions and large model-based control of robot motion and navigation for any robot operating on ROS.\n\nROS-LLM empowers you to utilize functionalities based on Large Language Models, such as GPT-4 and ChatGPT, for robot decision-making and control. \n\nThis framework is designed to be easy to extend. By simply providing a function interface for your robot, following the provided example, you can integrate and use ROS-LLM within ten minutes. \n\nROS-LLM offers a simple solution for quickly creating interactive and control experiences with any robot.\n\n![Related Schematics](llm_imgs/flow_diagram.png)\n## 🚀 Features\n\n- 🤖 **ROS Integration**: Smoothly interacts with the Robot Operating System (ROS) for expansive robotic control. \n\n- 🧠 **Large Language Models Support**: Leverages GPT-4 and ChatGPT for enhanced decision-making and task management.\n\n- 🗣️ **Natural Interaction**: Facilitates intuitive communication with robots through conversational engagement.\n\n- 🔄 **Flexible Control**: Utilizes LLM-based systems for tasks such as motion and navigation based on language model interpretation.\n\n- 🔌 **Simplified Extensibility**: Provides an easy interface for seamless robot function integration. \n\n- 🛠️ **Quick Development**: Creates interactive robot control experiences swiftly, sometimes in under ten minutes.\n\n- 📚 **Instructional Examples**: Offers comprehensive tutorials and examples for easier understanding and implementation.\n\n- 🗃️ **History Storage**: Retains local chat histories for convenient review and reference.\n\n\n\n## 🔥 Quickstart Guide\n\nFollow the instructions below to set up ROS-LLM:\n\n**1. Clone the Repository:**\n\nUse the command below to clone the repository.\n```bash\ngit clone https://github.com/Auromix/ROS-LLM.git\n```\n\n**2. Install Dependencies:**\n\nNavigate to the `llm_install` directory and execute the installation script.\n```bash\ncd ROS-LLM/llm_install\nbash dependencies_install.sh\n```\n\n**3. Configure OpenAI Settings:**\n\nIf you don't have an OpenAI API key, you can obtain one from [OpenAI Platform](https://platform.openai.com). Use the script below to configure your OpenAI API key.\n```bash\ncd ROS-LLM/llm_install\nbash config_openai_api_key.sh\n```\n\n**4. Configure AWS Settings (Optional):**\n\nFor cloud natural interaction capabilities, configure the AWS settings. If you prefer to use local ASR, this step can be skipped.\n\nFor low-performance edge embedded platforms, it is recommended to use ASR cloud services to reduce computing pressure, and for high-performance personal hosts, it is recommended to use local ASR services to speed up response\n\n```bash\ncd ROS-LLM/llm_install\nbash config_aws.sh\n```\n\n**4. Configure OpenAI Whisper Settings (Optional):**\n\nFor local natural interaction capabilities, configure the OpenAI Whisper settings. If you prefer to use cloud ASR, this step can be skipped.\n\nFor low-performance edge embedded platforms, it is recommended to use ASR cloud services to reduce computing pressure, and for high-performance personal hosts, it is recommended to use local ASR services to speed up response\n```bash\npip install -U openai-whisper\npip install setuptools-rust\n```\n\n**5. Build the Workspace:**\n\nNavigate to your workspace directory and build the workspace.\n```bash\ncd \u003cyour_ws\u003e\nrosdep install --from-paths src --ignore-src -r -y  # Install dependencies\ncolcon build --symlink-install\n```\n\n**6. Run the Demo:**\n\nSource the setup script and launch the Turtlesim demo with cloud ASR.\n```bash\nsource \u003cyour_ws\u003e/install/setup.bash\nros2 launch llm_bringup chatgpt_with_turtle_robot.launch.py\n```\nstart listening\n```bash\nros2 topic pub /llm_state std_msgs/msg/String \"data: 'listening'\" -1\n```\n\n## ⚙️ Configure Your Own Robot (Optional)\n\nTo use the framework with your own robot, modify the `llm_robot` and `llm_config` packages to suit your robot's specifications. This allows you to customize the behavior of your robot.\n\n\n## 🧑‍💻 Future Development Plans\n\nWe are continuously striving to enhance ROS-LLM to better serve the developers and roboticists in the community. Below are the key developments we plan to undertake in the upcoming updates:\n\n- [ ] Agent Mechanism\n\nAdding an agent mechanism allows long sequence tasks to be well divided.\n\n- [ ] Feedback Channel from External Functions\n\nWe plan to add a feedback mechanism for the robot to receive information from external functions. This would significantly assist model-based decision-making processes.\n\n- [ ] Navigation Interface\n\nA new interface for robot navigation is also in the pipeline. It will enable the utilization of this framework in navigation-oriented tasks.\n\n- [ ] Sensor Input Interface\n\nThe addition of other sensor input interfaces is another major development. This will incorporate environmental perception into model decision premises, preparing for functionalities such as obstacle avoidance.\n\n- [ ] Integration with Vision-based Models like Palm-e\n\nWe aim to extend the capabilities of ROS-LLM by integrating models that allow for visual input, like Palm-e. This would enable the use of advanced computer vision technologies for better environment interaction.\n\n- [ ] Continuous Optimization\n\nLast but not least, we will focus on continuous optimization of the framework. We are committed to improving the rationality and extensibility of ROS-LLM to make it easier for developers to customize and extend the framework according to their needs.\n\nKeep an eye on this repo for updates. Your suggestions and contributions are always welcome!\n\n\n\n## 🙋 To user\nIf you find this project useful, please consider giving it a ⭐️ star on GitHub! Your support helps us improve the project and encourages further development. Don't forget to also share it with your friends and colleagues who might it beneficial. Thank you for your support! \n\n## 🏆 Contributing\nContributions are welcome! Please read the [contributing guidelines](CONTRIBUTING.md) before submitting a pull request.\n\n\n\n## 🪪 License\n```\nCopyright 2023 Herman Ye @Auromix\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n    http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License. \n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAuromix%2FROS-LLM","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAuromix%2FROS-LLM","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAuromix%2FROS-LLM/lists"}