https://github.com/extelligence-ai/bagel
Chat with your robotics, drone, and IoT data — ChatGPT for the physical world.
https://github.com/extelligence-ai/bagel
ardupilot betaflight drone mcp px4-autopilot robotics ros2
Last synced: 13 days ago
JSON representation
Chat with your robotics, drone, and IoT data — ChatGPT for the physical world.
- Host: GitHub
- URL: https://github.com/extelligence-ai/bagel
- Owner: Extelligence-ai
- License: apache-2.0
- Created: 2025-06-27T23:40:24.000Z (4 months ago)
- Default Branch: stage
- Last Pushed: 2025-09-30T14:52:14.000Z (13 days ago)
- Last Synced: 2025-09-30T16:31:22.149Z (13 days ago)
- Topics: ardupilot, betaflight, drone, mcp, px4-autopilot, robotics, ros2
- Language: Python
- Homepage:
- Size: 83.6 MB
- Stars: 338
- Watchers: 10
- Forks: 22
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- awesome-ChatGPT-repositories - bagel - Understand robotics and drone data with MCP and LLMs. ChatGPT for Physical AI. (Chatbots)
README
Unlike cat videos, most data are trapped in the physical world. Bagel unlocks them and
**lets you chat with your physical data**—just like you do with ChatGPT. For example:
> Is my IMU sensor overheating?
Can’t wait to try it out? 👉 Check out the [Quickstart](#️-quickstart).
### 🥯 Key Features
- **Ask in plain language**: No deep domain expertise needed.
- **Transparent calculations**: Deterministic SQL queries. No black-box LLM math.
- **Broad LLM support**: Claude Code, Gemini, Cursor, Codex, and more.
- **Dockerized environments**: No local dependencies required.
- **Extensible capabilities**: Bagel can learn [new tricks](#-teach-bagel-a-new-trick).
- **Wide format coverage**: Missing your data format? [Open a ticket](https://github.com/Extelligence-ai/bagel/issues).
### ✅ Supported Data Formats
| Industry | Formats |
| ------------ | -------------------------- |
| **Robotics** | ROS1, ROS2 |
| **Drones** | PX4, ArduPilot, Betaflight |
| **IoT** | Coming soon... |
## 💬 What Can I Prompt?
You can ask Bagel almost anything. For example:
> What’s the correlation between current and voltage in the `/spot/status/battery_states` topic?
> I think the robot hit a pothole. Can you check for sudden deceleration on the z-axis to confirm?
> Can you help me tune the PID of my drone?
Time to put Bagel to the test: can it catch a drone doing barrel rolls? Spoiler: 🎉 It totally can.
## 💡 How Bagel Works
When you ask a question, Bagel analyzes your data source’s **metadata** and **topics** to
build a high-level understanding.
Based on your prompt, if further inspection is needed, Bagel identifies the most relevant topics
and **interprets their meaning and structure**. Bagel then writes the relevant topic messages
to an **Apache Arrow file** and uses **DuckDB** to generate and execute queries against it.
This process is repeated as needed, running new queries until Bagel finds the best answer
to your question.
LLMs excel at language but struggle with math. Bagel overcomes this by generating **deterministic**
DuckDB SQL queries. These queries are displayed for you to **audit**, and you can guide Bagel to
correct any errors.
## ⚡️ Quickstart
#### 📋 Prerequisites
Install [Docker Desktop](https://docs.docker.com/get-started/get-docker/), then clone this repo:
```bash
git clone https://github.com/Extelligence-ai/bagel.git; cd bagel
```
#### 🐳 Run with Docker
Open [compose.yaml](./compose.yaml) and pick a service based on your need (e.g., `ros2-kilted`).
```bash
docker compose run --service-ports ros2-kilted
```
> [!TIP]
> To give Bagel access to your local files, edit `compose.yaml` before starting Docker:
> uncomment and update the `volumes` section under your chosen service.
Your terminal should show output similar to:
```
INFO: Started server process [86]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
```
#### 🔗 Connect to an LLM
Once the Bagel MCP server is running, connect it to your preferred LLM.
Bagel should work with any MCP-enabled LLM, though some setup may be required.
We’ve tested several LLMs—expand the runbooks below for setup instructions.
📚 Setup runbooks for tested LLMs
- [Claude Code](./doc/runbooks/setup/claude_code.md)
- [Gemini CLI](./doc/runbooks/setup/gemini_cli.md)
- [Codex](./doc/runbooks/setup/codex.md)
- [Cursor](./doc/runbooks/setup/cursor.md)
- [Copilot](./doc/runbooks/setup/copilot.md)
Can't find your favorite LLM? [Open a ticket](https://github.com/Extelligence-ai/bagel/issues),
and we’ll create a runbook for it.
#### 🚀 Launch and Prompt
Now you're ready to launch your LLM and begin prompting. For example:
> Summarize the metadata of the ROS2 bag "./data/sample/ros2/mcap".
## 🐶 Teach Bagel a New Trick
Bagel learns new capabilities through [POML](https://microsoft.github.io/poml/latest/)
files—a structured set of instructions that describe a “trick,”
such as [computing latency statistics](./src/agent/diagnose/latency.poml).
#### ✍️ Create a .poml file
For example, let’s define `./src/agent/examples/woof.poml`.
```poml
Count the topics in the data source.
If the count is odd, say "woof", else say "meow".
Return the sound, the topic count, and a few cute emojis. Nothing else.
```
#### 🗣️ Use the capability
Prompt Bagel:
> Run the POML capability "./src/agent/examples/woof.poml" on the ROS2 bag "./data/sample/ros2/mcap".
Result:
```
meow 🐱 4 topics 🐱💤🎯
```
## 🫶 Contributing
We’d love your help! The easiest way to support the project is by giving it a ⭐ on GitHub.
Other great ways to contribute:
- Request new features
- Report bugs
- Improve documentation
- Add new capabilities
Before contributing, please review the [guidelines](./CONTRIBUTING.md).
Join the conversation in our [Discord server](https://discord.com/invite/QJDwuDGJsH) —
we hang out there regularly.
## 📄 License
Bagel is open source under the [Apache License 2.0](./LICENSE).