{"id":39799209,"url":"https://github.com/automatika-robotics/embodied-agents","last_synced_at":"2026-01-18T12:28:04.209Z","repository":{"id":255640710,"uuid":"687568766","full_name":"automatika-robotics/embodied-agents","owner":"automatika-robotics","description":"EmbodiedAgents is a fully-loaded ROS2 based framework for creating interactive physical agents that can understand, remember, and act upon contextual information from their environment.","archived":false,"fork":false,"pushed_at":"2026-01-10T12:37:39.000Z","size":8532,"stargazers_count":43,"open_issues_count":0,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2026-01-11T03:45:22.520Z","etag":null,"topics":["deeplearning","embodied-agent","embodied-ai","generative-ai","godel-machine","llm","machine-learning","multimodal","ollama","physical-ai","roboml","robotics","ros2","vllm"],"latest_commit_sha":null,"homepage":"https://automatika-robotics.github.io/embodied-agents/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/automatika-robotics.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-09-05T16:11:25.000Z","updated_at":"2026-01-09T23:55:48.000Z","dependencies_parsed_at":"2024-09-15T10:24:52.501Z","dependency_job_id":"40c1be90-a21d-4c11-964e-c5588f775c85","html_url":"https://github.com/automatika-robotics/embodied-agents","commit_stats":null,"previous_names":["automatika-robotics/ros-agents","automatika-robotics/embodied-agents"],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/automatika-robotics/embodied-agents","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automatika-robotics%2Fembodied-agents","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automatika-robotics%2Fembodied-agents/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automatika-robotics%2Fembodied-agents/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automatika-robotics%2Fembodied-agents/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/automatika-robotics","download_url":"https://codeload.github.com/automatika-robotics/embodied-agents/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automatika-robotics%2Fembodied-agents/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28536002,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T10:13:46.436Z","status":"ssl_error","status_checked_at":"2026-01-18T10:13:11.045Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deeplearning","embodied-agent","embodied-ai","generative-ai","godel-machine","llm","machine-learning","multimodal","ollama","physical-ai","roboml","robotics","ros2","vllm"],"created_at":"2026-01-18T12:28:03.578Z","updated_at":"2026-01-18T12:28:04.184Z","avatar_url":"https://github.com/automatika-robotics.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"docs/_static/EMBODIED_AGENTS_DARK.png\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"docs/_static/EMBODIED_AGENTS_LIGHT.png\"\u003e\n  \u003cimg alt=\"EmbodiedAgents Logo.\" src=\"docs/_static/EMBODIED_AGENTS_DARK.png\"\u003e\n\u003c/picture\u003e\n\u003cbr/\u003e\n\n🇨🇳 [简体中文](docs/README.zh.md) | 🇯🇵 [日本語](docs/README.ja.md)\n\n**_EmbodiedAgents_** is a fully-loaded ROS2 based framework for creating interactive physical agents that can understand, remember, and act upon contextual information from their environment.\n\n- **Production Ready Physical Agents:** Designed to be used with autonomous robot systems that operate in real world dynamic environments. _EmbodiedAgents_ makes it simple to create systems that make use of Physical AI.\n- **Intuitive API**: Simple pythonic API to utilize local or cloud based ML models (specifically **Multimodal LLMs** and other **transformer based architectures**) on robots, with all the benefits of component lifecycle management, health monitoring and fallback mechanisms to make your agents robust.\n- **Self-referential and Event Driven**: An agent created with _EmbodiedAgents_ can start, stop or reconfigure its own components based on internal and external events. For example, an agent can change the ML model for planning based on its location on the map or input from the vision model. _EmbodiedAgents_ makes it simple to create agents that are self-referential [Gödel machines](https://en.wikipedia.org/wiki/G%C3%B6del_machine).\n- **Semantic Memory**: Integrates vector databases, semantic routing and other supporting components to quickly build arbitrarily complex graphs for agentic information flow. No need to utilize bloated \"GenAI\" frameworks on your robot.\n- **Made in ROS2**: Utilizes ROS2 as the underlying distributed communications backbone. Theoretically, all devices that provide a ROS2 package can be utilized to send data to ML models, with callbacks implemented for most commonly used data types and infinite extensibility.\n\nJoin our [Discord](https://discord.gg/B9ZU6qjzND) 👾\n\nCheckout [Installation Instructions](https://automatika-robotics.github.io/embodied-agents/installation.html) 🛠️\n\nGet started with the [Quickstart Guide](https://automatika-robotics.github.io/embodied-agents/quickstart.html) 🚀\n\nGet familiar with [Basic Concepts](https://automatika-robotics.github.io/embodied-agents/basics/index.html) 📚\n\nDive right in with [Example Recipes](https://automatika-robotics.github.io/embodied-agents/examples/index.html) ✨\n\n## Installation 🛠️\n\n### Install a model serving platform\n\nThe core of _EmbodiedAgents_ is agnostic to model serving platforms. It currently supports [Ollama](https://ollama.com), [RoboML](https://github.com/automatika-robotics/robo-ml) and any platform or cloud provider with an OpenAI compatible API (e.g. [vLLM](https://github.com/vllm-project/vllm), [lmdeploy](https://github.com/InternLM/lmdeploy) etc.). Please install either of these by following the instructions provided by respective projects. Support for new platforms is being continuously added. If you would like to support a particular platform, please open an issue/PR.\n\n### Install _EmbodiedAgents_ (Ubuntu)\n\nFor ROS versions \u003e= _humble_, you can install _EmbodiedAgents_ with your package manager. For example on Ubuntu:\n\n`sudo apt install ros-$ROS_DISTRO-automatika-embodied-agents`\n\nAlternatively, grab your favorite deb package from the [release page](https://github.com/automatika-robotics/embodied-agents/releases) and install it as follows:\n\n`sudo dpkg -i ros-$ROS_DISTRO-automatica-embodied-agents_$version$DISTRO_$ARCHITECTURE.deb`\n\nIf the attrs version from your package manager is \u003c 23.2, install it using pip as follows:\n\n`pip install 'attrs\u003e=23.2.0'`\n\n### Install _EmbodiedAgents_ from source\n\n#### Get Dependencies\n\nInstall python dependencies\n\n```shell\npip install numpy opencv-python-headless 'attrs\u003e=23.2.0' jinja2 httpx setproctitle msgpack msgpack-numpy platformdirs tqdm websockets onnxruntime\n```\n\nDownload Sugarcoat🍬\n\n```shell\ngit clone https://github.com/automatika-robotics/sugarcoat\n```\n\n#### Install _EmbodiedAgents_\n\n```shell\ngit clone https://github.com/automatika-robotics/embodied-agents.git\ncd ..\ncolcon build\nsource install/setup.bash\npython your_script.py\n```\n\n## Quick Start 🚀\n\nUnlike other ROS package, _EmbodiedAgents_ provides a pure pythonic way of describing the node graph using [Sugarcoat🍬](https://www.github.com/automatika-robotics/sugarcoat). Copy the following recipe in a python script and run it.\n\n```python\nfrom agents.clients.ollama import OllamaClient\nfrom agents.components import VLM\nfrom agents.models import OllamaModel\nfrom agents.ros import Topic, Launcher\n\n# Define input and output topics (pay attention to msg_type)\ntext0 = Topic(name=\"text0\", msg_type=\"String\")\nimage0 = Topic(name=\"image_raw\", msg_type=\"Image\")\ntext1 = Topic(name=\"text1\", msg_type=\"String\")\n\n# Define a model client (working with Ollama in this case)\n# OllamaModel is a generic wrapper for all Ollama models\nllava = OllamaModel(name=\"llava\", checkpoint=\"llava:latest\")\nllava_client = OllamaClient(llava)\n\n# Define a VLM component (A component represents a node with a particular functionality)\nmllm = VLM(\n    inputs=[text0, image0],\n    outputs=[text1],\n    model_client=llava_client,\n    trigger=[text0],\n    component_name=\"vqa\"\n)\n# Additional prompt settings\nmllm.set_topic_prompt(text0, template=\"\"\"You are an amazing and funny robot.\n    Answer the following about this image: {{ text0 }}\"\"\"\n)\n# Launch the component\nlauncher = Launcher()\nlauncher.add_pkg(components=[mllm])\nlauncher.bringup()\n```\n\nAnd just like that we have an agent that can answer questions like **'What do you see?'**. Checkout the [Quick Start Guide](https://automatika-robotics.github.io/embodied-agents/quickstart.html) to learn more about how components and models work together.\n\n## Complex Physical Agents\n\nThe quickstart example above is just an amuse-bouche of what is possible with _EmbodiedAgents_. In _EmbodiedAgents_ we can create arbitrarily sophisticated component graphs. And furthermore our system can be configured to even change or reconfigure itself based on events internal or external to the system. Check out the code for the following agent [here](https://automatika-robotics.github.io/embodied-agents/examples/complete.html).\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"docs/_static/complete_dark.png\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"docs/_static/complete_light.png\"\u003e\n  \u003cimg alt=\"Elaborate Agent\" src=\"docs/_static/complete_dark.png\"\u003e\n\u003c/picture\u003e\n\n## Dynamic Web UI for EmbodiedAgent Recipes\n\nLeveraging the power of the underlying [**Sugarcoat**](https://github.com/automatika-robotics/sugarcoat) framework, **_EmbodiedAgents_** offers a **fully dynamic, auto-generated Web UI** for every recipe. This feature is built with **FastHTML** and eliminates manual GUI development, instantly providing a responsive interface for control and visualization.\n\nThe UI automatically creates:\n* Settings interfaces for all the components used in the recipe.\n* Real-time data visualizations and controls for component inputs/outputs.\n* WebSocket-based data streaming for all supported message types.\n\n### Example: VLM Agent UI\n\nA full interface is automatically generated for a VLM Q\\\u0026A agent (similat to the Quick Start example), providing simple controls for settings and displaying real-time text input/output.\n\n\u003cp align=\"center\"\u003e\n\u003cpicture align=\"center\"\u003e\n  \u003cimg alt=\"EmbodiedAgents UI Example GIF\" src=\"docs/_static/agents_ui.gif\" width=\"60%\"\u003e\n\u003c/picture\u003e\n\u003c/p\u003e\n\n## Copyright\n\nThe code in this distribution is Copyright (c) 2024 [Automatika Robotics](https://automatikarobotics.com/) unless explicitly indicated otherwise.\n\n_EmbodiedAgents_ is made available under the MIT license. Details can be found in the [LICENSE](LICENSE) file.\n\n## Contributions\n\n_EmbodiedAgents_ has been developed in collaboration between [Automatika Robotics](https://automatikarobotics.com/) and [Inria](https://inria.fr/). Contributions from the community are most welcome.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautomatika-robotics%2Fembodied-agents","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautomatika-robotics%2Fembodied-agents","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautomatika-robotics%2Fembodied-agents/lists"}