{"id":28516715,"url":"https://github.com/mit-spark/omniplanner","last_synced_at":"2025-08-19T00:08:28.820Z","repository":{"id":287473868,"uuid":"962330086","full_name":"MIT-SPARK/Omniplanner","owner":"MIT-SPARK","description":"Infrastructure for composing multiple types of planning paradigms. ","archived":false,"fork":false,"pushed_at":"2025-08-07T00:31:12.000Z","size":528,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-08-07T02:31:52.344Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MIT-SPARK.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-08T02:03:44.000Z","updated_at":"2025-08-07T00:31:15.000Z","dependencies_parsed_at":"2025-05-06T19:30:49.896Z","dependency_job_id":"6420de42-8375-45b1-a800-ebfb9a9cbb69","html_url":"https://github.com/MIT-SPARK/Omniplanner","commit_stats":null,"previous_names":["mit-spark/omniplanner"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MIT-SPARK/Omniplanner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FOmniplanner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FOmniplanner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FOmniplanner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FOmniplanner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MIT-SPARK","download_url":"https://codeload.github.com/MIT-SPARK/Omniplanner/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FOmniplanner/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271078587,"owners_count":24695473,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-09T04:12:31.966Z","updated_at":"2025-08-19T00:08:28.811Z","avatar_url":"https://github.com/MIT-SPARK.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Omniplanner\n\nOmniplanner provides an interface to solving DSG-grounded planning problems\nwith a variety of solvers and grounding mechanisms. The goal is to enable\nmodular design of command grounding and planning implementations, and a clean\nhook for transforming the output of a planner into robot-compatible input.\n\n## Motivation\n\nThere are many kinds of planning problems we might want a robot to solve. For\nexample, fully-observed Markovian tasks are well-modeled by PDDL, temporal\nconstraints may lead us to used LTL or STL, some problems reduce to\nspecial-purpose solvers like a TSP, if there is state or observation\nuncertainty we may need to consider POMDP formulations. Each of these problem\ntypes has a large research community surrounding it, and there are many solvers\nto choose from.\n\nIt is claimed that these problems are hard, but existing solvers do well enough\non instances that arise in practice. However, when we take a real robot and try\nto solve a real user's task, we run into difficulties that are orthogonal to\nthe hardness considered *within* each of these domains:\n\n*  How do we ground the user's instructions to symbols that the robot has (or\n   has not) perceived?\n\n* How do we decide which kind of problem formulation best captures the desired\n  task?\n\n* What is the relationship between the output of a planner and the API call we\n  make to get a robot moving?\n\n* How do we connect the perception system, planner implementation, and\n  downstream robot execution into a production-ready pipeline that supports\nconfiguration, logging, and monitoring without obscuring the structure of the\nunderlying planner?\n\nOmniplanner is a planning system that aims to separate, as generically as\npossible, concerns related to problem grounding, problem solving,\nrobot-specific plan post-processing, and runtime monitoring.\n\n## Architecture\n\nAt the highest level, the Omniplanner architecture can be considered in two\nhalves: The Omniplanner ROS node that provides an interface for combining\nplanning commands, scene representations, and robot commands, and the\nOmniplanner non-ROS code, which defines the generic interfaces that a planner\nor language grounding system needs to implement to work with the Omniplanner\nnode. Integrating a new planner with Omniplanner requires interacting with\nalmost none of the Omniplanner ROS code. Once you have defined the necessary\nplanning interface functions (the subject of the next section) for your\nplanner, all you need to do is create a small interface class the describes how\nto turn a ROS message into your planning domain.\n\n![system architecture](./docs/pipeline.png)\n\nThis system architecture presents the logical order of the Omniplanner planning\nsystem. This representation is how you should think of Omniplanner, although it\nis not a literal representation of the stack trace you will see, due to how the\nROS callbacks are implemented. The Omniplanner ROS plugins subscribe to a ROS\ntopic. It is up to you to decide how to turn a ROS message into a description\nof a planning problem that is solvable by an Omniplanner plugin. The resulting\nproblem description is then passed through the planning pipeline, which first\ngrounds the problem and then makes a plan. `ground_problem` and `make_plan` are\ntwo generic functions that you can extend to add functionality. The sequence of\ngrounding and planning steps (yellow boxes) is decided at *runtime* based on\nthe type of the original planning request and outputs of the intermediate\ngrounding/planning functions. After a plan has been found it gets transformed\ninto a sequence of actions that are aligned with a given robot's interface\n(although this last part of the pipeline is somewhat less polished than the\nearlier parts).\n\n### Examples\n\nWe provide [several examples of the non-ROS\nfunctionality](https://github.com/MIT-SPARK/Omniplanner/blob/feature/docs/omniplanner/examples)\nbased on Omniplanner's built-in planning options. These examples should run\nout-of-the-box, and demonstrate the construction of a planning problem and how\nto run the planning pipeline. Running the natural language example may require\nproviding and setting an API key environment variable.\n\n* (TODO) Omniplanner example roslaunch script and config and scene graph\n  publishing\n* (TODO) example integrating plugin from external repository\n\n\n### Omniplanner Planner Dispatch\n\nImplementing a planner plugin requires implementing Omniplanner's\n`ground_problem` and `make_plan` interface. At planning time, various planning\nand grounding methods can be combined through dispatching on the input and\noutput types of these modules. You need to implement [an interface like\nthis](https://github.com/MIT-SPARK/Omniplanner/blob/main/omniplanner/src/omniplanner/goto_points.py)\nto give the omniplanner node a hook into your planning and grounding behavior.\n\nNote that you do not necessarily need to implement the full grounding and\nplanning pipeline for a new plugin. You can leverage existing plugins. For\nexample, if you want to add a new method for grounding user commands and scene\nrepresentations to PDDL, you can implement `ground_problem`, but let an\nexisting PDDL solver plugin solve the actual problem.\n\n### Half of what I say is meaningless\n\nOmniplanner's ability to flexibly blend different methods of grounding and\nplanning depends on dynamic multiple dispatch. Multiple dispatch is a feature\nof a programming language that resolves which version of a function to call\ndepending on the runtime types of its arguments. Note that this is distinct\nfrom overloaded functions in languages like C++, and single (dynamic) dispatch\nthat you might find in a language like Java (and C++).\n\nWith multiple dispatch, you can have a single function (such as\n`ground_problem`) with multiple implementations. The implementation to use when\nthe function is called is determined by the types of the arguments. C++ style\nfunction overloading would require that you know the types at compile time. C++\nor Java style polymorphism allows you to define a function whose implementation\ndepends on the class it is attached to which is resolved at runtime, you are\nlimited to dynamic dispatch on only a single argument's type, and you are\nforced to make ontological commitments about inheritance structure.\n\nOmniplanner uses `plum`, [a multiple dispatch library for\nPython](https://github.com/beartype/plum). `plum` implements Julia's multiple\ndispatch semantics (including its type coercion system centered around the\n`convert` function, although we do not utilize the type coercion here yet).\nWhat this means in practice is that if you decorate your function with\n`@dispatch`, then when you call that function the implementation that is used\nwill be selected based on matching the runtime types of the arguments with the\ntype annotations in the function's signature.\n\n\n### Omniplanner ROS plugin\n\nThe second thing you need to do is implement the ROS hook. This entails writing\n[a class like\nthis](https://github.com/MIT-SPARK/Omniplanner/blob/main/omniplanner_ros/src/omniplanner_ros/goto_points_ros.py)\nwith a `get_plan_callback` function. You also need to implement the config\nregistration at the bottom of the file to enable constructing the plugin based\non the plugin YAML definition.\n\nThe planner plugins that Omniplanner loads are defined in a yaml file like\n[this one](https://github.com/MIT-SPARK/Awesome-DCIST-T4/blob/main/dcist_launch_system/config/spot_prior_dsg/omniplanner_plugins.yaml). The omniplanner node\ntakes the path to this [as a rosparam](https://github.com/MIT-SPARK/Awesome-DCIST-T4/blob/main/dcist_launch_system/config/spot_prior_dsg/omniplanner_node.yaml).\n\n## Generic Functors Pattern\n\nWhile the generic planning pipeline helps us separate concerns over grounding\nand planning, and it enables flexible extensions for mixing and matching parts\nof the pipeline, it does not automatically address the separation between\noperational real-robot concerns (for example, tracking a robot's name or IP\naddress) and the fundamental planning problem being solved (which probably\ndoesn't care about the robot's name).\n\n### The Problem\nIt is likely that the two \"ends\" of the Omniplanner pipeline -- the ROS plugin\nthat transforms a goal into a planning problem, and the plan compilation that\nturns the output plan into something the robot can execute -- will be robot\nspecific (or at least specific to the rest of your autonomy pipeline). The plan\ncompilation step may depend on information that the ROS plugin receives. This\ncreates a problem: how do we pass new information from the beginning of the\npipeline to the end of the pipeline without needing to change any of\nintermediate functions that get called?\n\n### The Functors\nThe answer is a simple technique, usually associated with functional\nprogramming languages like Haskell, called a `Functor`. Functors sound\ncomplicated, but they are actually pretty simple. In the context of\nprogramming, a functor is a \"container\" object with a well-defined way of\napplying a function that doesn't known anything about your container to the\ncontents of your container.\n\nLoosely, a class F is a functor if there is a function `fmap` that transforms a\nfunction into a new function that can be applied to an instance of F. Let's\nexamine the type signature of `fmap`:\n\n`fmap :: (a -\u003e b) -\u003e F a -\u003e F b`\n\nWe will consider two ways of parsing this. First, consider `fmap` as a function\nthat takes in a function that operates on type `a` and returns type `b`. `fmap`\nwill return a new function that operates on type `F a` and returns type `F b`.\nThis is the mostly useful way of *thinking* about `fmap`. However, the normal\nway that we use `fmap` is as a function that takes two arguments, a function\nand an instance of the functor `F a`, and returns `F b`.\n\n### The Solution\nWe can get a lot of flexibility in the Omniplanner pipeline by adding a little\nbit of extra support for functors. If the problem passed to `make_plan` doesn't\nmatch any other implementation, if the problem is a functor, then it will try\nto `fmap` `make_plan` across the problem. This behavior is incredibly powerful,\nbecause it means that we can have an arbitrarily complicated container class\n(presumably with a bunch of metadata that we need to carry along with us) that\nwe pass to the planning pipeline, and the pipeline will solve the problem even\nthough it doesn't know anything about the structure of this class. The only\nrequirement is that the creator of this complicated container class defined\n`fmap`.\n\n### Our Implementation\n\nFor our pipeline to optionally fmap when it gets a functor as input, we need\nour functors to be subclasses of a functor type that we can dispatch on. This\nbase class is called\n[`FunctorTrait`](https://github.com/MIT-SPARK/Omniplanner/blob/3353262d0e8cadab0528a6a1861c3e246076df5e/omniplanner/src/omniplanner/omniplanner.py#L59),\nand it does nothing other than enable dispatching. When writing a function type\nsignature that you want to dispatch on, you use the `Functor` type (instead of\n`FunctorTrait`), as `Functor` aggregates the user-defined functors with\nbuilt-in iterables. When you define a new functor, you probably need to use our\n[`@dispatchable_parametric`\ndecorator](https://github.com/MIT-SPARK/Omniplanner/blob/3353262d0e8cadab0528a6a1861c3e246076df5e/omniplanner/src/omniplanner/functor.py#L51).\nThis is necessary for getting Bear-types type inference for dataclasses with\ngeneric type parameters to work as it should, which may be of independent\ninterest.\n\nAn example functor that we use is the\n[RobotWrapper](https://github.com/MIT-SPARK/Omniplanner/blob/3353262d0e8cadab0528a6a1861c3e246076df5e/omniplanner/src/omniplanner/omniplanner.py#L59),\nwhich wraps a value with a robot name. The builtin `list` type also gets picked\nup and [treated as a\nfunctor](https://github.com/MIT-SPARK/Omniplanner/blob/3353262d0e8cadab0528a6a1861c3e246076df5e/omniplanner/src/omniplanner/functor.py#L65).\n\n### Multi-robot Support\n\nOmniplanner supports multi-robot planning. We aim to support assignment of\ngoals to robots either as part of the initial goal that is sent to Omniplanner,\nduring the problem grounding process, or during the planning process. For\nsimple problems or single-robot shakeouts, it is very useful to directly\ncommand a given robot with a goal. Other times, multiple robots are involved\nbut the multi-robot problem goal is trivially separable (e.g., by an LLM), and\nthe goal can be divided between the robots during the initial grounding phase.\nIf the problem is truly a difficult multi-robot coordination problem, then\nthe input to `make_plan` may be a grounded multi-robot problem and only at\nthe output of `make_plan` are we able to separate responsibility between\nrobots.\n\nThere are three firm requirements to get multi-robot planning working:\n\n* Robots need to be listed in the Omniplanner config file (see below)\n* The robot\u003c-\u003e map transform between the frames given in the config needs to exist.\n* The output of `make_plan` needs to be a [RobotWrapper](https://github.com/MIT-SPARK/Omniplanner/blob/3353262d0e8cadab0528a6a1861c3e246076df5e/omniplanner/src/omniplanner/omniplanner.py#L59),\n  and the associated robot name is used to send the plan to the right place\n\n\n### Example Configuration\n\nThe following is an example configuration file for Omniplanner to load.\nA new element in the `robots` list needs to be added for each additional\nrobot. Each item in the `planners` list defines an Omniplanner ROS plugin.\n\n```yaml\nrobots:\n  - robot_name: euclid\n    robot_type: spot\n    fixed_frame: map\n    body_frame: euclid/body\n  - robot_name: hamilton\n    robot_type: spot\n    fixed_frame: map\n    body_frame: hamilton/body\nplanners:\n  language_planner:\n    plugin:\n      type: LanguagePlanner\n      domain_type: Pddl\n      pddl_domain_name: GotoObjectDomain\n      llm_config: ${ADT4_DLS_PKG}/config/${config}/llm_config.yaml\n  tsp_planner:\n    plugin:\n      type: Tsp\n      solver: 2opt\n  region_rearrange_objects_pddl:\n    plugin:\n      type: Pddl\n      domain_name: RegionObjectRearrangementDomain\n```\n\n## Notes\n\n### Development Note\n\nCurrently, there is a final step of importing your custom config into the\nOmniplanner Node\n[here](https://github.com/MIT-SPARK/Omniplanner/blob/de84ccf5d5f71b6f41b04d9bceb24a11eaeb1fe5/omniplanner_ros/src/omniplanner_ros/omniplanner_node.py#L28),\nbut the intention is to do automatic plugin discovery. Automatic plugin\ndiscovery will enable all downstream planning plugins to be implemented without\ntouching the omniplanner node.\n\nThere's an edge case in `compile_plan` that may require special care: If the\noutput from your planner (your \"plan\" type) is a subclass of list, then you\nneed to implement a `compile_plan` override for `compile_plan(adaptor,\nSymbolicContext[YourPlanType])` *even if you do not need the symbolic context*.\nThis issue is caused by limitations of generic type inference of parameterized\ntypes. You can choose to either not inherit from list, or ensure that you have\nthis compile plan override implemented.\n\n### Is Omniplanner right for me?\n\nOmniplanner is inherently experimental in nature, but it seems to work pretty\nwell for now. We intend to keep the interface reasonably stable, but currently\nthere are no API stability guarantees. If you use Omniplanner, please let us\nknow, and we will try slightly harder to not break comptability.\n\nThere are two directional choices to note -- we currently only care about Hydra\n3D scene graphs as the world model for kicking off the grounding/planning\nprocess. The *planning* part of omniplanner is pretty generic and is not\ntightly integrated with 3D scene graphs, but the ROS subscription to the scene\ngraph is baked in and some work would be required to genericize the receiving\nof alternative world models.\n\nThe second directional choice is that we have leaned pretty heavily into\nmultiple dispatch as the mechanism for combinding planning and grounding\nmethods. This leads to pretty cool mixing and matching of planning/grounding\nmethods, but currently it is difficult to statically understand what the\nsequence of grounding/planning calls will be for a given problem. It is\nprobably possible to statically analyze the possible flow of calls to different\nplanning/grounding functions, but it would require some rather tight\nintegration with Mypy and probably the Python AST.\n\nFinally, Omniplanner has been built with reasonably low-rate planning in mind\n(on the order of 10 seconds per plan). There is no specific obstacle to\nrunning higher-rate planners, but we currently don't have a very well developed\nnotion of feedback. Everything happens as a \"feed-forward\" planning sequence\nconditioned on the goal and most recent 3D scene graph. As a result, this system\nwill work work best out-of-the-box with planners that run at these somewhat\nlow rates.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-spark%2Fomniplanner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmit-spark%2Fomniplanner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-spark%2Fomniplanner/lists"}