{"id":18621413,"url":"https://github.com/ethz-asl/panoptic_mapping","last_synced_at":"2025-05-16T11:05:25.641Z","repository":{"id":40607556,"uuid":"265213863","full_name":"ethz-asl/panoptic_mapping","owner":"ethz-asl","description":"A flexible submap-based framework towards spatio-temporally consistent volumetric mapping and scene understanding.","archived":false,"fork":false,"pushed_at":"2024-11-29T14:46:11.000Z","size":1407,"stargazers_count":303,"open_issues_count":9,"forks_count":35,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-04-12T08:17:04.990Z","etag":null,"topics":["consistent","mapping","multi-tsdfs","panoptic","panoptic-mapping","semantic","temporal","volumetric"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ethz-asl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-19T10:15:16.000Z","updated_at":"2025-03-18T15:56:50.000Z","dependencies_parsed_at":"2023-01-21T06:20:41.467Z","dependency_job_id":"593a952a-3055-4c55-a4bf-7f920c87a411","html_url":"https://github.com/ethz-asl/panoptic_mapping","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethz-asl%2Fpanoptic_mapping","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethz-asl%2Fpanoptic_mapping/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethz-asl%2Fpanoptic_mapping/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethz-asl%2Fpanoptic_mapping/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ethz-asl","download_url":"https://codeload.github.com/ethz-asl/panoptic_mapping/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254518384,"owners_count":22084374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consistent","mapping","multi-tsdfs","panoptic","panoptic-mapping","semantic","temporal","volumetric"],"created_at":"2024-11-07T04:11:33.727Z","updated_at":"2025-05-16T11:05:20.628Z","avatar_url":"https://github.com/ethz-asl.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Ubuntu 18 + ROS Melodic](https://github.com/ethz-asl/panoptic_mapping/actions/workflows/build_test_18.yml/badge.svg) ![Ubuntu 20 + ROS Noetic](https://github.com/ethz-asl/panoptic_mapping/actions/workflows/build_test_20.yml/badge.svg) ![Docker](https://github.com/ethz-asl/panoptic_mapping/actions/workflows/docker-publish.yml/badge.svg)\n\n# Panoptic Mapping\nThis package contains **panoptic_mapping**, a general framework for semantic volumetric mapping. We provide, among other, a submap-based approach that leverages panoptic scene understanding towards adaptive spatio-temporally consistent volumetric mapping, as well as regular, monolithic semantic mapping.\n\n![combined](https://user-images.githubusercontent.com/36043993/135645102-e5798e36-e2b0-4611-9260-ec9d54d38e47.png)\n\nMulti-resolution 3D Reconstruction, active and inactive panoptic submaps for temporal consistency, online change detection, and more.\n\n# Table of Contents\n**Credits**\n* [Paper](#Paper)\n* [Video](#Video)\n\n**Setup**\n* [Installation](#Installation)\n* [Datasets](#Datasets)\n\n**Examples**\n- [Running the Panoptic Mapper](#running-the-panoptic-mapper)\n- [Monolithic Semantic Mapping](#monolithic-semantic-mapping)\n- [Running the RIO Dataset](#running-the-rio-dataset)\n\n**Other**\n* [Contributing](#Contributing)\n\n# Paper\nIf you find this package useful for your research, please consider citing our paper:\n\n* Lukas Schmid, Jeffrey Delmerico, Johannes Schönberger, Juan Nieto, Marc Pollefeys, Roland Siegwart, and Cesar Cadena. \"**Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency**\" in *IEEE International Conference on Robotics and Automation (ICRA)*, pp. 8018-8024, 2022.\n  \\[ [IEEE](https://ieeexplore.ieee.org/document/9811877) | [ArXiv](https://arxiv.org/abs/2109.10165) | [Video](https://www.youtube.com/watch?v=A7o2Vy7_TV4) \\]\n  ```bibtex\n  @inproceedings{schmid2022panoptic,\n    title={Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency},\n    author={Schmid, Lukas and Delmerico, Jeffrey and Sch{\\\"o}nberger, Johannes and Nieto, Juan and Pollefeys, Marc and Siegwart, Roland and Cadena, Cesar},\n    booktitle={2022 IEEE International Conference on Robotics and Automation (ICRA)},\n    year={2022},\n    volume={},\n    number={},\n    pages={8018-8024},\n    doi={10.1109/ICRA46639.2022.9811877}}\n  }\n  ```\n\n# Video\nFor a short overview explaining the approach check out our video on youtube:\n\n[\u003cimg src=\"https://user-images.githubusercontent.com/36043993/155131772-60757e47-c458-4f5b-9c6b-a6e6cbca8976.jpg\" alt=\"youtube video\"\u003e](https://www.youtube.com/watch?v=A7o2Vy7_TV4)\n\n# Installation\nInstructions for different installation options. The repository was developed and tested on Ubuntu 18.04 with ROS melodic and Ubuntu 20.04 with ROS noetic.\n\n**Docker**\nTo run the panoptic mapper without installation, just use the docker image:\n```bash\ndocker pull ghcr.io/ethz-asl/panoptic_mapping:main\ndocker run -it ghcr.io/ethz-asl/panoptic_mapping:main bash\n```\n\n**VSCode**\nFor development with vscode, you can just clone this repository and follow the prompts to open it in a [devcontainer](https://code.visualstudio.com/docs/devcontainers/containers).\n\n**System Installation**\nTo install on linux, follow the instructions below matching your OS and ROS version:\n\n\u003cdetails\u003e\n  \u003csummary\u003eUbuntu 18.04 + ROS Melodic.\u003c/summary\u003e\n\u003cp\u003e\n\n**Prerequisites**\n\n1. If not already done so, install [ROS](http://wiki.ros.org/ROS/Installation) (Desktop-Full is recommended).\n\n2. If not already done so, create a catkin workspace with [catkin tools](https://catkin-tools.readthedocs.io/en/latest/):\n\n  ```shell script\n  sudo apt-get install python-catkin-tools\n  mkdir -p ~/catkin_ws/src\n  cd ~/catkin_ws\n  catkin init\n  catkin config --extend /opt/ros/melodic\n  catkin config --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo\n  catkin config --merge-devel\n  ```\n\n**Installation**\n\n1. Install system dependencies:\n\n  ```shell script\n  sudo apt-get install python-wstool python-catkin-tools autoconf libtool git\n  ```\n\n2. Move to your catkin workspace:\n\n  ```shell script\n  cd ~/catkin_ws/src\n  ```\n\n3. Download repo using [SSH](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh) or HTTPS:\n\n  ```shell script\n  git clone git@github.com:ethz-asl/panoptic_mapping.git  # SSH\n  git clone https://github.com/ethz-asl/panoptic_mapping.git  # HTTPS\n  ```\n\n4. Download and install package dependencies using ros install:\n\n  * If you created a new workspace.\n\n  ```shell script\n  wstool init . ./panoptic_mapping/panoptic_mapping_ssh.rosinstall    # SSH\n  wstool init . ./panoptic_mapping/panoptic_mapping_https.rosinstall  # HTTPS\n  wstool update\n  ```\n\n  * If you use an existing workspace. Notice that some dependencies require specific branches that will be checked out.\n\n  ```shell script\n  wstool merge -t . ./panoptic_mapping/panoptic_mapping.rosinstall\n  wstool update\n  ```\n\n5. Compile and source:\n\n  ```shell script\n  catkin build panoptic_mapping_utils\n  source ../devel/setup.bash\n  ```\n\u003c/p\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eUbuntu 20.04 + ROS Noetic.\u003c/summary\u003e\n\u003cp\u003e\n\n**Prerequisites**\n\n1. If not already done so, install [ROS](http://wiki.ros.org/ROS/Installation) (Desktop-Full is recommended).\n\n2. If not already done so, create a catkin workspace with [catkin tools](https://catkin-tools.readthedocs.io/en/latest/):\n\n  ```shell script\n  sudo apt-get install python3-catkin-tools\n  mkdir -p ~/catkin_ws/src\n  cd ~/catkin_ws\n  catkin init\n  catkin config --extend /opt/ros/noetic\n  catkin config --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo\n  catkin config --merge-devel\n  ```\n\n**Installation**\n\n1. Install system dependencies:\n\n  ```shell script\n  sudo apt-get install python3-pip python3-wstool python3-catkin-tools autoconf libtool git\n  pip3 install osrf-pycommon\n  ```\n\n2. Move to your catkin workspace:\n\n  ```shell script\n  cd ~/catkin_ws/src\n  ```\n\n3. Download repo using [SSH](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh) or HTTPS:\n\n  ```shell script\n  git clone git@github.com:ethz-asl/panoptic_mapping.git  # SSH\n  git clone https://github.com/ethz-asl/panoptic_mapping.git  # HTTPS\n  ```\n\n4. Download and install package dependencies using ros install:\n\n  * If you created a new workspace.\n\n  ```shell script\n  wstool init . ./panoptic_mapping/panoptic_mapping_ssh.rosinstall    # SSH\n  wstool init . ./panoptic_mapping/panoptic_mapping_https.rosinstall  # HTTPS\n  wstool update\n  ```\n\n  * If you use an existing workspace. Notice that some dependencies require specific branches that will be checked out.\n\n  ```shell script\n  wstool merge -t . ./panoptic_mapping/panoptic_mapping.rosinstall\n  wstool update\n  ```\n\n5. Compile and source:\n\n  ```shell script\n  catkin build panoptic_mapping_utils\n  source ../devel/setup.bash\n  ```\n\u003c/p\u003e\n\u003c/details\u003e\n\n\n# Datasets\nThe datasets described in the paper and used for the demo can be downloaded from the [ASL Datasets](https://projects.asl.ethz.ch/datasets/doku.php?id=panoptic_mapping).\n\nA utility script is provided to directly download the flat data:\n```\nroscd panoptic_mapping_utils\nexport FLAT_DATA_DIR=\"/home/$USER/Documents\"  # Or whichever path you prefer.\nchmod +x panoptic_mapping_utils/scripts/download_flat_dataset.sh\n./panoptic_mapping_utils/scripts/download_flat_dataset.sh\n```\n\nTo run the RIO demos, the [original dataset](https://waldjohannau.github.io/RIO/) needs to be downloaded and augmented with [our supplementary data](https://projects.asl.ethz.ch/datasets/doku.php?id=panoptic_mapping). Instructions on which scenes to download and how to combine them are found on our dataset page.\n\n# Examples\n## Running the Panoptic Mapper\nThis example explains how to run the Panoptic Multi-TSDF mapper on the flat dataset.\n\n1. First, download the flat dataset:\n    ```\n    export FLAT_DATA_DIR=\"/home/$USER/Documents\"  # Or whichever path you prefer.\n    chmod +x panoptic_mapping_utils/scripts/download_flat_dataset.sh\n    ./panoptic_mapping_utils/scripts/download_flat_dataset.sh\n    ```\n2. Replace the data `base_path` in `launch/run.launch (L10)` and `file_name` in `config/mapper/flat_groundtruth.yaml (L15)` to the downloaded path.\n3. Run the mapper:\n    ```\n    roslaunch panoptic_mapping_ros run.launch\n    ```\n4. You should now see the map being incrementally built:\n\n    \u003cimg src=\"https://user-images.githubusercontent.com/36043993/135860249-6334cc41-5758-457b-8f65-b017e2905804.png\" width=\"400\"\u003e\n\n5. After the map finished building, you can save the map:\n    ```\n    rosservice call /panoptic_mapper/save_map \"file_path: '/path/to/run1.panmap'\"\n    ```\n6. Terminate the mapper pressing Ctrl+C. You can continue the experiment on `run2` of the flat dataset by changing the `base_path`-ending in `launch/run.launch (L10)` to `run2`, and `load_map` and `load_path` in `launch/run.launch (L26-27)` to `true` and `/path/to/run1.panmap`, respectively. Optionally, you can also change the `color_mode` in `config/mapper/flat_groundtruth.yaml (L118)` to `change` to better highlight the change detection at work.\n     ```\n    roslaunch panoptic_mapping_ros run.launch\n    ```\n7. You should now see the map being updated based on the first run:\n\n    \u003cimg src=\"https://user-images.githubusercontent.com/36043993/135861611-4d576750-3104-4d73-87dc-60b7a4ad1df6.png\" width=\"400\"\u003e\n\n## Monolithic Semantic Mapping\nPanoptic Mapping supports also the monolithic use case. This example explains how to run the Panoptic Single-TSDF mapper on the flat dataset.\n\n1. If not already done so, download the flat dataset:\n\n   ```\n   export FLAT_DATA_DIR=\"/home/$USER/Documents\"  # Or whichever path you prefer.\n   chmod +x panoptic_mapping_utils/scripts/download_flat_dataset.sh\n   ./panoptic_mapping_utils/scripts/download_flat_dataset.sh\n   ```\n\n2. Replace the data `base_path` in `launch/run.launch (L10)` and `file_name` in `config/mapper/single_tsdf.yaml (L15)` to the downloaded path.\n\n3. To use the single-TSDF mapper and real segmentation predictions, set `use_detectron` in `launch/run.launch (L6)` to true and `config` in `launch/run.launch (L22)` to 'single_tsdf'.\n\n4. Run the mapper:\n\n   ```\n   roslaunch panoptic_mapping_ros run.launch\n   ```\n\n5. You should now see the map being incrementally built.\n\n6. Varying visualization modes are supported, the `classes` (default) will color the mesh according to the predicted semantic class. Other modes, such as `classification` will show the confidence of the aggregated predictions.\n\n![single_combined](https://user-images.githubusercontent.com/36043993/143055994-2703f57c-cf4a-4126-a43c-cb7f47b6da02.png)\n\nPredicted classes (left) and corresponding fusion confidence (right, low to high in red to green).\n\n## Running the RIO Dataset\nThis example explains how to run the Panoptic Multi-TSDF mapper on the RIO dataset.\n\n1. First, download the [original dataset](https://waldjohannau.github.io/RIO/) and [our supplementary data](https://projects.asl.ethz.ch/datasets/doku.php?id=panoptic_mapping).\n\n2. Set `use_rio` in `launch/run.launch (L5)` to true, replace the `data_path` in `(L16)` and `file_name` in `config/mapper/rio_groundtruth.yaml (L15)` to the downloaded path.\n\n3. Run the mapper:\n    ```\n    roslaunch panoptic_mapping_ros run.launch\n    ```\n4. You should now see the map being incrementally built:\n\n    \u003cimg src=\"https://user-images.githubusercontent.com/36043993/157041619-3f683ee7-2709-4bc5-84c2-8b1188ee52c5.png\" width=\"800\"\u003e\n\n    (Left in reconstructed color, right colored by submap)\n\n\n# Contributing\n**panoptic_mapping** is an open-source project, any contributions are welcome!\n\nFor issues, bugs, or suggestions, please open a [GitHub Issue](https://github.com/ethz-asl/panoptic_mapping/issues).\n\nTo add to this repository:\n* Please employ the [feature-branch workflow](https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow).\n* Setup our auto-formatter for coherent style (we follow the [google style guide](https://google.github.io/styleguide/cppguide.html)):\n    ```\n    # Download the linter\n    cd \u003clinter_dest\u003e\n    git clone git@github.com:ethz-asl/linter.git -b feature/noetic\n    cd linter\n    echo \". $(realpath setup_linter.sh)\" \u003e\u003e ~/.bashrc\n    bash\n    roscd panoptic_mapping/..\n    init_linter_git_hooks\n    # You're all set to go!\n    ```\n* Please open a [Pull Request](https://github.com/ethz-asl/panoptic_mapping/pulls) for your changes.\n* Thank you for contributing!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethz-asl%2Fpanoptic_mapping","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fethz-asl%2Fpanoptic_mapping","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethz-asl%2Fpanoptic_mapping/lists"}