{"id":18792447,"url":"https://github.com/prbonn/semantic-kitti-api","last_synced_at":"2025-05-15T01:04:24.481Z","repository":{"id":46066331,"uuid":"198577858","full_name":"PRBonn/semantic-kitti-api","owner":"PRBonn","description":"SemanticKITTI API for visualizing dataset, processing data, and evaluating results.","archived":false,"fork":false,"pushed_at":"2025-04-03T07:55:17.000Z","size":82,"stargazers_count":826,"open_issues_count":5,"forks_count":190,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-06T18:13:42.087Z","etag":null,"topics":["dataset","deep-learning","evaluation","labels","large-scale-dataset","machine-learning","semantic-scene-completion","semantic-segmentation"],"latest_commit_sha":null,"homepage":"http://semantic-kitti.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PRBonn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-24T07:02:52.000Z","updated_at":"2025-04-06T01:33:17.000Z","dependencies_parsed_at":"2024-05-14T14:43:40.962Z","dependency_job_id":"068437f2-7b2e-44af-b83c-6326c58d8a72","html_url":"https://github.com/PRBonn/semantic-kitti-api","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2Fsemantic-kitti-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2Fsemantic-kitti-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2Fsemantic-kitti-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2Fsemantic-kitti-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PRBonn","download_url":"https://codeload.github.com/PRBonn/semantic-kitti-api/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248785648,"owners_count":21161331,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataset","deep-learning","evaluation","labels","large-scale-dataset","machine-learning","semantic-scene-completion","semantic-segmentation"],"created_at":"2024-11-07T21:19:54.152Z","updated_at":"2025-04-13T21:30:15.617Z","avatar_url":"https://github.com/PRBonn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# API for SemanticKITTI\n\nThis repository contains helper scripts to open, visualize, process, and \nevaluate results for point clouds and labels from the SemanticKITTI dataset.\n\n- Link to original [KITTI Odometry Benchmark](http://www.cvlibs.net/datasets/kitti/eval_odometry.php) Dataset\n- Link to [SemanticKITTI dataset](http://semantic-kitti.org/).\n- Link to SemanticKITTI benchmark [competition](http://semantic-kitti.org/tasks.html).\n\n---\n##### Example of 3D pointcloud from sequence 13:\n\u003cimg src=\"https://image.ibb.co/kyhCrV/scan1.png\" width=\"1000\"\u003e\n\n---\n\n##### Example of 2D spherical projection from sequence 13:\n\u003cimg src=\"https://image.ibb.co/hZtVdA/scan2.png\" width=\"1000\"\u003e\n\n---\n\n##### Example of voxelized point clouds for semantic scene completion:\n\u003cimg src=\"https://user-images.githubusercontent.com/11506664/70214770-4d43ff80-173c-11ea-940d-3950d8f24eaf.png\" width=\"1000\"\u003e\n\n---\n\n## Data organization\n\nThe data is organized in the following format:\n\n```\n/kitti/dataset/\n          └── sequences/\n                  ├── 00/\n                  │   ├── poses.txt\n                  │   ├── image_2/\n                  │   ├── image_3/\n                  │   ├── labels/\n                  │   │     ├ 000000.label\n                  │   │     └ 000001.label\n                  |   ├── voxels/\n                  |   |     ├ 000000.bin\n                  |   |     ├ 000000.label\n                  |   |     ├ 000000.occluded\n                  |   |     ├ 000000.invalid\n                  |   |     ├ 000001.bin\n                  |   |     ├ 000001.label\n                  |   |     ├ 000001.occluded\n                  |   |     ├ 000001.invalid\n                  │   └── velodyne/\n                  │         ├ 000000.bin\n                  │         └ 000001.bin\n                  ├── 01/\n                  ├── 02/\n                  .\n                  .\n                  .\n                  └── 21/\n```\n\n- From [KITTI Odometry](http://www.cvlibs.net/datasets/kitti/eval_odometry.php): \n  - `image_2` and `image_3` correspond to the rgb images for each sequence.\n  - `velodyne` contains the pointclouds for each scan in each sequence. Each \n`.bin` scan is a list of float32 points in [x,y,z,remission] format. See\n[laserscan.py](auxiliary/laserscan.py) to see how the points are read.\n- From SemanticKITTI:\n  - `labels` contains the labels for each scan in each sequence. Each `.label` \nfile contains a uint32 label for each point in the corresponding `.bin` scan.\nSee [laserscan.py](auxiliary/laserscan.py) to see how the labels are read.\n  - `poses.txt` contain the manually looped-closed poses for each capture (in\nthe camera frame) that were used in the annotation tools to aggregate all\nthe point clouds.\n  - `voxels` contains all information needed for the task of semantic scene completion. Each `.bin` file contains for each voxel if that voxel is occupied by laser measurements in a packed binary format. This is the input to the semantic scene completion task and it corresponds to the voxelization of a single LiDAR scan. Each`.label` file contains for each voxel of the completed scene a label in binary format. The label is a 16-bit unsigned integer (aka uint16_t) for each voxel. `.invalid` and `.occluded` contain information about the occlusion of voxel. Invalid voxels are voxels that are occluded from each view position and occluded voxels are occluded in the first view point. See also [SSCDataset.py](auxiliary/SSCDataset.py) for more information on loading the data.\n\nThe main configuration file for the data is in `config/semantic-kitti.yaml`. In this file you will find:\n\n- `labels`: dictionary which maps numeric labels in `.label` file to a string class. Example: `10: \"car\"`\n- `color_map`: dictionary which maps numeric labels in `.label` file to a bgr color for visualization. Example `10: [245, 150, 100] # car, blue-ish`\n- `content`: dictionary with content of each class in labels, as a ratio to \nthe number of total points in the dataset. This can be obtained by running the\n[./content.py](./content.py) script, and is used to calculate the weights for the cross\nentropy in all baseline methods (in order handle class imbalance).\n- `learning_map`: dictionary which maps each class label to its cross entropy\nequivalent, for learning. This is done to mask undesired classes, map different\nclasses together, and because the cross entropy expects a value in\n[0, numclasses - 1]. We also provide [./remap_semantic_labels.py](./remap_semantic_labels.py),\na script that uses this dictionary to put the label files in the cross entropy format,\nso that you can use the labels directly in your training pipeline.\nExamples:\n  ```yaml\n    0 : 0     # \"unlabeled\"\n    1 : 0     # \"outlier\" to \"unlabeled\" -\u003e gets ignored in training, with unlabeled\n    10: 1     # \"car\"\n    252: 1    # \"moving-car\" to \"car\" -\u003e gets merged with static car class\n  ```\n- `learning_map_inv`: dictionary with inverse of the previous mapping, allows to\nmap back the classes only to the interest ones (for saving point cloud predictions\nin original label format). We also provide [./remap_semantic_labels.py](./remap_semantic_labels.py),\na script that uses this dictionary to put the label files in the original format,\nwhen instantiated with the `--inverse` flag.\n- `learning_ignore`: dictionary that contains for each cross entropy class if it\nwill be ignored during training and evaluation or not. For example, class `unlabeled` gets\nignored in both training and evaluation.\n- `split`: contains 3 lists, with the sequence numbers for training, validation, and evaluation.\n\n\n## Dependencies for API:\n\nSystem dependencies\n\n```sh\n$ sudo apt install python3-dev python3-pip python3-pyqt5.qtopengl # for visualization\n```\n\nPython dependencies\n\n```sh\n$ sudo pip3 install -r requirements.txt\n```\n\n## Scripts:\n\n**ALL OF THE SCRIPTS CAN BE INVOKED WITH THE --help (-h) FLAG, FOR EXTRA INFORMATION AND OPTIONS.**\n\n#### Visualization \n\n\n##### Point Clouds\n\nTo visualize the data, use the `visualize.py` script. It will open an interactive\nopengl visualization of the pointclouds along with a spherical projection of\neach scan into a 64 x 1024 image.\n\n```sh\n$ ./visualize.py --sequence 00 --dataset /path/to/kitti/dataset/\n```\n\nwhere:\n- `sequence` is the sequence to be accessed.\n- `dataset` is the path to the kitti dataset where the `sequences` directory is.\n\nNavigation:\n- `n` is next scan,\n- `b` is previous scan,\n- `esc` or `q` exits.\n\nIn order to visualize your predictions instead, the `--predictions` option replaces\nvisualization of the labels with the visualization of your predictions:\n\n```sh\n$ ./visualize.py --sequence 00 --dataset /path/to/kitti/dataset/ --predictions /path/to/your/predictions\n```\n\nTo directly compare two sets of data, use the `compare.py` script. It will open an interactive\nopengl visualization of the pointcloud labels.\n\n```sh\n$ ./compare.py --sequence 00 --dataset_a /path/to/dataset_a/ --dataset_b /path/to/kitti/dataset_b/\n```\n\nwhere:\n- `sequence` is the sequence to be accessed.\n- `dataset_a` is the path to a dataset in KITTI format where the `sequences` directory is.\n- `dataset_b` is the path to another dataset in KITTI format where the `sequences` directory is.\n\nNavigation:\n- `n` is next scan,\n- `b` is previous scan,\n- `esc` or `q` exits.\n\n#### Voxel Grids for Semantic Scene Completion\n\nTo visualize the data, use the `visualize_voxels.py` script. It will open an interactive\nopengl visualization of the voxel grids and options to visualize the provided voxelizations \nof the LiDAR data.\n\n```sh\n$ ./visualize_voxels.py --sequence 00 --dataset /path/to/kitti/dataset/\n```\n\nwhere:\n- `sequence` is the sequence to be accessed.\n- `dataset` is the path to the kitti dataset where the `sequences` directory is.\n\nNavigation:\n- `n` is next scan,\n- `b` is previous scan,\n- `esc` or `q` exits.\n\nNote: Holding the forward/backward buttons triggers the playback mode.\n\n\n#### LiDAR-based Moving Object Segmentation ([LiDAR-MOS](https://github.com/PRBonn/LiDAR-MOS))\n\nTo visualize the data, use the `visualize_mos.py` script. It will open an interactive\nopengl visualization of the voxel grids and options to visualize the provided voxelizations \nof the LiDAR data.\n\n```sh\n$ ./visualize_mos.py --sequence 00 --dataset /path/to/kitti/dataset/\n```\n\nwhere:\n- `sequence` is the sequence to be accessed.\n- `dataset` is the path to the kitti dataset where the `sequences` directory is.\n\nNavigation:\n- `n` is next scan,\n- `b` is previous scan,\n- `esc` or `q` exits.\n\nNote: Holding the forward/backward buttons triggers the playback mode.\n\n\n#### Evaluation\n\nTo evaluate the predictions of a method, use the [evaluate_semantics.py](./evaluate_semantics.py) to evaluate \nsemantic segmentation, [evaluate_completion.py](./evaluate_completion.py) to evaluate the semantic scene completion and [evaluate_panoptic.py](./evaluate_panoptic.py) to evaluate panoptic segmentation.\n**Important:** The labels and the predictions need to be in the original\nlabel format, which means that if a method learns the cross-entropy mapped\nclasses, they need to be passed through the `learning_map_inv` dictionary\nto be sent to the original dataset format. This is to prevent changes in the\ndataset interest classes from affecting intermediate outputs of approaches, \nsince the original labels will stay the same. \nFor semantic segmentation, we provide the `remap_semantic_labels.py` script to make this \nshift before the training, and once again before the evaluation, selecting which are the interest \nclasses in the configuration file. \nThe data needs to be either:\n\n- In a separate directory with this format:\n\n  ```\n  /method_predictions/\n            └── sequences\n                ├── 00\n                │   └── predictions\n                │         ├ 000000.label\n                │         └ 000001.label\n                ├── 01\n                ├── 02\n                .\n                .\n                .\n                └── 21\n  ```\n\n  And run:\n\n  ```sh\n  $ ./evaluate_semantics.py --dataset /path/to/kitti/dataset/ --predictions /path/to/method_predictions --split train/valid/test # depending of desired split to evaluate\n  ```\n\n  or \n\n    ```sh\n  $ ./evaluate_completion.py --dataset /path/to/kitti/dataset/ --predictions /path/to/method_predictions --split train/valid/test # depending of desired split to evaluate\n  ```\n\n  or \n\n    ```sh\n  $ ./evaluate_panoptic.py --dataset /path/to/kitti/dataset/ --predictions /path/to/method_predictions --split train/valid/test # depending of desired split to evaluate\n  ```\n\n  or for moving object segmentation\n\n    ```sh\n  $ ./evaluate_mos.py --dataset /path/to/kitti/dataset/ --predictions /path/to/method_predictions --split train/valid/test # depending of desired split to evaluate\n  ```  \n\n- In the same directory as the dataset\n\n  ```\n  /kitti/dataset/\n            ├── poses\n            └── sequences\n                ├── 00\n                │   ├── image_2\n                │   ├── image_3\n                │   ├── labels\n                │   │     ├ 000000.label\n                │   │     └ 000001.label\n                │   ├── predictions\n                │   │     ├ 000000.label\n                │   │     └ 000001.label\n                │   └── velodyne\n                │         ├ 000000.bin\n                │         └ 000001.bin\n                ├── 01\n                ├── 02\n                .\n                .\n                .\n                └── 21\n  ```\n\n  And run (which sets the predictions directory as the same directory as the dataset):\n\n  ```sh\n  $ ./evaluate_semantics.py --dataset /path/to/kitti/dataset/ --split train/valid/test # depending of desired split to evaluate\n  ```\n\nIf instead, the IoU vs distance is wanted, the evaluation is performed in the\nsame way, but with the [evaluate_semantics_by_distance.py](./evaluate_semantics_by_distance.py) script. This will\nanalyze the IoU for a set of 5 distance ranges: `{(0m:10m), [10m:20m), [20m:30m), [30m:40m), (40m:50m)}`. \n\n#### Validation\n\nTo ensure that your zip file is valid, we provide a small validation script [validate_submission.py](./validate_submission.py) that checks for the correct folder structure and consistent number of labels for each scan.\n\nThe submission folder expects to get an zip file containing the following folder structure (as the separate case above)\n\n  ```\n  ├ description.txt (optional)\n  sequences\n    ├── 11\n    │   └── predictions\n    │         ├ 000000.label\n    │         ├ 000001.label\n    │         ├ ...\n    ├── 12\n    │   └── predictions\n    │         ├ 000000.label\n    │         ├ 000001.label\n    │         ├ ...\n    ├── 13\n    .\n    .\n    .\n    └── 21\n  ```\n\nIn summary, you only have to provide the label files containing your predictions for every point of the scan and this is also checked by our validation script.\n\nRun:\n  ```sh\n  $ ./validate_submission.py --task {segmentation|completion|panoptic} /path/to/submission.zip /path/to/kitti/dataset\n  ```\nto check your `submission.zip`.\n\n***Note:*** We don't check if the labels are valid, since invalid labels are simply ignored by the evaluation script.\n\n#### (New!) Adding Approach Information\n\nIf you want to have more information on the leaderboard in the new updated Codalab competitions under the \"Detailed Results\", you have to provide an additional `description.txt` file to the submission archive containing information (here just an example):\n\n```\nname: Auto-MOS\npdf url: https://arxiv.org/pdf/2201.04501.pdf\ncode url: https://github.com/PRBonn/auto-mos\n```\n\nwhere `name` corresponds to the name of the method, `pdf url` is a link to the paper pdf url (or empty), and `code url` is a url that directs to the code (or empty). If the information is not available, we will use `Anonymous` for the name, and `n/a` for the urls.\n\n\n\n#### Statistics\n\n- [content.py](content.py) allows to evaluate the class content of the training\nset, in order to weigh the loss for training, handling imbalanced data.\n- [count.py](count.py) returns the scan count for each sequence in the data.\n\n#### Generation\n\n- [generate_sequential.py](generate_sequential.py) generates a sequence of scans using the manually looped closed poses used in our labeling tool, and stores them as individual point clouds. If, for example, we want to generate a dataset containing, for each point cloud, the aggregation of itself with the previous 4 scans, then:\n\n  ```sh\n  $ ./generate_sequential.py --dataset /path/to/kitti/dataset/ --sequence_length 5 --output /path/to/put/new/dataset \n  ```\n  \n- [remap_semantic_labels.py](remap_semantic_labels.py) allows to remap the labels\nto and from the cross-entropy format, so that the labels can be used for training,\nand the predictions can be used for evaluation. This file uses the `learning_map` and\n`learning_map_inv` dictionaries from the config file to map the labels and predictions.\n\n## Docker for API\n\nIf not installing the requirements is preferred, then a [docker](https://docs.docker.com/install/linux/docker-ce/ubuntu/) container is\nprovided to run the scripts.\n\nTo build and run the container in an interactive session, which allows to run\nX11 apps (and GL), and copies this repo to the working directory, use\n\n```\n$ ./docker.sh /path/to/dataset\n```\n\nWhere `/path/to/dataset` is the location of your semantic kitti dataset, and\nwill be available inside the image in `~/data` or `/home/developer/data` \ninside the container for further usage with the api. This is done by creating\na shared volume, so it can be any directory containing data that is to be used\nby the API scripts.\n\n## Citation:\n\nIf you use this dataset and/or this API in your work, please cite its [paper](https://arxiv.org/abs/1904.01416)\n\n```\n@inproceedings{behley2019iccv,\n    author = {J. Behley and M. Garbade and A. Milioto and J. Quenzel and S. Behnke and C. Stachniss and J. Gall},\n     title = {{SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences}},\n booktitle = {Proc. of the IEEE/CVF International Conf.~on Computer Vision (ICCV)},\n      year = {2019}\n}\n```\n\nAnd the paper for the [original KITTI dataset](http://www.cvlibs.net/datasets/kitti/eval_odometry.php):\n\n```\n@inproceedings{geiger2012cvpr,\n    author = {A. Geiger and P. Lenz and R. Urtasun},\n     title = {{Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite}},\n booktitle = {Proc.~of the IEEE Conf.~on Computer Vision and Pattern Recognition (CVPR)},\n     pages = {3354--3361},\n      year = {2012}}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Fsemantic-kitti-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprbonn%2Fsemantic-kitti-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Fsemantic-kitti-api/lists"}