{"id":23979345,"url":"https://github.com/filapro/unidet3d","last_synced_at":"2025-04-07T19:13:08.066Z","repository":{"id":255984110,"uuid":"854039918","full_name":"filaPro/unidet3d","owner":"filaPro","description":"[AAAI2025] UniDet3D: Multi-dataset Indoor 3D Object Detection","archived":false,"fork":false,"pushed_at":"2024-12-10T11:24:45.000Z","size":8162,"stargazers_count":104,"open_issues_count":3,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-31T18:18:28.652Z","etag":null,"topics":["3d-object-detection","mmdetection3d","pytorch","s3dis","scannet"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/filaPro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-08T08:40:30.000Z","updated_at":"2025-03-30T06:11:56.000Z","dependencies_parsed_at":"2024-09-08T09:57:58.254Z","dependency_job_id":"a02a95c1-6262-4c0c-b596-4df1c7d485d1","html_url":"https://github.com/filaPro/unidet3d","commit_stats":null,"previous_names":["filapro/unidet3d"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filaPro%2Funidet3d","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filaPro%2Funidet3d/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filaPro%2Funidet3d/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filaPro%2Funidet3d/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/filaPro","download_url":"https://codeload.github.com/filaPro/unidet3d/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247713258,"owners_count":20983683,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-object-detection","mmdetection3d","pytorch","s3dis","scannet"],"created_at":"2025-01-07T09:48:32.135Z","updated_at":"2025-04-07T19:13:07.403Z","avatar_url":"https://github.com/filaPro.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## UniDet3D: Multi-dataset Indoor 3D Object Detection\n\n**News**:\n * :fire: December, 2024. UniDet3D is now accepted at AAAI 2025.\n * :fire: September, 2024. UniDet3D is state-of-the-art in 6 indoor benchmarks: \u003cbr\u003e\n ScanNet [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/unidet3d-multi-dataset-indoor-3d-object/3d-object-detection-on-scannetv2)](https://paperswithcode.com/sota/3d-object-detection-on-scannetv2?p=unidet3d-multi-dataset-indoor-3d-object) \u003cbr\u003e\n ARKitScenes [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/unidet3d-multi-dataset-indoor-3d-object/3d-object-detection-on-arkitscenes)](https://paperswithcode.com/sota/3d-object-detection-on-arkitscenes?p=unidet3d-multi-dataset-indoor-3d-object) \u003cbr\u003e\n S3DIS [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/unidet3d-multi-dataset-indoor-3d-object/3d-object-detection-on-s3dis)](https://paperswithcode.com/sota/3d-object-detection-on-s3dis?p=unidet3d-multi-dataset-indoor-3d-object) \u003cbr\u003e\n MultiScan [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/unidet3d-multi-dataset-indoor-3d-object/3d-object-detection-on-multiscan)](https://paperswithcode.com/sota/3d-object-detection-on-multiscan?p=unidet3d-multi-dataset-indoor-3d-object) \u003cbr\u003e\n 3RScan [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/unidet3d-multi-dataset-indoor-3d-object/3d-object-detection-on-3rscan)](https://paperswithcode.com/sota/3d-object-detection-on-3rscan?p=unidet3d-multi-dataset-indoor-3d-object) \u003cbr\u003e\n ScanNet++ [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/unidet3d-multi-dataset-indoor-3d-object/3d-object-detection-on-scannet-1)](https://paperswithcode.com/sota/3d-object-detection-on-scannet-1?p=unidet3d-multi-dataset-indoor-3d-object).  \n\nThis repository contains an implementation of UniDet3D, a multi-dataset indoor 3D object detection method introduced in our paper:\n\n\u003e **UniDet3D: Multi-dataset Indoor 3D Object Detection**\u003cbr\u003e\n\u003e [Maksim Kolodiazhnyi](https://github.com/col14m),\n\u003e [Anna Vorontsova](https://github.com/highrut),\n\u003e [Matvey Skripkin](https://scholar.google.com/citations?user=hAlwb4wAAAAJ),\n\u003e [Danila Rukhovich](https://github.com/filaPro),\n\u003e [Anton Konushin](https://scholar.google.com/citations?user=ZT_k-wMAAAAJ)\n\u003e \u003cbr\u003e\n\u003e Artificial Intelligence Research Institute\u003cbr\u003e\n\u003e https://arxiv.org/abs/2409.04234\n\n### Installation\n\nFor convenience, we provide a [Dockerfile](Dockerfile).\nThis implementation is based on [mmdetection3d](https://github.com/open-mmlab/mmdetection3d) framework `v1.1.0`. If not using Docker, please follow [getting_started.md](https://github.com/open-mmlab/mmdetection3d/blob/22aaa47fdb53ce1870ff92cb7e3f96ae38d17f61/docs/en/get_started.md) for the installation instructions.\n\n\n### Getting Started\n\nPlease see [test_train.md](https://github.com/open-mmlab/mmdetection3d/blob/22aaa47fdb53ce1870ff92cb7e3f96ae38d17f61/docs/en/user_guides/train_test.md) for some basic usage examples.\n\n#### Data Preprocessing\n\nUniDet3D is trained and tested using 6 datasets: [ScanNet](data/scannet), [ARKitScenes](data/arkitscenes), [S3DIS](data/s3dis), [MultiScan](data/multiscan), [3RScan](data/3rscan), and [ScanNet++](data/scannetpp).\nPreprocessed data can be found at our [Hugging Face](https://huggingface.co/datasets/maksimko123/UniDet3D). Download each archive, unpack, and move into the corresponding directory in [data](data). Please comply with the license agreement before downloading the data.\n\nAlternatively, you can preprocess the data by youself. \nTraining data for 3D object detection methods that do not requires superpoints, e.g. [TR3D](https://github.com/SamsungLabs/tr3d) or [FCAF3D](https://github.com/SamsungLabs/fcaf3d), can be prepared according to the [instructions](data).\n\nSuperpoints for ScanNet and MultiScan are provided as a part of the original annotation. For the rest datasets, you can either download pre-computed superpoints at our [Hugging Face](https://huggingface.co/datasets/maksimko123/UniDet3D), or compute them using [superpoint_transformer](https://github.com/drprojects/superpoint_transformer).\n\n#### Training\n\nBefore training, please download the backbone [checkpoint](https://github.com/filapro/oneformer3d/releases/download/v1.0/oneformer3d_1xb4_scannet.pth) and save it under `work_dirs/tmp`.\n\nTo train UniDet3D on 6 datasets jointly, simply run the [training](tools/train.py) script:\n\n```bash\npython tools/train.py configs/unidet3d_1xb8_scannet_s3dis_multiscan_3rscan_scannetpp_arkitscenes.py\n```\n\nUniDet3D can also be trained on individual datasets, e.g., we provide a [config](configs/unidet3d_1xb8_scannet.py) for training using ScanNet solely.\n\n\n#### Testing\n\nTo test a trained model, you can run the [testing](tools/test.py) script:\n\n```bash\npython tools/test.py configs/unidet3d_1xb8_scannet_s3dis_multiscan_3rscan_scannetpp_arkitscenes.py \\\n    work_dirs/unidet3d_1xb8_scannet_s3dis_multiscan_3rscan_scannetpp_arkitscenes/epoch_1024.pth\n```\n\nUniDet3D can also be tested on individual datasets. To this end, simply remove the unwanted datasets from `val_dataloader.dataset.datasets` in the config file.\n\n#### Visualization\n\nTo visualize ground truth and predicted boxes, run the [testing](tools/test.py) script with additional arguments:\n\n```bash\npython tools/test.py configs/unidet3d_1xb8_scannet_s3dis_multiscan_3rscan_scannetpp_arkitscenes.py \\\n    work_dirs/unidet3d_1xb8_scannet_s3dis_multiscan_3rscan_scannetpp_arkitscenes/latest.pth --show \\\n    --show-dir work_dirs/unidet3d_1xb8_scannet_s3dis_multiscan_3rscan_scannetpp_arkitscenes\n```\nYou can also set `score_thr` in configs to `0.3` for better visualizations.\n\n### Trained Model\n\nPlease refer to the UniDet3D [checkpoint](https://github.com/filapro/unidet3d/releases/download/v1.0/unidet3d.pth) and [log file](https://github.com/filapro/unidet3d/releases/download/v1.0/log.txt). The corresponding metrics are given below (they might slightly deviate from the values reported in the paper due to the randomized training/testing procedure).\n\n| Dataset     | mAP\u003csub\u003e25\u003c/sub\u003e  | mAP\u003csub\u003e50\u003c/sub\u003e  |\n|:-----------:|:-----------------:|:-----------------:|\n| ScanNet     | 77.0              | 65.9              |\n| ARKitScenes | 60.1              | 47.2              |\n| S3DIS       | 76.7              | 65.3              |\n| MultiScan   | 62.6              | 52.3              |\n| 3RScan      | 63.6              | 44.9              |\n| ScanNet++   | 24.0              | 16.8              |\n\n### Predictions Example\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/user-attachments/assets/bb535823-cc9b-4482-a1b6-d10cf74c9389\" alt=\"UniDet3D predictions\"/\u003e\n\u003c/p\u003e\n\n### Citation\n\nIf you find this work useful for your research, please cite our paper:\n\n```\n@article{kolodiazhnyi2024unidet3d,\n  title={UniDet3D: Multi-dataset Indoor 3D Object Detection},\n  author={Kolodiazhnyi, Maxim and Vorontsova, Anna and Skripkin, Matvey and Rukhovich, Danila and Konushin, Anton},\n  journal={arXiv preprint arXiv:2409.04234},\n  year={2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilapro%2Funidet3d","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffilapro%2Funidet3d","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilapro%2Funidet3d/lists"}