{"id":13443563,"url":"https://github.com/Na-Z/SDCoT","last_synced_at":"2025-03-20T16:31:59.182Z","repository":{"id":127433609,"uuid":"450100540","full_name":"Na-Z/SDCoT","owner":"Na-Z","description":"[AAAI 2022 Oral] Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection","archived":false,"fork":false,"pushed_at":"2022-11-22T14:35:53.000Z","size":170,"stargazers_count":25,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-28T06:58:32.399Z","etag":null,"topics":["3d","3d-object-detection","class-incremental-learning","point-cloud","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Na-Z.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-01-20T13:05:05.000Z","updated_at":"2023-12-02T06:42:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"f253c39c-8f2f-413a-ab5b-67781c622cf0","html_url":"https://github.com/Na-Z/SDCoT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Na-Z%2FSDCoT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Na-Z%2FSDCoT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Na-Z%2FSDCoT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Na-Z%2FSDCoT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Na-Z","download_url":"https://codeload.github.com/Na-Z/SDCoT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244649945,"owners_count":20487522,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","3d-object-detection","class-incremental-learning","point-cloud","pytorch"],"created_at":"2024-07-31T03:02:03.872Z","updated_at":"2025-03-20T16:31:58.863Z","avatar_url":"https://github.com/Na-Z.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection \n\nCreated by \u003ca href=\"https://github.com/Na-Z\" target=\"_blank\"\u003eNa Zhao\u003c/a\u003e from \n\u003ca href=\"http://www.nus.edu.sg/\" target=\"_blank\"\u003eNational University of Singapore\u003c/a\u003e\n\n![teaser](framework.jpg)\n\n## Introduction\nThis repository is about the PyTorch implementation for our AAAI 2022 Paper \n\"[Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection](https://arxiv.org/pdf/2112.07241.pdf)\" by Na Zhao and Gim Hee Lee. \n\nDeep learning-based approaches have shown remarkable performance in the 3D object detection task. However, they suffer from a catastrophic performance drop on the originally trained classes when incrementally learning new classes without revisiting the old data. This \"catastrophic forgetting\" phenomenon impedes the deployment of 3D object detection approaches in real-world scenarios, where continuous learning systems are needed. In this paper, we study the unexplored yet important class-incremental 3D object detection problem and present the first solution - SDCoT, a novel static-dynamic co-teaching method. Our SDCoT alleviates the catastrophic forgetting of old classes via a static teacher, which provides pseudo annotations for old classes in the new samples and regularizes the current model by extracting previous knowledge with a distillation loss. At the same time, SDCoT consistently learns the underlying knowledge from new data via a dynamic teacher. We conduct extensive experiments on two benchmark datasets and demonstrate the superior performance of our SDCoT over baseline approaches in several incremental learning scenarios.\n\n\n## Setup\n- Install `python` --This repo is tested with `python 3.6.8`.\n- Install `pytorch` with CUDA -- This repo is tested with `torch 1.1`, `CUDA 9.0`. \nIt may wrk with newer versions, but that is not gauranteed.\n- Install Tensorboard -- This repo is tested with `tensorboard 1.14.0`.\n- Compile the CUDA layers for PointNet++, which is used in the backbone network:\n    ```\n    cd pointnet2\n    python setup.py install\n    ```\n- Install dependencies\n    ```\n    pip install -r requirements.txt\n    ```\n    \n    \n\n## Usage\n### Data preparation\nFor SUNRGB-D, follow the [README](https://github.com/Na-Z/SDCoT/blob/main/sunrgbd/README.md) under `sunrgbd` folder.\n\nFor ScanNet, follow the [README](https://github.com/Na-Z/SDCoT/blob/main/scannet/README.md) under `scannet` folder.\n\n### Running\nFor SUNRGB-D, you can modify line 40-46 in ./cfg/sunrgbd_cfg.py to set up the class splitting (splitting of base and novel classes), then use the following command to train and evaluate:\n\n- Training\n    - Base training:\n        ```\n        CUDA_VISIBLE_DEVICES=0 python main/train_bt.py --method basetrain --dataset sunrgbd --num_point 20000 --pc_augm\n        ```\n    - Baseline methods (set $FT_LAYERS='last' for \"Freeze and add\" and set $FT_LAYERS='all' for \"Fune-tuning\"):\n        ```\n        CUDA_VISIBLE_DEVICES=0 python main/train_ft.py --method finetune --dataset sunrgbd --num_point 20000 --model_checkpoint_path $MODEL_CHCECKPOINT_PATH --n_novel_class $N_NOVEL_CLASS --ft_layers $FT_LAYERS --lr $FT_LR --n_epochs 100 --pc_augm\n        ```\n    - Our method (SDCoT):\n        ```\n        CUDA_VISIBLE_DEVICES=0 python main/train_sdcot.py --method SDCoT --dataset sunrgbd --num_point 20000 --model_checkpoint_path $MODEL_CHCECKPOINT_PATH --n_novel_class $N_NOVEL_CLASS --pc_augm\n       ```\n\n- Testing:\n    ```\n    CUDA_VISIBLE_DEVICES=0 python main/eval.py --phase test --method $METHOD --dataset sunrgbd --num_point 20000 --model_checkpoint_path $MODEL_CHCECKPOINT_PATH --n_novel_class $N_NOVEL_CLASS --cluster_sampling vote_fps --use_3d_nms --use_cls_nms --per_class_proposal\n    ```   \n\n\nFor ScanNet, you can modify line 47-53 in ./cfg/scannet_cfg.py to set up the class splitting (splitting of base and novel classes), then use the following command to train and evaluate:\n\n- Training\n    - Base training:\n        ```\n        CUDA_VISIBLE_DEVICES=0 python main/train_bt.py --method basetrain --dataset scannet --num_point 40000 --pc_augm\n        ```\n    - Baseline methods (set $FT_LAYERS='last' for \"Freeze and add\" and set $FT_LAYERS='all' for \"Fune-tuning\"):\n        ```\n        CUDA_VISIBLE_DEVICES=0 python main/train_ft.py --method finetune --dataset scannet --num_point 40000 --model_checkpoint_path $MODEL_CHCECKPOINT_PATH --n_novel_class $N_NOVEL_CLASS --ft_layers $FT_LAYERS --lr $FT_LR --n_epochs 100 --pc_augm\n        ```\n    - Our method (SDCoT):\n        ```\n        CUDA_VISIBLE_DEVICES=0 python main/train_sdcot.py --method SDCoT --dataset scannet --num_point 40000 --model_checkpoint_path $MODEL_CHCECKPOINT_PATH --n_novel_class $N_NOVEL_CLASS --pc_augm\n       ```\n\n- Testing:\n    ```\n    CUDA_VISIBLE_DEVICES=0 python main/eval.py --phase test --method $METHOD --dataset scannet --num_point 40000 --model_checkpoint_path $MODEL_CHCECKPOINT_PATH --n_novel_class $N_NOVEL_CLASS --cluster_sampling vote_fps --use_3d_nms --use_cls_nms --per_class_proposal\n    ```   \n\nNote: You can use `CUDA_VISIBLE_DEVICES=0,1,2` to specify which GPU(s) to use. Without specifying CUDA devices, the training will use all the available GPUs and train with data parallel (Note that due to I/O load, training speedup is not linear to the nubmer of GPUs used). \nRun `python main/*.py -h` to see more argument options. While training you can check the log file on its progress, or use the TensorBoard to see loss curves.\n      \n\n## Acknowledgement\nOur implementation leverages on the source code from the following repositories:\n- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://github.com/facebookresearch/votenet)\n- [Pointnet2/Pointnet++ PyTorch](https://github.com/erikwijmans/Pointnet2_PyTorch)\n- [SESS: Self-Ensembling Semi-Supervised 3D Object Detection](https://github.com/Na-Z/sess)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNa-Z%2FSDCoT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FNa-Z%2FSDCoT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNa-Z%2FSDCoT/lists"}