https://github.com/MohamedAfham/CrossPoint

Official implementation of "CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding" (CVPR, 2022)
https://github.com/MohamedAfham/CrossPoint

3d-point-clouds cross-modal-learning deep-learning few-shot-learning object-classification point-cloud self-supervised-learning transfer-learning unsupervised-learning

Last synced: 3 months ago
JSON representation

Official implementation of "CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding" (CVPR, 2022)

• Host: GitHub
• URL: https://github.com/MohamedAfham/CrossPoint
• Owner: MohamedAfham
• Created: 2022-02-28T17:46:17.000Z (almost 3 years ago)
• Default Branch: master
• Last Pushed: 2023-04-27T01:16:04.000Z (almost 2 years ago)
• Last Synced: 2024-08-01T03:43:55.236Z (6 months ago)
• Topics: 3d-point-clouds, cross-modal-learning, deep-learning, few-shot-learning, object-classification, point-cloud, self-supervised-learning, transfer-learning, unsupervised-learning
• Language: Python
• Homepage: https://mohamedafham.github.io/CrossPoint/
• Size: 1.99 MB
• Stars: 228
• Watchers: 7
• Forks: 28
• Open Issues: 10
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding (CVPR'22)

#### [Paper Link](https://arxiv.org/abs/2203.00680) | [Project Page](https://mohamedafham.github.io/CrossPoint/) 

## Citation

If you find our work, this repository, or pretrained models useful, please consider giving a star ⭐ and citation.

```bibtex

@InProceedings{Afham_2022_CVPR,

    author    = {Afham, Mohamed and Dissanayake, Isuru and Dissanayake, Dinithi and Dharmasiri, Amaya and Thilakarathna, Kanchana and Rodrigo, Ranga},

    title     = {CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding},

    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},

    month     = {June},

    year      = {2022},

    pages     = {9902-9912}

}

```

# :rocket: News

* **(Mar 25, 2023)**

  * An implementation supporting PyTorchDistributedDataParallel (DDP) is available [here](https://github.com/auniquesun/CrossPoint-DDP). Thanks to [Jerry Sun](https://auniquesun.github.io/)

* **(Mar 2, 2022)**

  * Paper accepted at CVPR 2022 :tada: 

* **(Mar 2, 2022)** 

  * Training and evaluation codes for [CrossPoint](https://openaccess.thecvf.com/content/CVPR2022/html/Afham_CrossPoint_Self-Supervised_Cross-Modal_Contrastive_Learning_for_3D_Point_Cloud_Understanding_CVPR_2022_paper.html), along with pretrained models are released.

## Dependencies

Refer `requirements.txt` for the required packages.

## Pretrained Models

CrossPoint pretrained models with DGCNN feature extractor are available [here.](https://drive.google.com/drive/folders/10TVEIRUBCh3OPulKI4i2whYAcKVdSURn?usp=sharing)

## Download data

Datasets are available [here](https://drive.google.com/drive/folders/1dAH9R3XDV0z69Bz6lBaftmJJyuckbPmR?usp=sharing). Run the command below to download all the datasets (ShapeNetRender, ModelNet40, ScanObjectNN, ShapeNetPart) to reproduce the results.

```

cd data

source download_data.sh

```

## Train CrossPoint

Refer `scripts/script.sh` for the commands to train CrossPoint.

## Downstream Tasks

### 1. 3D Object Classification 

Run `eval_ssl.ipynb` notebook to perform linear SVM object classification in both ModelNet40 and ScanObjectNN datasets.

### 2. Few-Shot Object Classification

Refer `scripts/fsl_script.sh` to perform few-shot object classification.

### 3. 3D Object Part Segmentation

Refer `scripts/script.sh` for fine-tuning experiment for part segmentation in ShapeNetPart dataset.

## Acknowledgements

Our code borrows heavily from [DGCNN](https://github.com/WangYueFt/dgcnn) repository. We thank the authors of DGCNN for releasing their code. If you use our model, please consider citing them as well.