https://github.com/SamsungLabs/cloud_transformers

[ICCV, 2021] Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks https://arxiv.org/abs/2007.11679
https://github.com/SamsungLabs/cloud_transformers

Last synced: about 2 months ago
JSON representation

[ICCV, 2021] Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks https://arxiv.org/abs/2007.11679

• Host: GitHub
• URL: https://github.com/SamsungLabs/cloud_transformers
• Owner: SamsungLabs
• License: mit
• Created: 2021-08-04T20:52:44.000Z (about 3 years ago)
• Default Branch: main
• Last Pushed: 2023-04-07T10:56:19.000Z (over 1 year ago)
• Last Synced: 2024-07-21T21:43:15.750Z (2 months ago)
• Language: Python
• Size: 93.8 KB
• Stars: 33
• Watchers: 7
• Forks: 5
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks

This is an official PyTorch code repository of the paper "[Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks

](https://arxiv.org/abs/2007.11679)" (**ICCV**, 2021). 

Here, we present a versatile point cloud processing block that yields state-of-the-art results on many tasks.   

The key idea is to process point clouds with many cheap low-dimensional *different* projections followed by standard convolutions. And we do so both *in parallel* and *sequentially*.

# Datasets

We provide links to the datasets we used to train/evaluate. After unpacking and preparation, please edit the dataset path (`data:path` field) in `configs/*.yaml`

* [ShapeNet Single-View 3D Reconstruction](https://github.com/lmb-freiburg/what3d)

* [ShapeNet Inpainting (GRNet version)](https://drive.google.com/drive/folders/1P_W1tz5Q4ZLapUifuOE4rFAZp6L1XTJz)

* [ScanObjectNN](https://github.com/hkust-vgd/scanobjectnn)

* [S3DIS KPConv protocol](https://github.com/zeliu98/CloserLook3D/tree/master/pytorch)

* [S3DIS 1 x 1 protocol](https://shapenet.cs.stanford.edu/media/indoor3d_sem_seg_hdf5_data.zip)

# Pre-trained models

We provide our pre-trained models' weights in a single [archive](https://disk.yandex.ru/d/IqL8tMfleVTnyQ).

# Building Dependencies

To install and build all the modules required, please run:

```

bash ./install_deps.sh

```

# Code Structure

In `layers/cloud_transform.py` the core operations are implemented (rasterization `Splat` and de-rasterization `Slice`). 

While in `layers\mutihead_ct_*.py` we provide slightly different versions of Multi-Headed Cloud Transform (MHCT).

The model zoo is situated in `model_zoo`, where the models for corresponding tasks are constructed of Multi-Headed Cloud Transforms.

# Run

We train our models in multi-GPU setting using DistributedDataParallel. To train on `n` GPUs, please run the following commands:

```

python train_${SCRIPT_NAME}.py ${EXP_NAME} -c configs/${CONFIG_NAME}.yaml --master localhost:3315 --rank 0 --num_nodes n

...

python train_${SCRIPT_NAME}.py ${EXP_NAME} -c configs/${CONFIG_NAME}.yaml --master localhost:3315 --rank  --num_nodes n

```

The semantics for evaluation scripts is almost the same:

```bash

python eval_${SCRIPT_NAME}.py ${EXP_NAME} -c configs/eval/${CONFIG_NAME}.yaml

```

# Cite

If you find our work helpful, please do not hesitate to cite us.

```

@inproceedings{mazur2021cloudtransformers,

  title={Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks},

  author={Mazur, Kirill and Lempitsky, Victor},

  booktitle={International Conference on Computer Vision (ICCV)},

  year={2021}

}

```