https://github.com/interdigitalinc/tearingnet
Source code for the paper "TearingNet: Point Cloud Autoencoder to Learn Topology-Friendly Representations"
https://github.com/interdigitalinc/tearingnet
Last synced: about 1 year ago
JSON representation
Source code for the paper "TearingNet: Point Cloud Autoencoder to Learn Topology-Friendly Representations"
- Host: GitHub
- URL: https://github.com/interdigitalinc/tearingnet
- Owner: InterDigitalInc
- License: other
- Created: 2021-08-30T13:52:21.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2021-08-30T14:46:05.000Z (almost 5 years ago)
- Last Synced: 2025-03-24T21:11:21.089Z (about 1 year ago)
- Language: Python
- Size: 11.1 MB
- Stars: 27
- Watchers: 0
- Forks: 5
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# TearingNet: Point Cloud Autoencoder to Learn Topology-Friendly Representations
Created by Jiahao Pang, Duanshun Li, and Dong Tian from InterDigital
## Introduction
This repository contains the implementation of our TearingNet paper accepted in CVPR 2021.
Given a point cloud dataset containing objects with various genera, or scenes with multiple objects, we propose the TearingNet, which is an autoencoder tackling the challenging task of representing the point clouds using a fixed-length descriptor.
Unlike existing works directly deforming predefined primitives of genus zero (e.g., a 2D square patch) to an object-level point cloud, our TearingNet
is characterized by a proposed Tearing network module and a Folding network module interacting with each other iteratively.
Particularly, the Tearing network module learns the point cloud topology explicitly.
By breaking the edges of a primitive graph, it tears the graph into patches or with holes to emulate the topology of a target point cloud, leading to faithful reconstructions.
## Installation
* We use Python 3.6, PyTorch 1.3.1 and CUDA 10.0, example commands to set up a virtual environment with anaconda are:
```
conda create tearingnet python=3.6
conda activate tearingnet
conda install pytorch=1.3.1 torchvision=0.4.2 cudatoolkit=10.0 -c pytorch
```
* Clone our repo to a folder, assume the name of the folder is ``TearingNet``.
* Checkout the nndistance folder from 3D Point Capsule Network, put it under ``TearingNet/util``, build it according to the instructions of 3D Point Capsule Network.
* We adopt the ShapeNet part data loader from 3D Point Capsule Network. Checkout shapenet_part_loader.py and put it under the ``TearingNet/dataloaders``.
* Install the TensorboardX, Open3D and the h5py packages, example commands are:
```
conda install -c open3d-admin open3d
conda install -c conda-forge tensorboardx
conda install -c anaconda h5py
```
## Data Preparation
### KITTI Multi-Object Dataset
* Our KITTI Multi-Object (KIMO) Dataset is constructed with kitti_dataset.py of PCDet (commit 95d2ab5). Please clone and install PCDet, then prepare the KITTI dataset according to their instructions.
* Assume the name of the cloned folder is ``PCDet``, please replace the ``create_groundtruth_database()`` function in ``kitti_dataset.py`` by our modified one provided in ``TearingNet/util/pcdet_create_grouth_database.py``.
* Prepare the KITTI dataset, then generate the data infos according to the instructions in the README.md of PCDet.
* Create the folders ``TearingNet/dataset`` and ``TearingNet/dataset/kittimulobj`` then put the newly-generated folder ``PCDet/data/kitti/kitti_single`` under ``TearingNet/dataset/kittimulobj``. Also, put the newly-generated file ``PCDet/data/kitti/kitti_dbinfos_object.pkl`` under the ``TearingNet/dataset/kittimulobj`` folder.
* Instead of assembling several single-object point clouds together and write down as a multi-object point cloud, we generate the parameters that parameterize the multi-object point clouds then assemble them on the fly during training/testing. To obtain the parameters, run our prepared scripts as follows under the ``TearingNet`` folder. These scripts generate the training and testing splits of the KIMO-5 dataset:
```
./scripts/launch.sh ./scripts/gen_data/gen_kitti_mulobj_train_5x5.sh
./scripts/launch.sh ./scripts/gen_data/gen_kitti_mulobj_test_5x5.sh
```
* The file structure of the KIMO dataset after these steps becomes:
```
kittimulobj
├── kitti_dbinfos_object.pkl
├── kitti_mulobj_param_test_5x5_2048.pkl
├── kitti_mulobj_param_train_5x5_2048.pkl
└── kitti_single
├── 0_0_Pedestrian.bin
├── 1000_0_Car.bin
├── 1000_1_Car.bin
├── 1000_2_Van.bin
...
```
### CAD Model Multi-Object Dataset
* Create the ``TearingNet/dataset/cadmulobj`` folder to hold the CAD Model Multi-Object (CAMO) dataset.
* Our CAMO dataset is based on the CAD models of "person", "car", "cone", "plant" from ModelNet40, and "motorbike" from the ShapeNetPart dataset. Use the scripts download_shapenet_part16_catagories.sh and download_modelnet40_same_with_pointnet.sh from 3D Point Capsule Network to download these two datasets, then orgainze them according to the following file structure under the ``TearingNet/dataset`` folder:
```
dataset
├── cadmulobj
├── kittimulobj
├── modelnet40
│ └── modelnet40_ply_hdf5_2048
│ ├── ply_data_test0.h5
│ ├── ply_data_test_0_id2file.json
│ ├── ply_data_test1.h5
│ ├── ply_data_test_1_id2file.json
│ ...
└── shapenet_part
├── shapenetcore_partanno_segmentation_benchmark_v0
│ ├── 02691156
│ │ ├── points
│ │ │ ├── 1021a0914a7207aff927ed529ad90a11.pts
│ │ │ ├── 103c9e43cdf6501c62b600da24e0965.pts
│ │ │ ├── 105f7f51e4140ee4b6b87e72ead132ed.pts
...
```
* Extract the "person", "car", "cone" and "plant" models from ModelNet40, and the "motorbike" models from the ShapeNet part dataset, by running the following Python script under the ``TearingNet`` folder:
```
python util/cad_models_collector.py
```
* The previous step generates the file ``TearingNet/dataset/cadmulobj/cad_models.npy``, based on which we generate the parameters for the CAMO dataset. To do so, launch the following scripts:
```
./scripts/launch.sh ./scripts/gen_data/gen_cad_mulobj_train_5x5.sh
./scripts/launch.sh ./scripts/gen_data/gen_cad_mulobj_test_5x5.sh
```
* The file structure of the CAMO dataset after these steps becomes:
```
cadmulobj
├── cad_models.npy
├── cad_mulobj_param_test_5x5.npy
└── cad_mulobj_param_train_5x5.npy
```
## Experiments
### Training
We employ a two-stage training strategy to train the TearingNet. The first step is to train a FoldingNet (E-Net & F-Net in paper). Take the KIMO dataset as an example, launch the following scripts under the ``TearingNet`` folder:
```
./scripts/launch.sh ./scripts/experiments/train_folding_kitti.sh
```
Having finished the first step, a pretrained model will be saved in ``TearingNet/results/train_folding_kitti``. To load the pretrained FoldingNet into a TearingNet configuration and perform training, launch the following scripts:
```
./scripts/launch.sh ./scripts/experiments/train_tearing_kitti.sh
```
To see the meanings of the parameters in ``train_folding_kitti.sh`` and ``train_tearing_kitti.sh``, check the Python script ``TearinNet/util/option_handler.py``.
### Reconstruction
To perform the reconstruction experiment with the trained model, launch the following scripts:
```
./scripts/launch.sh ./scripts/experiments/reconstruction.sh
```
One may write down the reconstructions in PLY format by setting a positive ``PC_WRITE_FREQ`` value. Again, please refer to ``TearinNet/util/option_handler.py`` for the meanings of individual parameters.
### Counting
To perform the counting experiment with the trained model, launch the following scripts:
```
./scripts/launch.sh ./scripts/experiments/counting.sh
```
## Citing this Work
Please cite our work if you find it useful for your research:
```
@inproceedings{pang2021tearingnet,
title={TearingNet: Point Cloud Autoencoder to Learn Topology-Friendly Representations},
author={Pang, Jiahao and Li, Duanshun, and Tian, Dong},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2021}
}
```
## Related Projects
* 3D Point Capsule Networks
* AtlasNet
* AtlasNetV2
* PCDet
