https://github.com/TRI-ML/sdflabel

Official PyTorch implementation of CVPR 2020 oral "Autolabeling 3D Objects With Differentiable Rendering of SDF Shape Priors"
https://github.com/TRI-ML/sdflabel

Last synced: about 2 months ago
JSON representation

Official PyTorch implementation of CVPR 2020 oral "Autolabeling 3D Objects With Differentiable Rendering of SDF Shape Priors"

• Host: GitHub
• URL: https://github.com/TRI-ML/sdflabel
• Owner: TRI-ML
• License: mit
• Created: 2020-03-29T18:59:36.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2021-10-19T15:04:59.000Z (almost 3 years ago)
• Last Synced: 2024-07-21T21:41:15.287Z (2 months ago)
• Language: Python
• Homepage:
• Size: 10.5 MB
• Stars: 159
• Watchers: 20
• Forks: 20
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
## Autolabeling 3D Objects With Differentiable Rendering of SDF Shape Priors

Official [PyTorch](https://pytorch.org/) implementation of the CVPR 2020 paper "Autolabeling 3D Objects With Differentiable Rendering of SDF Shape Priors" by the ML Team at [Toyota Research Institute (TRI)](https://www.tri.global/), cf. [References](#references) below.

[**[Full paper]**](https://arxiv.org/pdf/1911.11288.pdf) [**[YouTube]**](https://www.youtube.com/watch?v=Utzj-kfWHP4)



 









## Setting up your environment

To set up the environment using conda, use the following commands:

```

conda env create -n sdflabel -f environment.yml

conda activate sdflabel

```

Add the sdfrenderer directory to *PYTHONPATH*:

```

export PYTHONPATH="${PYTHONPATH}:/path/to/sdfrenderer"

```

## Optimization demo

To run the optimization demo, first download the [data folder](https://drive.google.com/file/d/1cvLeXDhaghjzCK-gmnQbxh8rTvd71miw/view?usp=sharing).

Then, extract the archive to the root folder of the project and run the following command:

```

python main.py configs/config_refine.ini --demo

```

## Training CSS network

To train the CSS network, run the following command:

```

python main.py configs/config_train.ini --train

```

### Dataset format

The dataset of crops represents a collection of detected *RGB patches (CSS input)*, corresponding *NOCS patches (CSS output)*, and a JSON DB file comprising the patch relevant information (most importantly *SDF latent vectors* corresponding to the depicted 3D models).

An example of such dataset is located in the *data/db/crops* folder.

## Optimization

Download [KITTI 3D](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) and modify the *kitti_path* in the config file *config_refine.ini* accordingly.

To run optimization on the KITTI 3D dataset, run the following command:

```

python main.py configs/config_refine.ini --refine

```

Upon completion, autolabels will be stored to the *output* folder specified in the config file (*output* -> *labels*).

To evaluate the generated dump, run:

```

python main.py configs/config_refine.ini --evaluate

```

## License

The source code is released under the [MIT license](LICENSE.md).

## References

#### Autolabeling 3D Objects With Differentiable Rendering of SDF Shape Priors (CVPR 2020 oral)

*Sergey Zakharov\*, Wadim Kehl\*, Arjun Bhargava, Adrien Gaidon*

```

@inproceedings{sdflabel,

author = {Sergey Zakharov and Wadim Kehl and Arjun Bhargava and Adrien Gaidon},

title = {Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priors},

booktitle = {IEEE Computer Vision and Pattern Recognition (CVPR)},

month = {June},

year = {2020}

}

```