https://github.com/yifanfeng97/multi-modal-generation-for-shrec22

Multi-modal data generation for 3D objects.
https://github.com/yifanfeng97/multi-modal-generation-for-shrec22

3d blender data-generation multi-modal multi-view pointclouds voxel

Last synced: 4 months ago
JSON representation

Multi-modal data generation for 3D objects.

• Host: GitHub
• URL: https://github.com/yifanfeng97/multi-modal-generation-for-shrec22
• Owner: yifanfeng97
• Created: 2022-07-15T09:56:15.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2022-08-07T15:48:07.000Z (over 2 years ago)
• Last Synced: 2024-08-01T03:40:45.902Z (7 months ago)
• Topics: 3d, blender, data-generation, multi-modal, multi-view, pointclouds, voxel
• Language: Python
• Homepage:
• Size: 1.47 MB
• Stars: 6
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

        
# Introduction

In our paper "SHREC'22: Open-Set 3D Object Retrieval", we have released two datasets for open-set 3D object retrieval. The two datasets are generated based on the [Modelnet40](https://modelnet.cs.princeton.edu/) dataset. Here, we release the core code for multi-modal data generation including **Voxel Modality Generation**, **Pointcloud Modality Generation**, and **24 Images Modality Generation**.

The computed fully datasets can be download [here](https://www.moon-lab.tech/shrec22). Examlpe code can be found [here](https://github.com/yifanfeng97/OS-MN40-Example).

![intro](docs/graphical_abstract.jpg)

# Settings for generating voxel and pointcloud modalities

Create a virtual environment with command

```bash

conda create -n shrec22 python=3.8

```

Then, activate the environment, and install the following libraries with pip

```bash

pip install open3d, Pillow, numpy, rich, scipy

```

### Usage

Run the following code. The generated voxel and pointcloud will be stored in the `out` direction.

```bash

python generate_voxel_pointcloud.py

```

## Settings for generating 24 images modality

- blender 3.0

Install the PIL library for the inside python in blender. Change the work direction to the direction of the  inside python of blender. Then, run the following command:

```bash

pip install Pillow

```

### Usage

Open `generate_24_images.blend` with blender 3.0, then run the code. Then, rendered 24 images can be found in the `out` folder.

# Evaluation

Given a given distance matrix, you can evaluate its mAP, NDCG, ANMRR, NN with functions in the metrics.py. 

# Citation

```

@article{feng2022shrec,

  title={SHREC’22 track: Open-Set 3D Object Retrieval},

  author={Feng, Yifan and Gao, Yue and Zhao, Xibin and Guo, Yandong and Bagewadi, Nihar and Bui, Nhat-Tan and Dao, Hieu and Gangisetty, Shankar and Guan, Ripeng and Han, Xie and others},

  journal={Computers \& Graphics},

  year={2022},

  publisher={Elsevier}

}

```