An open API service indexing awesome lists of open source software.

https://github.com/JudgementH/RefAny3D


https://github.com/JudgementH/RefAny3D

Last synced: 4 months ago
JSON representation

Awesome Lists containing this project

README

          

# RefAny3D: 3D Asset-Referenced Diffusion Models for Image Generation

 
 
HuggingFace
HuggingFace

![teaser](assets/teaser.png)

## TODO List
- [x] Inference code and pretrained models.
- [x] Training code.
- [x] Training dataset.

## Quickstart

### Create environment
1. Clone the repository and create a conda environment:
```
git clone https://github.com/JudgementH/RefAny3D.git
conda create -n r3d python=3.10
conda activate r3d
```
2. Install requirements
```
pip install -r requirements.txt
```

### Inference
```
python demo.py \
--prompt "" \
--glb_path "" \
--output_dir ""

```

Alternatively, we provide some example scripts:

```
bash scripts/demo_chair.sh

bash scripts/demo_traffic_cone.sh
```

## Training

### Download Dataset

First, download the training [dataset](https://huggingface.co/datasets/JudgementH/RefAny3D-Dataset) and organize it in the appropriate directory structure.

### Configure Training

1. Edit the training configuration file `train/config/train_refany3d.yaml`:
- Update `json_file` path under `train.dataset` section (default: `"./dataset/meta_data.json"`)
- Update `data_root_path` path under `train.dataset` section (default: `"./dataset"`)

Example configuration:
```yaml
train:
dataset:
json_file: ""
data_root_path: ""
```

### Run Training

Execute the training script:

```
bash train/scripts/train_refany3d.sh
```

**Note:** The training script uses `accelerate` with 8 GPU processes by default. You can modify the number of processes in `train/scripts/train_refany3d.sh` by adjusting the `--num_processes` parameter.