https://cyberagentailab.github.io/layout-dm/
LayoutDM: Discrete Diffusion Model for Controllable Layout Generation [Inoue+, CVPR2023]
https://cyberagentailab.github.io/layout-dm/
cvpr2023 diffusion-models generative-ai layout pytorch
Last synced: about 1 month ago
JSON representation
LayoutDM: Discrete Diffusion Model for Controllable Layout Generation [Inoue+, CVPR2023]
- Host: GitHub
- URL: https://cyberagentailab.github.io/layout-dm/
- Owner: CyberAgentAILab
- License: apache-2.0
- Created: 2023-03-08T08:53:51.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2023-10-24T03:01:49.000Z (over 1 year ago)
- Last Synced: 2024-11-15T07:57:44.504Z (6 months ago)
- Topics: cvpr2023, diffusion-models, generative-ai, layout, pytorch
- Language: Python
- Homepage: https://cyberagentailab.github.io/layout-dm
- Size: 978 KB
- Stars: 228
- Watchers: 4
- Forks: 26
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-diffusion-categorized - [Project
README
# LayoutDM: Discrete Diffusion Model for Controllable Layout Generation (CVPR2023)
This repository is an official implementation of the paper titled above.
Please refer to [project page](https://cyberagentailab.github.io/layout-dm/) or [paper](https://arxiv.org/abs/2303.08137) for more details.
## Setup
Here we describe the setup required for the model training and evaluation.
### Requirements
We check the reproducibility under this environment.
- Python3.7
- CUDA 11.3
- [PyTorch](https://pytorch.org/get-started/locally/) 1.10
We recommend using Poetry (all settings and dependencies in [pyproject.toml](pyproject.toml)).
Pytorch-geometry provides independent pre-build wheel for a *combination* of PyTorch and CUDA version (see [PyG:Installation](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html
) for details). If your environment does not match the one above, please update the dependencies.
### How to install
1. Install poetry (see [official docs](https://python-poetry.org/docs/)). We recommend to make a virtualenv and install poetry inside it.
```bash
curl -sSL https://install.python-poetry.org | python3 -
```
2. Install dependencies (it may be slow..)
```bash
poetry install
```
3. Download resources and unzip
``` bash
wget https://github.com/CyberAgentAILab/layout-dm/releases/download/v1.0.0/layoutdm_starter.zip
unzip layoutdm_starter.zip
```
The data is decompressed to the following structure:
```
download
- clustering_weights
- datasets
- fid_weights
- pretrained_weights
```
## Experiment
**Important**: we find some critical errors that cannot be fixed quickly in using multiple GPUs. Please set `CUDA_VISIBLE_DEVICES=` to force the model use a single GPU.
Note: our main framework is based on [hydra](https://hydra.cc/). It is convenient to handle dozens of arguments hierarchically but may require some additional efforts if one is new to hydra.
### Demo
Please run a jupyter notebook in [notebooks/demo.ipynb](notebooks/demo.ipynb). You can get and render the results of six layout generation tasks on two datasets (Rico and PubLayNet).
### Training
You can also train your own model from scratch, for example by
```bash
bash bin/train.sh rico25 layoutdm
```
, where the first and second argument specifies the dataset ([choices](src/trainer/trainer/config/dataset)) and the type of experiment ([choices](src/trainer/trainer/config/experiment)), respectively.
Note that for training/testing, style of the arguments is `key=value` because we use hydra, unlike popular `--key value` (e.g., [argparse](https://docs.python.org/3/library/argparse.html)).
### Testing
```bash
poetry run python3 -m src.trainer.trainer.test \
cond= \
job_dir= \
result_dir= \
```
`` can be: (unconditional, c, cwh, partial, refinement, relation)
For example, if you want to test the provided LayoutDM model on `C->S+P`, the command is as follows:
```
poetry run python3 -m src.trainer.trainer.test cond=c dataset_dir=./download/datasets job_dir=./download/pretrained_weights/layoutdm_rico result_dir=tmp/dummy_results
```
Please refer to [TestConfig](src/trainer/trainer/hydra_configs.py#L12) for more options available.
Below are some popular options for
- `is_validation=true`: used to evaluate the generation performance on validation set instead of test set. This must be used when tuning the hyper-parameters.
- `sampling=top_p top_p=`: use top-p sampling with p= instead of default sampling.
### Evaluation
```bash
poetry run python3 eval.py
```
## Citation
If you find this code useful for your research, please cite our paper:
```
@inproceedings{inoue2023layout,
title={{LayoutDM: Discrete Diffusion Model for Controllable Layout Generation}},
author={Naoto Inoue and Kotaro Kikuchi and Edgar Simo-Serra and Mayu Otani and Kota Yamaguchi},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2023},
pages={10167-10176},
}
```