Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/wix-incubator/DLT
Diffusion Layout Transformer implementation.
https://github.com/wix-incubator/DLT
continous-diffusion ddpm diffusion discrete-diffusion generative-ai generative-models iccv2023 layout-generation layouts magazine publaynet rico web-design
Last synced: 3 months ago
JSON representation
Diffusion Layout Transformer implementation.
- Host: GitHub
- URL: https://github.com/wix-incubator/DLT
- Owner: wix-incubator
- License: apache-2.0
- Created: 2023-08-09T11:12:19.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2023-08-29T10:36:57.000Z (over 1 year ago)
- Last Synced: 2024-10-30T00:39:25.473Z (3 months ago)
- Topics: continous-diffusion, ddpm, diffusion, discrete-diffusion, generative-ai, generative-models, iccv2023, layout-generation, layouts, magazine, publaynet, rico, web-design
- Language: Python
- Homepage: https://wix-incubator.github.io/DLT/
- Size: 3.81 MB
- Stars: 49
- Watchers: 33
- Forks: 2
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-diffusion-categorized - [Code
README
#
[ICCV 23] DLT: Conditioned layout generation with Joint Discrete-Continuous Diffusion Layout Transformer
This repository is an official implementation of DLT paper. Please, refer to the [paper](https://arxiv.org/abs/2110.00000)
for more details and [project page](https://wix-incubator.github.io/DLT/) for general overview.
| Unconditional | Category | Category + Size |
|--------------------------------------------|-----------------------------------|---------------------------------------|
|  |  |  |
### Dev environment
- Operating System: Ubuntu 18.04
- CUDA Version: 11.6
- Python Version: 3.9
### Requirements
All relevant requirements are listed in [environment.yml](environment.yml). We recommend using
[conda](https://docs.conda.io/en/latest/) to create the appropriate environment and install the dependencies:
```bash
conda env create -f environment.yml
conda activate dlt
```
### Datasets
Please download the public datasets at the following webpages. Put it in your folder and update
`./dlt/configs/remote/dataset_config.yaml` accordingly.
1. [RICO](https://interactionmining.org/rico)
2. [PubLayNet](https://github.com/ibm-aur-nlp/PubLayNet)
3. [Magazine](https://xtqiao.com/projects/content_aware_layout/)
### Training
You can train the model using any config script in [configs](./dlt/configs) folder. For example, if you want to train the
provided DLT model on publaynet dataset, the command is as follows:
```bash
cd dlt
python main.py --config configs/remote/dlt_publaynet_config.yaml --workdir
```
Please, see that code is accelerator agnostic. if you don't want to log results to wandb, just set `--workdir test`
in args.
### Evaluation
To generate samples for evaluation on the test set, follow these steps:
- train the model using the above command
- Run the following command:
```bash
# put weights in config.logs folder
DATASET = "publaynet" # or "rico" or "magazine"
python generate_samples.py --config configs/remote/dlt_{$DATASET}_config.yaml \\
--workdir --epoch --cond_type \\
--save True
# get all the metrics
# update path to pickle file in dlt/evaluation/metric_comp.py
./download_fid_model.sh
python metric_comp.py
```
where `` can be: (all, whole_box, loc) - (unconditional, category, category+size) respectively,
`` is the epoch number of the model you want to evaluate, and `` is the path to the folder where
the model weights are saved (e.g. rico_final). The generated samples will be saved in `logs//samples` folder if `save` True.
An output from it is pickle file with generated samples. You can use it to calculate metrics.
The folder with weights after training has this structure:
```
logs
├── magazine_final
│ ├── checkpoints
│ └── samples
├── publaynet_final
│ ├── checkpoints
│ └── samples
└── rico_final
├── checkpoints
└── samples
```
## Citation
If you find this code useful for your research, please cite our paper:
```
@misc{levi2023dlt,
title={DLT: Conditioned layout generation with Joint Discrete-Continuous Diffusion Layout Transformer},
author={Elad Levi and Eli Brosh and Mykola Mykhailych and Meir Perez},
year={2023},
eprint={2303.03755},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```