{"id":13634890,"url":"https://github.com/davidhalladay/LayoutTransformer","last_synced_at":"2025-04-18T23:33:15.674Z","repository":{"id":89117469,"uuid":"344035305","full_name":"davidhalladay/LayoutTransformer","owner":"davidhalladay","description":"Cheng-Fu Yang*, Wan-Cyuan Fan*, Fu-En Yang, Yu-Chiang Frank Wang, \"LayoutTransformer: Scene Layout Generation with Conceptual and Spatial Diversity\", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.","archived":false,"fork":false,"pushed_at":"2022-04-03T04:58:05.000Z","size":1289,"stargazers_count":58,"open_issues_count":4,"forks_count":4,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-08-02T00:21:42.001Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/davidhalladay.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-03T07:11:39.000Z","updated_at":"2024-06-03T09:52:23.000Z","dependencies_parsed_at":"2023-06-14T00:45:52.994Z","dependency_job_id":null,"html_url":"https://github.com/davidhalladay/LayoutTransformer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidhalladay%2FLayoutTransformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidhalladay%2FLayoutTransformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidhalladay%2FLayoutTransformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidhalladay%2FLayoutTransformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/davidhalladay","download_url":"https://codeload.github.com/davidhalladay/LayoutTransformer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223786390,"owners_count":17202581,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T00:00:37.013Z","updated_at":"2024-11-09T04:31:16.157Z","avatar_url":"https://github.com/davidhalladay.png","language":"Jupyter Notebook","funding_links":[],"categories":["CV"],"sub_categories":["Generation"],"readme":"# [LayoutTransformer-Scene-Layout-Generation-with-Conceptual-and-Spatial-Diversity](https://openaccess.thecvf.com/content/CVPR2021/papers/Yang_LayoutTransformer_Scene_Layout_Generation_With_Conceptual_and_Spatial_Diversity_CVPR_2021_paper.pdf)\nCheng-Fu Yang*, Wan-Cyuan Fan*, Fu-En Yang, Yu-Chiang Frank Wang, \"LayoutTransformer: Scene Layout Generation with Conceptual and Spatial Diversity\", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.\n\n# LayoutTransformer \nPytorch implementation for LT-Net. The goal is to generate scene layout with conceptual and spatial diversity.\n\n### Overview\n\u003cimg src=\"./figures/archi.png\" width=\"940px\" height=\"360px\"/\u003e\n\n### UPdates\n- The training code on the VG-msdn dataset might have some minor errors. We will fix them ASAP.\n\n### Data\n- Please setup conda envirnment first by following command.\n    - Create conda env\n```\nconda create -n ltnet python=3.6\nconda activate ltnet\n```\n    - Install pip packages\n```\npip install -r requirements.txt \n```\n\n### Data\n- COCO dataset\n    - Download the annotations from [COCO](https://cocodataset.org/#download).\n    - i.e., 2017 Train/Val annotations [241MB] and 2017 Stuff Train/Val annotations [1.1GB]\n    - Extract the annotations to `data/coco/`\n- VG-MSDN dataset\n    - Download the VG-MSDN dataset from [VG-MSDN](https://drive.google.com/file/d/1WjetLwwH3CptxACrXnc1NCcccWUVDO76/view). (This dataset origins from [FactorizableNet](https://github.com/yikang-li/FactorizableNet))\n    - Extract the annotations (i.e., all json files) to `data/vg_msdn/`\n\n### Training\nAll code was developed and tested on Ubuntu 20.04 with Python 3.7 (Anaconda) and PyTorch 1.7.1.\n\n#### Pre-train the Obj/Rel Rredictor\n- Pre-train Predictor module for COCO dataset:\n```\npython3 train.py --cfg_path ./configs/coco/coco_pretrain.yaml\n```\n- Pre-train Predictor model for VG-MSDN dataset: \n```\npython3 train.py --cfg_path ./configs/vg_msdn/vg_msdn_pretrain.yaml\n```\n#### Full module\n- Train full model for COCO dataset:\n```\npython3 train.py --cfg_path ./configs/coco/coco_seq2seq_v9_ablation_4.yaml\n```\n- Train full model for VG-MSDN dataset: \n```\npython3 train.py --cfg_path ./configs/vg_msdn/vg_msdn_seq2seq_v24.yaml\n```\n\n`*.yml` files include configuration for training and testing.\n\n\u003e Please note that you might need to modify the config file to fit the corresponding path on your device if the data is placed in other places.\n\n### Pretrained Model Weights\nGoogle drive: [Download](https://drive.google.com/drive/folders/1pPJxX0ih6pgUpKjeIjIICso6SpOGHoaI?usp=sharing)\n#### Obj/Rel Predictor \n- [COCO](https://drive.google.com/drive/folders/1I6Hfkdv58_mkT7JJhnIR5we3TTM_L-mN?usp=sharing). Download and save it to `saved/coco_F_pretrain_no_linear`\n- [VG-MSDN](https://drive.google.com/drive/folders/1E3ZQgMEAr5yoIRzcQHmUltfLOnrgHVch?usp=sharing). Download and save it to `saved/vg_msdn_F_pretrain_no_linear`\n#### LT-Net Full Model \n- [COCO](https://drive.google.com/drive/folders/1pDHGIob1nc480AysXgdAqO8BIBUaS9r-?usp=sharing). Download and save it to `saved/coco_F_seq2seq_v9_ablation_4`\n- [VG-MSDN](https://drive.google.com/drive/folders/1F9J2t9QAHCdcSYZKI3lLkDJcVAQh2jpD?usp=sharing). Download and save it to `saved/vg_msdn_F_seq2seq_v24`\n\n### Evaluation\n\n#### LayoutTransformer full model   \n- Evaluate full model for COCO dataset: (Please download or train your LayoutTransformer for COCO first.)\n```\npython3 train.py --cfg_path [PATH_TO_CONFIG_FILE] --checkpoint [PATH_TO_THE_WEIGHT_FOR_LAYOUTTRASFORMER] --eval_only\n```\nFor example,\n```\npython3 train.py --cfg_path configs/coco/coco_seq2seq_v9_ablation_4.yaml --checkpoint ./saved/coco_F_seq2seq_v9_ablation_4/checkpoint_50_0.44139538748348955.pth --eval_only\n```\n- Evaluate full model for VG-MSDN dataset: (Please download or train your LayoutTransformer for vg-msdn first.)\n```\npython3 train.py --cfg_path [PATH_TO_CONFIG_FILE] --checkpoint [PATH_TO_THE_WEIGHT_FOR_LAYOUTTRASFORMER] --eval_only\n```\nFor example,\n```\npython3 train.py --cfg_path configs/vg_msdn/vg_msdn_seq2seq_v24.yaml --checkpoint ./saved/vg_msdn_F_seq2seq_v24/checkpoint_50_0.16316922369277578.pth --eval_only\n```\n### Citation\n\nIf you find this useful for your research, please use the following.\n\n```\n@InProceedings{Yang_2021_CVPR,\n    author    = {Yang, Cheng-Fu and Fan, Wan-Cyuan and Yang, Fu-En and Wang, Yu-Chiang Frank},\n    title     = {LayoutTransformer: Scene Layout Generation With Conceptual and Spatial Diversity},\n    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n    month     = {June},\n    year      = {2021},\n    pages     = {3732-3741}\n}\n```\n\n### Acknowledgements\nThis code borrows heavily from [Transformer](https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/transformer.py) repository. Many thanks.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidhalladay%2FLayoutTransformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidhalladay%2FLayoutTransformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidhalladay%2FLayoutTransformer/lists"}