{"id":16835913,"url":"https://github.com/xingangpan/shadegan","last_synced_at":"2025-03-17T04:32:43.785Z","repository":{"id":46697480,"uuid":"422550617","full_name":"XingangPan/ShadeGAN","owner":"XingangPan","description":"Code for ShadeGAN (NeurIPS2021)","archived":false,"fork":false,"pushed_at":"2022-10-17T19:34:12.000Z","size":13962,"stargazers_count":145,"open_issues_count":3,"forks_count":22,"subscribers_count":15,"default_branch":"main","last_synced_at":"2025-02-27T18:02:55.054Z","etag":null,"topics":["3d-aware-image-synthesis","deep-learning","gan","generative-adversarial-network"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2110.15678","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/XingangPan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-29T11:32:34.000Z","updated_at":"2025-01-09T23:21:23.000Z","dependencies_parsed_at":"2022-09-13T15:12:29.174Z","dependency_job_id":null,"html_url":"https://github.com/XingangPan/ShadeGAN","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XingangPan%2FShadeGAN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XingangPan%2FShadeGAN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XingangPan%2FShadeGAN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XingangPan%2FShadeGAN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/XingangPan","download_url":"https://codeload.github.com/XingangPan/ShadeGAN/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243841990,"owners_count":20356585,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-aware-image-synthesis","deep-learning","gan","generative-adversarial-network"],"created_at":"2024-10-13T12:11:45.439Z","updated_at":"2025-03-17T04:32:42.834Z","avatar_url":"https://github.com/XingangPan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis\n### [Project Page](https://xingangpan.github.io/projects/ShadeGAN.html) | [Paper](https://arxiv.org/pdf/2110.15678.pdf)\n\u003e **A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis** \u003cbr\u003e\n\u003e [Xingang Pan](https://xingangpan.github.io/), [Xudong Xu](https://sheldontsui.github.io/), [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/), [Christian Theobalt](https://people.mpi-inf.mpg.de/~theobalt/), [Bo Dai](http://daibo.info/)\u003cbr\u003e\n\u003e *NeurIPS2021*\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"ShadeGAN_demo.gif\", width=\"900\"\u003e\n\u003c/p\u003e\n\nIn this repository, we present **ShadeGAN**, a generative model for shape-accurate 3D-aware image synthesis.  \nOur method adopts a multi-lighting constraint that resolves the shape-color ambiguity and leads to more accurate 3D shapes.\n\n## Requirements\n\n* python\u003e=3.7\n* [pytorch](https://pytorch.org/)\u003e=1.8.1\n* other dependencies\n    ```sh\n    pip install -r requirements.txt\n    ```\n\n## Lighting priors and pretrained weights\n\nTo download lighting priors and pretrained weights, simply run:\n```sh\nsh scripts/download.sh\n```\n\n## Testing\n\n#### Rendering Images\n\n```sh\nsh scripts/render.sh\n```\nThis would generate images of multiple viewpoints and lightings by default.\n\n#### Extracting 3D Shapes\n\n```sh\npython extract_shapes.py weights/pretrain/celeba_noview/generator.pth --curriculum CelebA_ShadeGAN_noview --seed 0 5 8 43 --ema\n```\n\n#### Evaluation Metrics\n\nTo evaluate metrics, you need to download dataset first as mentioned in Training below.\n\nTo generate real images for evaluation run  \n```sh\npython fid_evaluation.py --dataset CelebA --dataset_path path/to/dataset/\\*.jpg\n```\n\nTo calculate fid/kid/inception scores run  \n```sh\npython eval_metrics.py weights/pretrain/celeba_view/generator.pth --real_image_dir EvalImages/CelebA_real_images_128 --curriculum CelebA_ShadeGAN_view --num_steps 6 --delta 0.06423 --ema\n```\nwhere `delta` denotes the integration range along the ray for volume rendering. We record the `delta` for different pretrained models at `weights/pretrain/delta.txt`.\n\n## Training\n\n#### Download datasets\n\nCelebA: Download at [CelebA website](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)  \nCats: Please follow the instruction at [GRAF](https://github.com/autonomousvision/graf)  \nBFM: Please follow the instruction at [Unsup3d](https://github.com/elliottwu/unsup3d)\n\n#### Start Training\n\nBefore training, please update the `dataset_path` field in the curriculum to point to your images.\n\nWe provide our training scripts under `scripts` folder. For example, to train ShadeGAN on the CelebA dataset, simply run:\n```sh\nsh scripts/run_celeba.sh\n```\nThis would run on 4 GPUs by default. You may change the number of GPUs by revising `CUDA_VISIBLE_DEVICES` in the scripts.\n\n## Tips\n\n* If the number of GPUs for training is changed, you may need to adjust the `batch_size` in the curriculum to keep the total batchsize the same.  \n* In case of 'out of memory', you could increase `batch_split` in the curriculum to reduce memory consumption.  \n* For CelebA and BFM, both models depedent and independent of viewing direction are provided. The former has better FID while the latter has slightly better shapes.\n* For BFM dataset, training could sometimes fall to the hollow-face solution where the face is concave. To prevent this, you could initialize with our pretrained models such as `weights/pretrain/bfm_noview/pretrain5k-*`.\n\n## Acknowledgement\n\nThis code is developed based on the [official pi-GAN implementation](https://github.com/marcoamonteiro/pi-GAN).\n\n## Citation\n\nIf you find our work useful in your research, please cite:\n```\n@inproceedings{pan2021shadegan,\n    title   = {A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis},\n    author  = {Pan, Xingang and Xu, Xudong and Loy, Chen Change and Theobalt, Christian and Dai, Bo},\n    booktitle = {Advances in Neural Information Processing Systems},\n    year    = {2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxingangpan%2Fshadegan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxingangpan%2Fshadegan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxingangpan%2Fshadegan/lists"}