{"id":20002329,"url":"https://github.com/apchenstu/sofgan","last_synced_at":"2025-04-12T23:29:28.621Z","repository":{"id":40676035,"uuid":"277756367","full_name":"apchenstu/sofgan","owner":"apchenstu","description":"[TOG 2022] SofGAN: A Portrait Image Generator with Dynamic Styling","archived":false,"fork":false,"pushed_at":"2023-10-17T22:12:26.000Z","size":123601,"stargazers_count":772,"open_issues_count":20,"forks_count":102,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-04-04T02:08:35.413Z","etag":null,"topics":["3drendering","faces","gan"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apchenstu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-07T08:08:56.000Z","updated_at":"2025-03-16T02:04:32.000Z","dependencies_parsed_at":"2024-11-13T05:32:33.991Z","dependency_job_id":null,"html_url":"https://github.com/apchenstu/sofgan","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apchenstu%2Fsofgan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apchenstu%2Fsofgan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apchenstu%2Fsofgan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apchenstu%2Fsofgan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apchenstu","download_url":"https://codeload.github.com/apchenstu/sofgan/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248646842,"owners_count":21139079,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3drendering","faces","gan"],"created_at":"2024-11-13T05:20:46.665Z","updated_at":"2025-04-12T23:29:28.602Z","avatar_url":"https://github.com/apchenstu.png","language":"Python","funding_links":[],"categories":["Image Generation \u0026 Editing"],"sub_categories":[],"readme":"# SofGAN (TOG 2022)\n## [Project page](https://apchenstu.github.io/sofgan/) |  [Paper](https://arxiv.org/abs/2007.03780)\nThis repository contains the official **PyTorch** implementation for the paper: [SofGAN: A Portrait Image Generator with Dynamic Styling](https://arxiv.org/abs/2007.03780).\nWe propose a **SofGAN** image generator to decouple the latent space of portraits into two subspaces: a geometry space and a texture space.\nExperiments on **SofGAN** show that our system can generate high quality portrait images with independently controllable geometry and texture attributes.\u003cbr\u003e\u003cbr\u003e\n\n![Teaser](https://github.com/apchenstu/apchenstu.github.io/blob/master/sofgan/img/semantic_level.png)\n\n### Colab Demo\n\n[Here](https://colab.research.google.com/drive/1V03JfVsuOamgncWMXoOOSRsQPzd35x5v?usp=sharing) we provide a **Colab** demo, which basically demonstrated the capbility of **style transfer** and **free-viewpoint protrait**. \n\n## Installation\n\n![version](https://img.shields.io/badge/pytorch-%3E%3D%201.7.1-blue) ![version](https://img.shields.io/badge/Ubuntu-%3E%3D16.04-blue) ![version](https://img.shields.io/badge/torchvision-%3E%3D0.8.2-blue)![version](https://img.shields.io/badge/cudatoolkit-%3D%3D10.2-blue)\n\n\nInstall environment:\n```bash\ngit clone https://github.com/apchenstu/sofgan.git --recursive\nconda install pytorch==1.7.1 torchvision==0.8.2 cudatoolkit=10.2 -c pytorch\npip install tqdm argparse scikit-image lmdb config-argparse dlib\n```\n\n## Training\nPlease see each subsection for training on different datasets. Available training datasets:\n\n* [FFHQ](https://github.com/NVlabs/stylegan)\n* [CelebA](https://github.com/switchablenorms/CelebAMask-HQ)\n* [Your own data](#your-own-data) (portrait images or segmaps)\n\nWe also provide our pre-process [ffhq and celeba segmaps](https://1drv.ms/u/s!Ard0t_p4QWIMhhQQz82NRbxRIjZ5?e=uYiBpg) (in our classes labels). You may also want to re-train the [SOF model](https://github.com/walnut-REE/sof/) base on your own multi-view segmaps.\n\n## Run\n\n```bash\nCUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 --master_port=9999 train.py \\\n    --num_worker 4  --resolution 1024\n   --name $exp_name\n   --iter 10000000\n   --batch 1 --mixing 0.9 \\\n   path/to/your/image/folders \\\n   --condition_path path/to/your/segmap/folders\n```\n\nIn our experiments, 4x **Nividia 2080Ti** GPU  would take around `20` days to reach `10000k` iterations.  Adjusting the image resolution and max iterations to suit your own dataset. Emperically, for datasets like [FFHQ](https://github.com/NVlabs/stylegan) and [CelebA](https://github.com/switchablenorms/CelebAMask-HQ)(resolution `1024x1024`) the network would converge after `1000k` iterations and achieve fancy results.\n\n***Notice***: training on none pair-wise data (image/segmap) is encouraged. Since it's one of the key features of our **SofGAN**.\n\n## Rendering\nWe provide a rendering script in `renderer.ipynb`, where you can restyle your own photos, videos and generate free-viewpoint portrait images while maintaining the geometry consistency.\nJust to download our [checkpoints](https://drive.google.com/file/d/17SW3MurX_78_CfT29DaBZuqnKEjkbzQv/view?usp=sharing) and unzip to the root folder.\n\n## UI Illustration\n   The Painter is included in `Painter`, you can pull down and drawing on-the-fly.\n   Before that, you need to install the enviroment with ```pip install -r ./Painter/requirements.txt```\n\n![UI](https://github.com/apchenstu/GIFs/blob/main/sofgan.gif)\n\n## IOS App\nYou could download and try the [Wand](https://apps.apple.com/cn/app/wand/id1574341319), an **IOS** App developed by [Deemos](https://www.deemos.com/).\n\n![two-dimensions](https://github.com/apchenstu/GIFs/blob/main/two-dimensions.gif)\n\n## Online Demo\nNew Folder\n\n## Relevant Works\n[**StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows (TOG 2021)**](https://arxiv.org/abs/2008.02401)\u003cbr\u003e\nRameen Abdal, Peihao Zhu, Niloy Mitra, Peter Wonka\n\n[**SEAN: Image Synthesis With Semantic Region-Adaptive Normalization (CVPR 2020)**](https://arxiv.org/abs/1911.12861)\u003cbr\u003e\nPeihao Zhu, Rameen Abdal, Yipeng Qin, Peter Wonka\n\n[**StyleRig: Rigging StyleGAN for 3D Control over Portrait Images (CVPR 2020)**](https://gvv.mpi-inf.mpg.de/projects/StyleRig/)\u003cbr\u003e\nA. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H.P. Seidel, P. Pérez, M. Zollhöfer, Ch. Theobalt\n\n[**StyleGAN2: Analyzing and Improving the Image Quality of {StyleGAN} (CVPR 2020)**](https://arxiv.org/abs/1912.04958)\u003cbr\u003e\nTero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila\n\n[**SPADE: Semantic Image Synthesis with Spatially-Adaptive Normalization (CVPR 2019)**](https://arxiv.org/abs/1903.07291)\u003cbr\u003e\nTaesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu\n\n## Citation\nIf you find our code or paper helps, please cite:\n```\n@article{sofgan,\n  title={Sofgan: A portrait image generator with dynamic styling},\n  author={Chen, Anpei and Liu, Ruiyang and Xie, Ling and Chen, Zhang and Su, Hao and Yu, Jingyi},\n  journal={ACM Transactions on Graphics (TOG)},\n  volume={41},\n  number={1},\n  pages={1--26},\n  year={2022},\n  publisher={ACM New York, NY}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapchenstu%2Fsofgan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapchenstu%2Fsofgan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapchenstu%2Fsofgan/lists"}