{"id":13561615,"url":"https://github.com/williamyang1991/StyleGANEX","last_synced_at":"2025-04-03T17:31:21.719Z","repository":{"id":142916581,"uuid":"608939722","full_name":"williamyang1991/StyleGANEX","owner":"williamyang1991","description":"[ICCV 2023] StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces","archived":false,"fork":false,"pushed_at":"2023-09-19T11:36:14.000Z","size":18997,"stargazers_count":520,"open_issues_count":9,"forks_count":40,"subscribers_count":24,"default_branch":"main","last_synced_at":"2025-03-29T11:03:39.208Z","etag":null,"topics":["face","face-editing","face-manipulation","stylegan2"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/williamyang1991.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-03T03:05:03.000Z","updated_at":"2025-03-23T11:34:47.000Z","dependencies_parsed_at":"2024-08-01T13:17:23.107Z","dependency_job_id":"48fcace5-0698-4ddc-a623-56d9814d10d5","html_url":"https://github.com/williamyang1991/StyleGANEX","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/williamyang1991%2FStyleGANEX","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/williamyang1991%2FStyleGANEX/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/williamyang1991%2FStyleGANEX/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/williamyang1991%2FStyleGANEX/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/williamyang1991","download_url":"https://codeload.github.com/williamyang1991/StyleGANEX/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247046912,"owners_count":20874737,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["face","face-editing","face-manipulation","stylegan2"],"created_at":"2024-08-01T13:00:59.206Z","updated_at":"2025-04-03T17:31:16.710Z","avatar_url":"https://github.com/williamyang1991.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"# StyleGANEX - Official PyTorch Implementation\n\nhttps://user-images.githubusercontent.com/18130694/224256980-03fb15e7-9858-4300-9d35-7604d03c69f9.mp4\n\nThis repository provides the official PyTorch implementation for the following paper:\n\n**StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces**\u003cbr\u003e\n[Shuai Yang](https://williamyang1991.github.io/), [Liming Jiang](https://liming-jiang.com/), [Ziwei Liu](https://liuziwei7.github.io/) and [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/)\u003cbr\u003e\nIn ICCV 2023. \u003cbr\u003e\n[**Project Page**](https://www.mmlab-ntu.com/project/styleganex/) | [**Paper**](https://arxiv.org/abs/2303.06146) | [**Supplementary Video**](https://youtu.be/8oK0TXQmxg8) \u003cbr\u003e\n\n\n\u003ca href=\"http://colab.research.google.com/github/williamyang1991/StyleGANEX/blob/master/inference_playground.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e\n[![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/PKUWilliamYang/StyleGANEX)\n![visitors](https://visitor-badge.laobi.icu/badge?page_id=williamyang1991/styleganex)\n\u003c!--![visitors](https://visitor-badge.glitch.me/badge?page_id=williamyang1991/styleganex)--\u003e\n\n\u003e **Abstract:** *Recent advances in face manipulation using StyleGAN have produced impressive results. However, StyleGAN is inherently limited to cropped aligned faces at a fixed image resolution it is pre-trained on.  In this paper, we propose a simple and effective solution to this limitation by using dilated convolutions to rescale the receptive fields of shallow layers in StyleGAN, without altering any model parameters.  This allows fixed-size small features at shallow layers to be extended into larger ones that can accommodate variable resolutions, making them more robust in characterizing unaligned faces. To enable real face inversion and manipulation, we introduce a corresponding encoder that provides the first-layer feature of the extended StyleGAN in addition to the latent style code. We validate the effectiveness of our method using unaligned face inputs of various resolutions in a diverse set of face manipulation tasks, including facial attribute editing, super-resolution, sketch/mask-to-face translation, and face toonification.*\n\n**Features**:\u003cbr\u003e \n- **Support for Unaligned Faces**: StyleGANEX can manipulate normal field-of-view face images and videos.\n- **Compatibility**: StyleGANEX can directly load pre-trained StyleGAN parameters without retraining.\n- **Flexible Manipulation**: StyleGANEX retains the style representation and editing ability of StyleGAN.\n\n![overview](https://user-images.githubusercontent.com/18130694/224257328-b6d9bac1-d467-468f-9dba-c89dfed8b931.jpg)\n\n## Updates\n\n- [07/2023] Training code is released.\n- [07/2023] The paper is accepted to ICCV 2023 😁!\n- [03/2023] Integrated to 🤗 [Hugging Face](https://huggingface.co/spaces/PKUWilliamYang/StyleGANEX). Enjoy the web demo!\n- [03/2023] Inference code is released.\n- [03/2023] This website is created.\n\n\n## Installation\n\n**Clone this repo:**\n```bash\ngit clone https://github.com/williamyang1991/StyleGANEX.git\ncd StyleGANEX\n```\n**Dependencies:**\n\nWe have tested on:\n- CUDA 10.1\n- PyTorch 1.7.1\n- Pillow 8.3.1; Matplotlib 3.4.2; opencv-python 4.5.3; tqdm 4.61.2; Ninja 1.10.2; dlib 19.24.0; gradio 3.4\n\n\u003cbr/\u003e\n\n## (1) Inference\n\n### Inference Notebook \n\u003ca href=\"http://colab.research.google.com/github/williamyang1991/StyleGANEX/blob/master/inference_playground.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e \n\nTo help users get started, we provide a Jupyter notebook found in `./inference_playground.ipynb` that allows one to visualize the performance of StyleGANEX.\nThe notebook will download the necessary pretrained models and run inference on the images found in `./data/`.\n\n### Gradio demo\nWe also provide a UI for testing StyleGANEX that is built with gradio. Running the following command in a terminal will launch the demo:\n```\npython app_gradio.py\n```\nThis demo is also hosted on [Hugging Face](https://huggingface.co/spaces/PKUWilliamYang/StyleGANEX).\n\n### Pre-trained Models\n\nPre-trained models can be downloaded from [Google Drive](https://drive.google.com/drive/folders/1zGssOxjdklMd_5kdBKV9VkENnS5EXZlx?usp=share_link),\n[Baidu Cloud](https://pan.baidu.com/s/15SZrkvgduUaMI33LJGnEJg?pwd=luck ) (access code: luck)\nor [Hugging Face](https://huggingface.co/PKUWilliamYang/StyleGANEX/tree/main/pretrained_models):\n\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003cth\u003eTask\u003c/th\u003e\u003cth\u003eModel\u003c/th\u003e\u003cth\u003eDescription\u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003eInversion\u003c/td\u003e\u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/157twOAYihuy_b6l_XrmP7QUOMhkNC2Ii/view?usp=share_link\"\u003estyleganex_inversion.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained model for StyleGANEX inversion\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd rowspan=\"4\"\u003eImage translation\u003c/td\u003e\u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/1ewbdY_0fRZS6GIboFcvx6QDBbqHXprvR/view?usp=share_link\"\u003estyleganex_sr32.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained model specially for 32x face super resolution\u003c/td\u003e\n    \u003c/tr\u003e  \n    \u003ctr\u003e\n        \u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/1XQ4vp8DB2dSrvQVj3xifSl4sUGMxr4zK/view?usp=share_link\"\u003estyleganex_sr.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained model for 4x-48x face super resolution\u003c/td\u003e\n    \u003c/tr\u003e   \n    \u003ctr\u003e\n        \u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/1L3iRp3UE-_Or0NqzUtqZPLQ9hQ_9yax5/view?usp=share_link\"\u003estyleganex_sketch2face.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained model for skech-to-face translation\u003c/td\u003e\n    \u003c/tr\u003e  \n    \u003ctr\u003e\n        \u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/1rHC63z1tUX63-56RUc0thhHSVAQr8Cf2/view?usp=share_link\"\u003estyleganex_mask2face.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained model for parsing map-to-face translation\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd rowspan=\"5\"\u003eVideo editing\u003c/td\u003e\u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/164eD7pafO74xiCCFLzhb56ofOBwIUnpL/view?usp=share_link\"\u003estyleganex_edit_hair.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained model for hair color editing on videos\u003c/td\u003e\n    \u003c/tr\u003e  \n    \u003ctr\u003e\n        \u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/1tH-vlpn5THyD-HoOQrYGuUENH49daAFX/view?usp=share_link\"\u003estyleganex_edit_age.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained model for age editing on videos\u003c/td\u003e\n    \u003c/tr\u003e  \n    \u003ctr\u003e\n        \u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/16Kth67C1AX3SjZS3030DChB_eHpqGOsn/view?usp=share_link\"\u003estyleganex_toonify_cartoon.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained Cartoon model for video face toonification\u003c/td\u003e\n    \u003c/tr\u003e  \n    \u003ctr\u003e\n        \u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/1OkCw4mrrCvTnEfPOeUxjv7yFpzzQ5trf/view?usp=share_link\"\u003estyleganex_toonify_arcane.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained Arcane model for video face toonification\u003c/td\u003e\n    \u003c/tr\u003e   \n    \u003ctr\u003e\n        \u003ctd\u003e\u003ca href=\"https://drive.google.com/file/d/1_XZjvj-rQvT2q3hiqPZ8gIClMo3ZGVH-/view?usp=share_link\"\u003estyleganex_toonify_pixar.pt\u003c/a\u003e\u003c/td\u003e\u003ctd\u003epre-trained Pixar model for video face toonification\u003c/td\u003e\n    \u003c/tr\u003e   \n    \u003ctr\u003e\n        \u003cth colspan=\"2\"\u003eSupporting model\u003c/th\u003e\u003cth\u003e \u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd colspan=\"2\"\u003e\u003ca href=\"https://drive.google.com/file/d/1jY0mTjVB8njDh6e0LP_2UxuRK3MnjoIR/view\"\u003efaceparsing.pth\u003c/a\u003e\u003c/td\u003e\u003ctd\u003eBiSeNet for face parsing from \u003ca href=\"https://github.com/zllrunning/face-parsing.PyTorch\"\u003eface-parsing.PyTorch\u003c/a\u003e\u003c/td\u003e\n    \u003c/tr\u003e      \n\u003c/table\u003e\n\nThe downloaded models are suggested to be put into `./pretrained_models/`\n\n### StyleGANEX Inversion\n\nWe can embed a face image into the latent space of StyleGANEX to obtain its w+ latent code and the first-layer feature f with `inversion.py`.\n\n```python\npython inversion.py --ckpt STYLEGANEX_MODEL_PATH --data_path FACE_IMAGE_PATH\n```\nThe results are saved in the folder `./output/`.\nThe results contain a reconstructed image `FILE_NAME_inversion.jpg` and a `FILE_NAME_inversion.pt` file.\nYou can obtain w+ latent code and the first-layer feature f by \n```python \nlatents = torch.load('./output/FILE_NAME_inversion.pt')\nwplus_hat = latents['wplus'].to(device) # w+\nf_hat = [latents['f'][0].to(device)]    # f\n```\nThe `./inference_playground.ipynb` provides some face editing examples based on `wplus_hat` and `f_hat`.\n\n### Image Translation\n\n`image_translation.py` supports face super-resolution, sketch-to-face translation and parsing map-to-face translation.\n\n```python\npython image_translation.py --ckpt STYLEGANEX_MODEL_PATH --data_path FACE_INPUT_PATH\n```\nThe results are saved in the folder `./output/`.\n\nAdditional notes to consider:\n- `--parsing_model_ckpt` (default: `pretrained_models/faceparsing.pth`): path to the pre-trained parsing model\n- `--resize_factor` (default: 32): super resolution resize factor\n    - For [styleganex_sr.pt](), should be in [4, 48]\n    - For [styleganex_sr32.pt](), should be 32\n- `--number` (default: 4): output number of multi-modal translation (for sketch/mask-to-face translation task)\n- `--use_raw_data` (default: False): \n    - if not specified, apply possible pre-processing to the input data\n        - For [styleganex_sr/sr32.pt](), the input face image, e.g., `./data/ILip77SbmOE.png` will be downsampled based on `--resize_factor`. The downsampled image will be also saved in `./output/`.\n        - For [styleganex_sketch2face.pt](), no pre-processing will be applied.\n        - For [styleganex_mask2face.pt](), the input face image, e.g., `./data/ILip77SbmOE.png` will be transformed into a parsing map. The parsing map and its visualization version will be also saved in `./output/`.\n    - if specified, directly load input data without pre-processing\n        - For [styleganex_sr/sr32.pt](), the input should be downsampled face images, e.g., `./data/ILip77SbmOE_45x45.png`\n        - For [styleganex_sketch2face.pt](), the input should be a one-channel sketch image e.g., `./data/234_sketch.jpg`\n        - For [styleganex_mask2face.pt](), the input should be a one-channel parsing map e.g., `./data/ILip77SbmOE_mask.png`\n\n\n### Video Editing\n\n`video_editing.py` supports video facial attribute editing and video face toonification.\n\n```python\npython video_editing.py --ckpt STYLEGANEX_MODEL_PATH --data_path FACE_INPUT_PATH\n```\nThe results are saved in the folder `./output/`.\n\nAdditional notes to consider:\n- `--data_path`: the input can be either an image or a video.\n- `--scale_factor`: for attribute editing task ([styleganex_edit_hair/age]()), control the editing degree.\n\n\u003cbr/\u003e\n\n## (2) Training \n\n### Preparing your Data\n- As with pSp, we provide support for numerous datasets and experiments (encoding, translation, etc.).\n    - Refer to `configs/paths_config.py` to define the necessary data paths and model paths for training and evaluation. \n    - Refer to `configs/transforms_config.py` for the transforms defined for each dataset/experiment. \n    - Finally, refer to `configs/data_configs.py` for the source/target data paths for the train and test sets\n      as well as the transforms.\n- If you wish to experiment with your own dataset, you can simply make the necessary adjustments in \n    - `data_configs.py` to define your data paths.\n    - `transforms_configs.py` to define your own data transforms.\n    \nAs an example, assume we wish to run encoding using ffhq (`dataset_type=ffhq_encode`). \nWe first go to `configs/paths_config.py` and define:\n``` \ndataset_paths = {\n    'ffhq': '/path/to/ffhq/realign320x320'\n    'ffhq_test': '/path/to/ffhq/realign320x320_test'\n}\n```\nThe transforms for the experiment are defined in the class `EncodeTransforms` in `configs/transforms_config.py`.   \nFinally, in `configs/data_configs.py`, we define:\n``` \nDATASETS = {\n   'ffhq_encode': {\n        'transforms': transforms_config.EncodeTransforms,\n        'train_source_root': dataset_paths['ffhq'],\n        'train_target_root': dataset_paths['ffhq'],\n        'test_source_root': dataset_paths['ffhq_test'],\n        'test_target_root': dataset_paths['ffhq_test'],\n    },\n}\n``` \nWhen defining our datasets, we will take the values in the above dictionary.\n\nThe 1280x1280 ffhq images can be obtain by the modified script of official ffhq:\n- Download the in-the-wild images with `python script/download_ffhq1280.py --wilds`\n- Reproduce the aligned 1280×1280 images wiht `python script/download_ffhq1280.py --align`\n- 320x320 ffhq images can be obtained by setting `output_size=320, transform_size=1280` in Line 272 of download_ffhq1280.py\n\n### Downloading supporting models\nPlease download the pre-trained models to support the training of StyleGANEX\n| Path | Description\n| :--- | :----------\n|[original_stylegan](https://drive.google.com/file/d/1EM87UquaoQmk17Q8d5kYIAHqu0dkYqdT/view)| StyleGAN trained with the FFHQ dataset\n|[toonify_model](https://drive.google.com/drive/folders/1GZQ6Gs5AzJq9lUL-ldIQexi0JYPKNy8b) | StyleGAN finetuned on cartoon dataset for image toonification ([cartoon](https://drive.google.com/file/d/1w7BJDiSw5_ybelv7jL_Jeu1T-oWEWUmH/view?usp=drive_link), [pixar](https://drive.google.com/file/d/1phftRYbsp34pL5Yqapz3c_Wv0G4L0vy2/view?usp=drive_link), [arcane](https://drive.google.com/file/d/1HysdpShIAbHtf6T9R-hVULjUShgA5AL1/view?usp=drive_link))\n|[original_psp_encoder](https://drive.google.com/file/d/1bMTNWkh5LArlaWSc_wa8VKyq2V42T2z0/view?usp=sharing)  | pSp trained with the FFHQ dataset for StyleGAN inversion.\n|[pretrained_encoder](https://drive.google.com/file/d/1RkQbKZUoTSBKRPwAcpXCPmLExIIs84eO/view?usp=drive_link)  | StyleGANEX encoder pretrained with the synthetic data for StyleGAN inversion.\n|[styleganex_encoder](https://drive.google.com/file/d/157twOAYihuy_b6l_XrmP7QUOMhkNC2Ii/view?usp=share_link)  | StyleGANEX encoder trained with the FFHQ dataset for StyleGANEX inversion.\n|[editing_vector](https://drive.google.com/drive/folders/1zGssOxjdklMd_5kdBKV9VkENnS5EXZlx?usp=share_link)  | Editing vectors for editing face attributes ([age](https://drive.google.com/file/d/1j2373q_xETMJoJGaLQriLrzzzpTQdgWD/view?usp=drive_link), [hair color](https://drive.google.com/file/d/1qCelrIaF4GieKqjwvi5siQbYwKTzqK4M/view?usp=drive_link))\n|[augmentation_vector](https://drive.google.com/file/d/1cOspde7cWY7AYsTQ0X4_DXRfv_hdZ2Y0/view?usp=drive_link)  | Editing vectors for data augmentation\n\nThe main training script can be found in `scripts/train.py`.   \nIntermediate training results are saved to `opts.exp_dir`. This includes checkpoints, train outputs, and test outputs.  \n\n### Training styleganex\n\nNote: Our default code is a CPU-compatible version. You can switch to a more efficient version by using cpp extention. \nTo do so, please change `models.stylegan2.op` to `models.stylegan2.op_old`\nhttps://github.com/williamyang1991/StyleGANEX/blob/73b580cc7eb757e36701c094456e9ee02078d03e/models/stylegan2/model.py#L8\n\n#### Training the styleganex encoder\nFirst pretrain encoder on synthetic 1024x1024 images. You can download our pretrained encoder [here](https://drive.google.com/file/d/1RkQbKZUoTSBKRPwAcpXCPmLExIIs84eO/view?usp=drive_link)\n```\npython scripts/pretrain.py \\\n--exp_dir=/path/to/experiment \\\n--ckpt=/path/to/original_psp_encoder \\\n--max_steps=2000\n```\nThen finetune encoder on real 1280x1280 ffhq images based on the [pretrained encoder](https://drive.google.com/file/d/1RkQbKZUoTSBKRPwAcpXCPmLExIIs84eO/view?usp=drive_link)\n```\npython scripts/train.py \\\n--dataset_type=ffhq_encode \\\n--exp_dir=/path/to/experiment \\\n--checkpoint_path=/path/to/pretrained_encoder \\\n--max_steps=100000 \\\n--workers=8 \\\n--batch_size=8 \\\n--val_interval=2500 \\\n--save_interval=50000 \\\n--start_from_latent_avg \\\n--id_lambda=0.1 \\\n--w_norm_lambda=0.001 \\\n--affine_augment \\\n--random_crop \\\n--crop_face\n```\n#### Sketch to Face\n```\npython scripts/train.py \\\n--dataset_type=ffhq_sketch_to_face \\\n--exp_dir=/path/to/experiment \\\n--stylegan_weights=/path/to/original_stylegan \\\n--max_steps=100000 \\\n--workers=8 \\\n--batch_size=8 \\\n--val_interval=2500 \\\n--save_interval=10000 \\\n--start_from_latent_avg \\\n--w_norm_lambda=0.005 \\\n--affine_augment \\\n--random_crop \\\n--crop_face \\\n--use_skip \\\n--skip_max_layer=1 \\\n--label_nc=1 \\\n--input_nc=1 \\\n--use_latent_mask\n```\n#### Segmentation Map to Face\n```\npython scripts/train.py \\\n--dataset_type=ffhq_seg_to_face \\\n--exp_dir=/path/to/experiment \\\n--stylegan_weights=/path/to/original_stylegan \\\n--max_steps=100000 \\\n--workers=8 \\\n--batch_size=8 \\\n--val_interval=2500 \\\n--save_interval=10000 \\\n--start_from_latent_avg \\\n--w_norm_lambda=0.005 \\\n--affine_augment \\\n--random_crop \\\n--crop_face \\\n--use_skip \\\n--skip_max_layer=2 \\\n--label_nc=19 \\\n--input_nc=19 \\\n--use_latent_mask \n```\n#### Super Resolution\n``` \npython scripts/train.py \\\n--dataset_type=ffhq_super_resolution \\\n--exp_dir=/path/to/experiment \\\n--checkpoint_path=/path/to/styleganex_encoder \\\n--max_steps=100000 \\\n--workers=4 \\\n--batch_size=4 \\\n--val_interval=2500 \\\n--save_interval=10000 \\\n--start_from_latent_avg \\\n--adv_lambda=0.1 \\\n--affine_augment \\\n--random_crop \\\n--crop_face \\\n--use_skip \\\n--skip_max_layer=4 \\\n--resize_factors=8\n```\nFor one model supporting multiple resize factors, set `--skip_max_layer=2` and `--resize_factors=1,2,4,8,16`\n#### Video Editing\n```\npython scripts/train.py \\\n--dataset_type=ffhq_edit \\\n--exp_dir=/path/to/experiment \\\n--checkpoint_path=/path/to/styleganex_encoder \\\n--max_steps=100000 \\\n--workers=2 \\\n--batch_size=2 \\\n--val_interval=2500 \\\n--save_interval=10000 \\\n--start_from_latent_avg \\\n--adv_lambda=0.1 \\\n--tmp_lambda=30 \\\n--affine_augment \\\n--crop_face \\\n--use_skip \\\n--skip_max_layer=7 \\\n--editing_w_path=/path/to/editing_vector \\\n--direction_path=/path/to/augmentation_vector \\\n--use_att=1 \\\n--generate_training_data\n```\n#### Video Toonification\n```\npython scripts/train.py \\\n--dataset_type=toonify \\\n--exp_dir=/path/to/experiment \\\n--checkpoint_path=/path/to/styleganex_encoder \\\n--max_steps=55000 \\\n--workers=2 \\\n--batch_size=2 \\\n--val_interval=2500 \\\n--save_interval=10000 \\\n--start_from_latent_avg \\\n--adv_lambda=0.1 \\\n--tmp_lambda=30 \\\n--affine_augment \\\n--crop_face \\\n--use_skip \\\n--skip_max_layer=7 \\\n--toonify_weights=/path/to/toonify_model\n```\n\n### Additional Notes\n- See `options/train_options.py` for all training-specific flags. \n- If you wish to generate images from segmentation maps, please specify `--label_nc=N`  and `--input_nc=N` where `N` \nis the number of semantic categories. \n- Similarly, for generating images from sketches, please specify `--label_nc=1` and `--input_nc=1`.\n- Specifying `--label_nc=0` (the default value), will directly use the RGB colors as input.\n\n## (3) Results\n\nOverview of StyleGANEX inversion and facial attribute/style editing on unaligned faces:\n\n![result](https://user-images.githubusercontent.com/18130694/224259844-c9b37f4f-c786-48cd-a92f-121606b14b36.jpg)\n\nVideo facial attribute editing:\n\nhttps://user-images.githubusercontent.com/18130694/224287063-7465a301-4d11-4322-819a-59d548308ce1.mp4\n\n\u003cbr/\u003e\n\nVideo face toonification:\n\nhttps://user-images.githubusercontent.com/18130694/224287136-7e5ce82d-664f-4a23-8ed3-e7005efb3b24.mp4\n\n## Citation\n\nIf you find this work useful for your research, please consider citing our paper:\n\n```bibtex\n@inproceedings{yang2023styleganex,\n title = {StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces},\n author = {Yang, Shuai and Jiang, Liming and Liu, Ziwei and and Loy, Chen Change},\n booktitle = {ICCV},\n year = {2023},\n}\n```\n\n## Acknowledgments\n\nThe code is mainly developed based on [stylegan2-pytorch](https://github.com/rosinality/stylegan2-pytorch), [pixel2style2pixel](https://github.com/eladrich/pixel2style2pixel) and [VToonify](https://github.com/williamyang1991/VToonify).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwilliamyang1991%2FStyleGANEX","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwilliamyang1991%2FStyleGANEX","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwilliamyang1991%2FStyleGANEX/lists"}