{"id":13594399,"url":"https://github.com/lazavgeridis/mddgan","last_synced_at":"2025-04-09T07:32:24.981Z","repository":{"id":80413930,"uuid":"486588139","full_name":"lazavgeridis/mddgan","owner":"lazavgeridis","description":"Implementation of my B.Sc. Thesis titled \"MddGAN : Multilinear Analysis of the GAN Latent Space\".","archived":false,"fork":false,"pushed_at":"2022-09-16T15:03:13.000Z","size":31231,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-02T16:52:06.830Z","etag":null,"topics":["decomposition","deep-learning","generative-adversarial-network","interpretability","latent-space","pytorch","unsupervised-learning"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lazavgeridis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-04-28T12:37:36.000Z","updated_at":"2024-04-22T03:18:12.000Z","dependencies_parsed_at":"2023-06-07T13:30:25.107Z","dependency_job_id":null,"html_url":"https://github.com/lazavgeridis/mddgan","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lazavgeridis%2Fmddgan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lazavgeridis%2Fmddgan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lazavgeridis%2Fmddgan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lazavgeridis%2Fmddgan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lazavgeridis","download_url":"https://codeload.github.com/lazavgeridis/mddgan/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223375324,"owners_count":17135351,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["decomposition","deep-learning","generative-adversarial-network","interpretability","latent-space","pytorch","unsupervised-learning"],"created_at":"2024-08-01T16:01:32.894Z","updated_at":"2025-04-09T07:32:24.975Z","avatar_url":"https://github.com/lazavgeridis.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"# Introduction\n\nThis repo contains the implementation of my B.sc. Thesis titled\n**\"MddGAN : Multilinear Analysis of the GAN Latent Space\"**. The thesis\ntext can be found [here](https://pergamos.lib.uoa.gr/uoa/dl/object/3059772).\n\nIn short, this thesis proposes an unsupervised method to discover a wide range\nof interpretable vector directions by analyzing the space of the generator's\nparameters, otherwise known as the _GAN latent space_. The extracted directions\ncan then be exploited in order to produce impressive visual edits, on par with\nthe current SOTA methods. Furthermore, the proposed method does not only reveal\nthe explanatory factors learnt by the generator, but it also attempts to arrange\nthem along the dimensions of the produced multilinear basis, according to the\nsemantic content they encode.\n\n\n# Sample Results\n\n**StyleGAN2 FFHQ**\n\n![stylegan2_ffhq](images/stylegan2ffhq_chart.jpg)\n\n\n**StyleGAN AnimeFaces**\n\n![stylegan_animeface](images/stylegananime_chart.jpg)\n\n**Note:** Every image on the above charts is generated by the corresponding GAN\nmodel, even the ones on the \"Initial Image\" column.\n\n# Usage\nTo replicate the exact environment used during the development of this repo,\nsimply run:\n```\npip install -r requirements.txt\n```\n\n## Discovering semantic concepts in the GAN latent space\n### Basic Execution\n```\npython discover_semantics.py [model_name] [method_name]\n```\nwhere `model_name` refers to the name of the GAN model you want to discover\nsemantics for and `method_name` refers to the method to use when analyzing\nthe latent space of the selected GAN model. The list of valid `model_name`'s\nto use can be found at [mddgan/models/model_zoo.py](models/model_zoo.py),\nwhile `method_name` can be either one of `mddgan`, `sefa` or `both`.\n\nFor instance, some sample executions are:\n\n```\n# Analyze StyleGAN2 FFHQ model\npython discover_semantics.py stylegan2_ffhq1024 [method_name]\n\n# Analyze StyleGAN LSUN Bedroom model\npython discover_semantics.py stylegan_bedroom256 [method_name]\n\n# Analyze ProGAN CelebaHQ model\npython discover_semantics.py pggan_celebahq1024 [method_name]\n```\n\n### Analyzing Specific Layer Ranges\nNote that in the case of **StyleGAN/StyleGAN2** models, e.g `stylegan2_ffhq1024` and\n`stylegan_bedroom256` from above, the default behaviour of the program is to analyze\n_all layers_ of the selected model, which will discover directions\nthat impact multiple variation factors at once. However, this behaviour can be modified by\nusing the `layer_range` option. For example, to extract semantics that effect the overall\ngeometric properties of the image, you probably want to target the initial layers:\n\n```\npython discover_semantics.py stylegan2_car512 [method_name] --layer_range 0-3\n```\nIn general, the argument to `layer_range` indicates the layer indices of\nthe model to analyze and is of the form: $idx_{1} - idx_{2}$, where\n$idx \\in [0, L - 1]$. Here, $L$ is denoting the total number of layers in $G$,\nand for ProGAN/StyleGAN/StyleGAN2 models that synthesize $1024 \\times 1024$\nresolution images, $L = 18$.\n\n### Attempting to Group the Discovered Semantics\nOther than simply discovering surprising directions, MddGAN can additionally\nseparate them into groups. In essence, by tensorizing the produced multilinear\nbasis $\\mathcal{\\mathbf{B}}$, one can attempt to gather all directions encoding\nthe same variability factor by slicing tensor $\\mathcal{\\mathbf{B}}$ on the\nappropriate mode (dimension). To achieve this, we can use the `num_modes` option.\nThe argument to `num_modes` sets the estimated number of variation factors the\nGenerator has learnt to model. For instance, assuming 3 modes of variation:\n\n```\npython discover_semantics.py stylegan2_car512 mddgan --num_modes 3\n```\n\n### Reducing the Number of Discovered Directions\nFinally, to discover a reduced number of directions, the `num_components`\noption can be used. For instance, to discover 200 directions instead of\nthe default 512, run:\n\n```\npython discover_semantics.py stylegan2_car512 mddgan --num_components 200\n```\n\n### Selecting the Editing Magnitude\ntalk about the magnitude of the edit `start_distance` and `end_distance`.\n\n## Evaluation\n### FID Scores\nIn the directory [mddgan/fid_files](fid_files), we provide some pre-computed\nFID scores, calculated by editing synthesized images using directions that\nimpact distinctive facial attributes (pose, gender, age, smile, eyeglasses).\n\nFor example, to plot the FID scores for the pose discovered attribute and for\nthe StyleGAN CelebaHQ model, comparing MddGAN to InterFaceGAN, run:\n\n```\npython plot_fid.py stylegan_celebahq1024 interfacegan pose\n```\n\nThe program will locate the corresponding file, in this case the file is \n[mddgan/fid_files/stylegan_celebahq1024_interfacegan_pose.txt](fid_files/stylegan_celebahq1024_interfacegan_pose.txt),\ngather the FID scores and produce the corresponding plot.\n\n### Correlation Between Discovered Attributes \nBased on prior work, we know that some of the semantics discovered by\nunsupervised methods (and not only) might be coupled with each other.\nFor instance, the _eyeglasses_ direction is frequently correlated with\n_gender_ and _age_, because in the respective training datasets (e.g CelebaHQ,\nFFHQ), people that wear eyeglasses are usually older males.\n\nTo measure the correlation between 2 discovered facial attributes, we use\ncosine similarity.\n\n```\npython cosine_similarity.py [model_name] [method_name]\n```\n\nThe above will compute the correlation between the discovered attributes and\nproduce a correlation matrix.\n\n\n## Google Colab Notebooks\nThe code of this repo requires a machine with an Nvidia GPU to run (with the exception of `cosine_similarity.py`\nand `plot_fid.py`) .\nHowever, if you don't have one available, you can still run the following Google Colab notebooks to\nrecreate the figures present in the thesis:\n* Figure 4.2 : `layer_ranges_chart.ipynb` -\u003e [Colab link](https://colab.research.google.com/github/lazavgeridis/mddgan/blob/main/notebooks/layer_ranges_chart.ipynb)\n* Figures 4.3-4.4 and 4.6-4.11 : `interpolation_across_each_mode.ipynb` -\u003e [Colab link](https://colab.research.google.com/github/lazavgeridis/mddgan/blob/main/notebooks/interpolation_across_each_mode.ipynb)\n* Figure 5.1-5.2 and 5.4 : `mddgan_comparisons_v1.ipynb` -\u003e [Colab link](https://colab.research.google.com/github/lazavgeridis/mddgan/blob/main/notebooks/mddgan_comparisons_v1.ipynb)\n* Figure 5.3 : `diversity_of_discovered_semantics.ipynb` -\u003e [Colab link](https://colab.research.google.com/github/lazavgeridis/mddgan/blob/main/notebooks/diversity_of_discovered_semantics.ipynb)\n\n\n\n# Acknowledgements\nThis project could not exist if it weren't for the excellent implementations\nlisted below:\n* The [SeFa](https://github.com/genforce/sefa) project, from which a substantial\npart of the code of this project is inspired. The [mddgan/models](models)\ndirectory used here is borrowed from SeFa.\n* The [InterFaceGAN](https://github.com/genforce/interfacegan) project, from\nwhich we borrow the ProGAN and StyleGAN directions used in our comparisons.\n* The [GANLatentDiscovery](https://github.com/anvoynov/GANLatentDiscovery)\nproject, from which we got the inspiration for the core visualization operation\nimplemented here.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flazavgeridis%2Fmddgan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flazavgeridis%2Fmddgan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flazavgeridis%2Fmddgan/lists"}