{"id":20373178,"url":"https://github.com/st235/hse.deepgenerativenetworks","last_synced_at":"2025-06-15T13:34:10.210Z","repository":{"id":226798357,"uuid":"769533322","full_name":"st235/HSE.DeepGenerativeNetworks","owner":"st235","description":"Pix2Pix and CycleGAN training code","archived":false,"fork":false,"pushed_at":"2024-03-10T22:35:08.000Z","size":38726,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-15T06:49:40.048Z","etag":null,"topics":["cyclegan","deep-learning","gan","pix2pix"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/st235.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-09T10:53:56.000Z","updated_at":"2024-03-10T15:29:07.000Z","dependencies_parsed_at":"2025-01-15T05:44:25.294Z","dependency_job_id":"bc09d2e3-5f4b-4595-89a4-c93f59f4e3aa","html_url":"https://github.com/st235/HSE.DeepGenerativeNetworks","commit_stats":null,"previous_names":["st235/hse.deepgenerativenetworks"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/st235%2FHSE.DeepGenerativeNetworks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/st235%2FHSE.DeepGenerativeNetworks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/st235%2FHSE.DeepGenerativeNetworks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/st235%2FHSE.DeepGenerativeNetworks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/st235","download_url":"https://codeload.github.com/st235/HSE.DeepGenerativeNetworks/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241921836,"owners_count":20042763,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cyclegan","deep-learning","gan","pix2pix"],"created_at":"2024-11-15T01:16:59.847Z","updated_at":"2025-03-04T20:44:07.070Z","avatar_url":"https://github.com/st235.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pix2Pix and CycleGAN\r\n\r\nA lightweight implementation of Pix2Pix and CycleGAN training and inference pipeline.\r\n\r\n## Implementation\r\n\r\nAll architectures are implemented for _PyTorch framework_.\r\n\r\n### Dataset\r\n\r\n[The dataset](http://efrosgans.eecs.berkeley.edu/pix2pix/datasets/facades.tar.gz) consists of 256x512 images. \r\nEvery image is a side by side concatenation of buildings facades segmentation map and buildings facades photos.\r\n\r\nSee the example below:\r\n\r\n![Dataset Image Example](./images/dataset_example.jpg)\r\n\r\nThe code reads the image (using OpenCV) and crops it in halfs. \r\nThe implementation is available [under `dataset` folder](./dataset/facades_dataset.py).\r\n\r\n### Pix2Pix\r\n\r\nThe Pix2Pix GAN architecture consists of a generator and a discriminator models.\r\nBoth generator and discriminator use modules of the form Convolution-\u003eBatchNorm-\u003eReLu.\r\n\r\n![Pix2Pix Architecture](./images/pix2pix_architecture.jpg)\r\n\r\n- A _U-Net model_ architecture is used for the **generator**, instead of the common encoder-decoder architecture.\r\nThe U-Net model is quite similar to encoder-decoder as it also involves downsampling to a bottleneck and upsampling again to an output image.\r\nThe difference comes from skip-connections - links between layers of the same size in the encoder and the decoder allowing to be circumvent the bottleneck.\r\n- Discriminator is implemented as PatchGAN architecture. This is a deep convolutional neural network designed to classify patches of an input image as real or fake,\r\nrather than the entire image.\r\n\r\nThe diagram below shows comparison of `encoder-decoder` to `u-net` used in the generator of _Pix2Pix_.\r\n\r\n![Encoder-Decoder vs. U-Net](./images/encdec-vs-unet.png)\r\n\r\n#### Training\r\n\r\nTraining script is available in [`pix2pix.ipynb`](./pix2pix.ipynb)\r\n\r\n#### Results\r\n\r\n| Example 1                                | Example 2                                 | Example 3                            |\r\n|--------------------------------------------|----------------------------------------------|------------------------------------|\r\n| ![Example 1](./images/labels_002.jpg) | ![Example 2](./images/labels_004.jpg) | ![Example 3](./images/labels_009.jpg) |\r\n| ![Example 1](./images/pix2pix_002.jpg) | ![Example 2](./images/pix2pix_004.jpg) | ![Example 3](./images/pix2pix_009.jpg) |\r\n\r\n### CycleGAN\r\n\r\nUnlike other GANs, CycleGAN comes really handy when you don't have a pairing images set.\r\nFor example, when converting _horses_ to _zebras_ you usually don't have zebra images in the same surrounding and/or in similar poses. \r\n\r\nThe typical CycleGAN architecture includes 2 generators and 2 discriminators that work \r\nin sync to map images from the source domain to the target domain and vice versa using the same model.\r\nTypical CycleGAN architecture looks similar to the diagram below:\r\n\r\n![CycleGAN Architecture](./images/cyclegan_architecture.webp)\r\n\r\n- A **generator** is an autoencoder that takes an input image, extracts features from it, and generates another image. \r\nThe generator network consists of three main stages: convolutional block, residual block, and transposed convolutional block.\r\nThe encoder output passes through the transformer stage which mainly consists of 6 to 9 residual blocks. \r\n- A **discriminator** is a typical CNN that includes multiple convolutional layers. This network takes an input \r\nimage and classifies it as real or fake. The CycleGAN discriminator is different from that used in the regular GAN: \r\nit maps from the _256x256_ image to an _NxN_ array of outputs X. In that array, each element signifies whether a patch in the image is real or fake. \r\n\r\n\r\n#### Training\r\n\r\nTraining script is available at [`cyclegan.ipynb`](./cyclegan.ipynb)\r\n\r\n#### Results\r\n\r\n| Example 1                                | Example 2                                 | Example 3                            |\r\n|--------------------------------------------|----------------------------------------------|------------------------------------|\r\n| ![Example 1](./images/labels_001.jpg) | ![Example 2](./images/labels_007.jpg) | ![Example 3](./images/labels_008.jpg) |\r\n| ![Example 1](./images/cyclegan_001.jpg) | ![Example 2](./images/cyclegan_007.jpg) | ![Example 3](./images/cyclegan_008.jpg) |\r\n\r\n### Comparison\r\n\r\nThe detailed comparison available at [`comparison.ipynb`](./comparison.ipynb). The notebook offers \r\nside-by-side images comparison obtained from _Pix2Pix_ and _CycleGAN_, including a few custom segmentation maps.\r\n\r\n#### Custom segmentation maps\r\n\r\nThe custom segmentation maps set is rather small and was created for fun. \r\nThe set includes 4 images: small house, big building, theatre, and a shop.\r\n\r\nThe results are given in the next ordering: \r\n\r\n1. custom facades segmentation, \r\n2. pix2pix generations, \r\n3. cyclegan generations.\r\n\r\n\r\n| Small house                                | Big building                                 | Theatre                            | Shop                         |\r\n|--------------------------------------------|----------------------------------------------|------------------------------------|------------------------------|\r\n| ![Small house](./examples/small_house.png) | ![Big building](./examples/big_building.png) | ![Theatre](./examples/theatre.png) | ![Shop](./examples/shop.png) |\r\n| ![Small house](./images/pix2pix_small_house.jpg) | ![Big building](./images/pix2pix_big_building.jpg) | ![Theatre](./images/pix2pix_theatre.jpg) | ![Shop](./images/pix2pix_shop.jpg) |\r\n| ![Small house](./images/cyclegan_small_house.jpg) | ![Big building](./images/cyclegan_big_building.jpg) | ![Theatre](./images/cyclegan_theatre.jpg) | ![Shop](./images/cyclegan_shop.jpg) |\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fst235%2Fhse.deepgenerativenetworks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fst235%2Fhse.deepgenerativenetworks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fst235%2Fhse.deepgenerativenetworks/lists"}