{"id":20663819,"url":"https://github.com/vita-group/sandwich-batch-normalization","last_synced_at":"2025-07-31T12:39:20.685Z","repository":{"id":52636373,"uuid":"339266409","full_name":"VITA-Group/Sandwich-Batch-Normalization","owner":"VITA-Group","description":"[WACV 2022] \"Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity\" by Xinyu Gong, Wuyang Chen, Tianlong Chen and Zhangyang Wang","archived":false,"fork":false,"pushed_at":"2021-12-29T09:45:55.000Z","size":23010,"stargazers_count":50,"open_issues_count":0,"forks_count":5,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-19T18:52:42.417Z","etag":null,"topics":["adversarial-training","batch-normalization","gan","nas","neural-architecture-search","normalization","pytorch","sabn","sandwich-batch-normalization","style-transfer"],"latest_commit_sha":null,"homepage":"https://github.com/VITA-Group/Sandwich-Batch-Normalization","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VITA-Group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-16T02:39:54.000Z","updated_at":"2024-11-18T06:32:14.000Z","dependencies_parsed_at":"2022-08-21T15:20:17.494Z","dependency_job_id":null,"html_url":"https://github.com/VITA-Group/Sandwich-Batch-Normalization","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/VITA-Group/Sandwich-Batch-Normalization","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FSandwich-Batch-Normalization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FSandwich-Batch-Normalization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FSandwich-Batch-Normalization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FSandwich-Batch-Normalization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VITA-Group","download_url":"https://codeload.github.com/VITA-Group/Sandwich-Batch-Normalization/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VITA-Group%2FSandwich-Batch-Normalization/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268040182,"owners_count":24185845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-31T02:00:08.723Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-training","batch-normalization","gan","nas","neural-architecture-search","normalization","pytorch","sabn","sandwich-batch-normalization","style-transfer"],"created_at":"2024-11-16T19:20:03.356Z","updated_at":"2025-07-31T12:39:20.648Z","avatar_url":"https://github.com/VITA-Group.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity\n\n[![MIT licensed](https://img.shields.io/badge/license-MIT-brightgreen.svg)](LICENSE.md)\n\nCode for [Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity](https://arxiv.org/abs/2102.11382).\n\n## Introduction\nWe present Sandwich Batch Normalization (SaBN), an extremely easy improvement of Batch Normalization (BN) with only a few lines of code changes.\n\n![method](imgs/architect.png)\n\nWe demonstrate the prevailing effectiveness of SaBN as a drop-in replacement in four tasks:\n1. **conditional image generation**,\n2. **neural architecture search**,\n3. **adversarial training**,\n4. **arbitrary neural style transfer**.\n\n## Usage\nCheck each of them for more information:\n1. [GAN](https://github.com/VITA-Group/Sandwich-Batch-Normalization/blob/main/GAN)\n2. [NAS](https://github.com/VITA-Group/Sandwich-Batch-Normalization/blob/main/NAS)\n3. [Adv](https://github.com/VITA-Group/Sandwich-Batch-Normalization/blob/main/Adv)\n4. [NST](https://github.com/VITA-Group/Sandwich-Batch-Normalization/blob/main/NST)\n\n## Main Results\n\n### 1. Conditional Image Generation\nUsing SaBN in conditional generation task enables an immediate performance boost. Evaluation results on CIFAR-10 are shown below:\n\n|       Model      | Inception Score ↑ |     FID ↓     |\n|------------------|-----------------|--------------|\n| AutoGAN          |       8.43      |        10.51 |\n| BigGAN           |       8.91      |         8.57 |\n| SNGAN            |       8.76      |        10.18 |\n| **AutoGAN-SaBN** (ours) |   8.72 (+0.29)  |  9.11 (−1.40) |\n| **BigGAN-SaBN** (ours) |   9.01 (+0.10)   | 8.03 (−0.54) |\n| **SNGAN-SaBN** (ours) |   8.89 (+0.13)  |  8.97 (−1.21) |\n\nVisual results on ImageNet (128*128 resolution):\n\nSNGAN          |  SNGAN-SaBN (ours)\n:-------------------------:|:-------------------------:\n![CIFAR100](imgs/sngan_imagenet.png)  |  ![ImageNet](imgs/sngan_sabn_imagenet.png)\n\n\n### 2. Neural Architecture Search\nWe adopted DARTS as the baseline search algorithm. Results on NAS-Bench-201 are presented below:\n\n| Method            | CIFAR-100 (top1) |  ImageNet (top1)  |\n|-------------------|:----------------:|:----------------:|\n| DARTS             |   44.05 ± 7.47   |   36.47 ± 7.06   |\n| DARTS-SaBN (ours) | **71.56 ± 1.39** | **45.85 ± 0.72** |\n\nCIFAR-100            |  ImageNet16-120\n:-------------------------:|:-------------------------:\n![CIFAR100](imgs/DARTS_e35_cifar100.png)  |  ![ImageNet](imgs/DARTS_e35_imagenet100.png)\n\n### 3. Adversarial Training\nEvaluation results:\n\n| Evaluation |   BN  | AuxBN (clean branch) | SaAuxBN (clean branch) (ours) |\n|:----------:|:-----:|:--------------------:|:----------------------:|\n| Clean (SA) | 84.84 |         94.47        |          **94.62**         |\n\n|  Evaluation |   BN  | AuxBN (adv branch) | SaAuxBN (adv branch) (ours) |\n|:-----------:|:-----:|:------------------:|:--------------------:|\n|  Clean (SA) | **84.84** |        83.42       |         84.08        |\n| PGD-10 (RA) | 41.57 |        43.05       |         **44.93**        |\n| PGD-20 (RA) | 40.02 |        41.60       |         **43.14**        |\n\n### 4. Arbitrary Neural Style Transfer\n\nThe model equipped with the proposed SaAdaIN achieves lower style \u0026 content loss on both training and testing set.\n\n**Training loss**:\n\nTraining style loss            |  Training content loss\n:-------------------------:|:-------------------------:\n![st](imgs/st_losses.png)  |  ![ct](imgs/ct_losses.png)\n\n**Validation loss**:\n\nValidation style loss            | Validation content loss\n:-------------------------:|:-------------------------:\n![val_st](imgs/val_st_losses.png)  |  ![val_ct](imgs/val_ct_losses.png)\n\n## Citation\nIf you find this work is useful to your research, please cite our paper:\n```bibtex\n@InProceedings{Gong_2022_WACV,\n  title={Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity},\n  author={Gong, Xinyu and Chen, Wuyang and Chen, Tianlong and Wang, Zhangyang},\n  journal={Winter Conference on Applications of Computer Vision (WACV)},\n  year={2022}\n}\n```\n\n## Acknowledgement\n1. NAS codebase from [NAS-Bench-201](https://github.com/D-X-Y/AutoDL-Projects/blob/main/docs/NAS-Bench-201.md).\n2. NST codebase from [AdaIN](https://github.com/naoto0804/pytorch-AdaIN).\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Fsandwich-batch-normalization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvita-group%2Fsandwich-batch-normalization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvita-group%2Fsandwich-batch-normalization/lists"}