{"id":17983154,"url":"https://github.com/kaiyangzhou/mixstyle-release","last_synced_at":"2025-04-09T12:05:42.121Z","repository":{"id":41469345,"uuid":"282191005","full_name":"KaiyangZhou/mixstyle-release","owner":"KaiyangZhou","description":"Domain Generalization with MixStyle (ICLR'21)","archived":false,"fork":false,"pushed_at":"2022-10-06T12:44:06.000Z","size":36617,"stargazers_count":293,"open_issues_count":9,"forks_count":40,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-02T10:14:13.434Z","etag":null,"topics":["domain-generalization","feature-statistics","iclr2021","image-classification","person-reidentification","reinforcement-learning","representation-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KaiyangZhou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-24T10:30:14.000Z","updated_at":"2025-04-01T16:35:47.000Z","dependencies_parsed_at":"2023-01-19T10:31:30.884Z","dependency_job_id":null,"html_url":"https://github.com/KaiyangZhou/mixstyle-release","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaiyangZhou%2Fmixstyle-release","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaiyangZhou%2Fmixstyle-release/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaiyangZhou%2Fmixstyle-release/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaiyangZhou%2Fmixstyle-release/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KaiyangZhou","download_url":"https://codeload.github.com/KaiyangZhou/mixstyle-release/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248036063,"owners_count":21037092,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["domain-generalization","feature-statistics","iclr2021","image-classification","person-reidentification","reinforcement-learning","representation-learning"],"created_at":"2024-10-29T18:16:17.688Z","updated_at":"2025-04-09T12:05:42.101Z","avatar_url":"https://github.com/KaiyangZhou.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MixStyle\n\nThis repo contains the code of our ICLR'21 paper, \"Domain Generalization with MixStyle\".\n\nThe OpenReview link is https://openreview.net/forum?id=6xHJ37MVxxp.\n\n**########## Updates ############**\n\n**[06-10-2022]** New paper \"[On-Device Domain Generalization](https://arxiv.org/abs/2209.07521)\" is out! Code, models and datasets: https://github.com/KaiyangZhou/on-device-dg.\n\n**[12-10-2021]** Please note that the latest version for Dassl is `v0.5.0` (the changes might affect the performance if the original images are not square). See [this](https://github.com/KaiyangZhou/Dassl.pytorch#whats-new) for more details.\n\n**[06-07-2021]** Extension of our ICLR'21 paper is out: [MixStyle Neural Networks for Domain Generalization and Adaptation](https://arxiv.org/abs/2107.02053). This work extends the conference version mainly in the following ways: 1) A simple algorithmic extension enabling MixStyle to cope with unlabeled data; 2) New evidence showing that MixStyle works exceptionally well with extremely limited labels; 3) New experiments covering semi-supervised domain generalization and unsupervised domain adaptation. Code for reproducing the new experiments is available at `imcls/`.\n\n**[28-06-2021]** A new implementation of MixStyle is out, which merges `MixStyle2` to `MixStyle` and switches between random and cross-domain mixing using `self.mix`. The new features can be found [here](https://github.com/KaiyangZhou/Dassl.pytorch/issues/23).\n\n**[12-04-2021]** A variable `self._activated` is added to MixStyle to better control the computational flow. To deactivate MixStyle without modifying the model code, one can do\n```python\ndef deactivate_mixstyle(m):\n    if type(m) == MixStyle:\n        m.set_activation_status(False)\n\nmodel.apply(deactivate_mixstyle)\n```\nSimilarly, to activate MixStyle, one can do\n```python\ndef activate_mixstyle(m):\n    if type(m) == MixStyle:\n        m.set_activation_status(True)\n\nmodel.apply(activate_mixstyle)\n```\nNote that `MixStyle` has been included in [Dassl.pytorch](https://github.com/KaiyangZhou/Dassl.pytorch). See [the code](https://github.com/KaiyangZhou/Dassl.pytorch/blob/master/dassl/modeling/backbone/resnet.py#L280) for details.\n\n**[05-03-2021]** You might also be interested in our recently released survey on domain generalization at https://arxiv.org/abs/2103.02503, which summarizes the ten-year development in domain generalization, with coverage on the history, datasets, related problems, methodologies, potential directions, and so on.\n\n**##############################**\n\n**A brief introduction**: The key idea of MixStyle is to probablistically mix instance-level feature statistics of training samples across source domains. MixStyle improves model robustness to domain shift by implicitly synthesizing new domains at the feature level for regularizing the training of convolutional neural networks. This idea is largely inspired by [neural style transfer](https://arxiv.org/abs/1703.06868) which has shown that feature statistics are closely related to image style and therefore arbitrary image style transfer can be achieved by switching the feature statistics between a content and a style image.\n\nMixStyle is very easy to implement. Below we show a brief implementation of it in PyTorch. The full code can be found [here](https://github.com/KaiyangZhou/Dassl.pytorch/blob/master/dassl/modeling/ops/mixstyle.py).\n\n```python\nimport random\nimport torch\nimport torch.nn as nn\n\n\nclass MixStyle(nn.Module):\n    \"\"\"MixStyle.\n    Reference:\n      Zhou et al. Domain Generalization with MixStyle. ICLR 2021.\n    \"\"\"\n\n    def __init__(self, p=0.5, alpha=0.1, eps=1e-6, mix='random'):\n        \"\"\"\n        Args:\n          p (float): probability of using MixStyle.\n          alpha (float): parameter of the Beta distribution.\n          eps (float): scaling parameter to avoid numerical issues.\n          mix (str): how to mix.\n        \"\"\"\n        super().__init__()\n        self.p = p\n        self.beta = torch.distributions.Beta(alpha, alpha)\n        self.eps = eps\n        self.alpha = alpha\n        self.mix = mix\n        self._activated = True\n\n    def __repr__(self):\n        return f'MixStyle(p={self.p}, alpha={self.alpha}, eps={self.eps}, mix={self.mix})'\n\n    def set_activation_status(self, status=True):\n        self._activated = status\n\n    def update_mix_method(self, mix='random'):\n        self.mix = mix\n\n    def forward(self, x):\n        if not self.training or not self._activated:\n            return x\n\n        if random.random() \u003e self.p:\n            return x\n\n        B = x.size(0)\n\n        mu = x.mean(dim=[2, 3], keepdim=True)\n        var = x.var(dim=[2, 3], keepdim=True)\n        sig = (var + self.eps).sqrt()\n        mu, sig = mu.detach(), sig.detach()\n        x_normed = (x-mu) / sig\n\n        lmda = self.beta.sample((B, 1, 1, 1))\n        lmda = lmda.to(x.device)\n\n        if self.mix == 'random':\n            # random shuffle\n            perm = torch.randperm(B)\n\n        elif self.mix == 'crossdomain':\n            # split into two halves and swap the order\n            perm = torch.arange(B - 1, -1, -1) # inverse index\n            perm_b, perm_a = perm.chunk(2)\n            perm_b = perm_b[torch.randperm(B // 2)]\n            perm_a = perm_a[torch.randperm(B // 2)]\n            perm = torch.cat([perm_b, perm_a], 0)\n\n        else:\n            raise NotImplementedError\n\n        mu2, sig2 = mu[perm], sig[perm]\n        mu_mix = mu*lmda + mu2 * (1-lmda)\n        sig_mix = sig*lmda + sig2 * (1-lmda)\n\n        return x_normed*sig_mix + mu_mix\n```\n\nHow to apply MixStyle to your CNN models? Say you are using ResNet as the CNN architecture, and want to apply MixStyle after the 1st and 2nd residual blocks, you can first instantiate the MixStyle module using\n```python\nself.mixstyle = MixStyle(p=0.5, alpha=0.1)\n```\nduring network construction (in `__init__()`), and then apply MixStyle in the forward pass like\n```python\ndef forward(self, x):\n    x = self.conv1(x) # 1st convolution layer\n    x = self.res1(x) # 1st residual block\n    x = self.mixstyle(x)\n    x = self.res2(x) # 2nd residual block\n    x = self.mixstyle(x)\n    x = self.res3(x) # 3rd residual block\n    x = self.res4(x) # 4th residual block\n    ...\n```\n\nIn our paper, we have demonstrated the effectiveness of MixStyle on three tasks: image classification, person re-identification, and reinforcement learning. The source code for reproducing all experiments can be found in `mixstyle-release/imcls`, `mixstyle-release/reid`, and `mixstyle-release/rl`, respectively.\n\n*Takeaways* on how to apply MixStyle to your tasks:\n- Applying MixStyle to multiple lower layers is recommended (e.g., insert MixStyle after `res1` and `res2` in ResNets).\n- Do not apply MixStyle to the last layer that is the closest to the prediction layer.\n- Different tasks might favor different combinations.\n- If you want to use the same configuration for all tasks/datasets for fair comparison, we suggest adding MixStyle to two consecutive layers, such as `res1` and `res2` in ResNets.\n\nFor more analytical studies, please read our paper at https://openreview.net/forum?id=6xHJ37MVxxp.\n\nPlease also read the extended paper at https://arxiv.org/abs/2107.02053 for a more comprenehsive picture of MixStyle.\n\nTo cite MixStyle in your publications, please use the following bibtex entry\n\n```\n@inproceedings{zhou2021mixstyle,\n  title={Domain Generalization with MixStyle},\n  author={Zhou, Kaiyang and Yang, Yongxin and Qiao, Yu and Xiang, Tao},\n  booktitle={ICLR},\n  year={2021}\n}\n\n@article{zhou2021mixstylenn,\n  title={MixStyle Neural Networks for Domain Generalization and Adaptation},\n  author={Zhou, Kaiyang and Yang, Yongxin and Qiao, Yu and Xiang, Tao},\n  journal={arXiv:2107.02053},\n  year={2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaiyangzhou%2Fmixstyle-release","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkaiyangzhou%2Fmixstyle-release","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaiyangzhou%2Fmixstyle-release/lists"}