{"id":19320166,"url":"https://github.com/invictus717/unidg","last_synced_at":"2025-06-17T00:36:07.565Z","repository":{"id":200612286,"uuid":"705263500","full_name":"invictus717/UniDG","owner":"invictus717","description":"Towards Unified and Effective Domain Generalization","archived":false,"fork":false,"pushed_at":"2023-11-27T13:18:44.000Z","size":764,"stargazers_count":30,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-26T09:11:45.902Z","etag":null,"topics":["artificial-intelligence","computer-vision","domain-generalization","machine-learning","test-time-adaptation"],"latest_commit_sha":null,"homepage":"https://invictus717.github.io/Generalization/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/invictus717.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-10-15T14:26:12.000Z","updated_at":"2025-03-06T02:19:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"cef67edd-5e64-4c48-a966-aec1e9d5fe9f","html_url":"https://github.com/invictus717/UniDG","commit_stats":null,"previous_names":["invictus717/unidg"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/invictus717%2FUniDG","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/invictus717%2FUniDG/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/invictus717%2FUniDG/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/invictus717%2FUniDG/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/invictus717","download_url":"https://codeload.github.com/invictus717/UniDG/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248582407,"owners_count":21128369,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","computer-vision","domain-generalization","machine-learning","test-time-adaptation"],"created_at":"2024-11-10T01:27:22.234Z","updated_at":"2025-04-12T14:40:46.167Z","avatar_url":"https://github.com/invictus717.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# *Towards Unified and Effective Domain Generalization*\n\n\u003cdiv align=\"center\"\u003e\n    \u003cspan class=\"author-block\"\u003e\n    \u003ca href=\"https://invictus717.github.io/\" target=\"_blank\"\u003eYiyuan Zhang\u003c/a\u003e\u003csup\u003e1,2*\u003c/sup\u003e,\u003c/span\u003e\n    \u003cspan class=\"author-block\"\u003e\n    \u003ca href=\"https://kxgong.github.io/\" target=\"_blank\"\u003eKaixiong Gong\u003c/a\u003e\u003csup\u003e1,2*\u003c/sup\u003e,\u003c/span\u003e\n    \u003cspan class=\"author-block\"\u003e\n    \u003ca href=\"https://dingxiaohan.xyz/\" target=\"_blank\"\u003eXiaohan Ding\u003c/a\u003e\u003csup\u003e4\u003c/sup\u003e,\n    \u003c/span\u003e\n    \u003c/br\u003e\n    \u003ca href='http://kpzhang93.github.io/' target='_blank'\u003eKaipeng Zhang\u003c/a\u003e\u003csup\u003e2,†\u003c/sup\u003e\n    \u003cspan class=\"author-block\"\u003e\n    \u003ca href=\"https://scholar.google.com/citations?user=tGxDe8sAAAAJ\u0026hl=en\u0026oi=ao\" target=\"_blank\"\u003eFangrui Lv\u003c/a\u003e\u003csup\u003e5\u003c/sup\u003e,\n    \u003c/span\u003e\n    \u003cspan class=\"author-block\"\u003e\n    \u003ca href=\"https://people.eecs.berkeley.edu/~keutzer/\" target=\"_blank\"\u003eKurt Keutzer\u003c/a\u003e\u003csup\u003e3\u003c/sup\u003e,\n    \u003c/span\u003e\n    \u003cspan class=\"author-block\"\u003e\n    \u003ca href=\"http://people.eecs.berkeley.edu/~xyyue/\" target=\"_blank\"\u003eXiangyu Yue\u003c/a\u003e\u003csup\u003e1†\u003c/sup\u003e\n    \u003c/span\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n    \u003csup\u003e1\u003c/sup\u003e\n    \u003ca href='http://mmlab.ie.cuhk.edu.hk/' target='_blank'\u003eMultimedia Lab, The Chinese University of Hong Kong\u003c/a\u003e\u0026emsp;\n    \u003c/br\u003e\n    \u003csup\u003e2\u003c/sup\u003e \u003ca href='https://github.com/OpenGVLab' target='_blank'\u003eOpenGVLab，Shanghai AI Laboratory \n    \u003c/a\u003e\n    \u003csup\u003e3\u003c/sup\u003e \u003ca href='https://eecs.berkeley.edu/' target='_blank'\u003eUC Berkeley\u003c/a\u003e\n    \u003csup\u003e4\u003c/sup\u003e \u003ca href='https://ai.tencent.com/' target='_blank'\u003eTencent AI Lab\u003c/a\u003e\n    \u003csup\u003e5\u003c/sup\u003e \u003ca href='https://ai.tencent.com/' target='_blank'\u003eTsinghua University\u003c/a\u003e\n    \u003c/br\u003e\n    \u003csup\u003e*\u003c/sup\u003e Equal Contribution\u0026emsp;\n    \u003csup\u003e†\u003c/sup\u003e Corresponding Authors\u0026emsp;\n\u003c/div\u003e\n\n-----------------\n[![arXiv](https://img.shields.io/badge/arXiv-2310.10008-b31b1b.svg?style=flat)](https://arxiv.org/abs/2310.10008)\n[![website](https://img.shields.io/badge/Project-Website-brightgreen)](https://invictus717.github.io/Generalization/)\n[![LICENSE](https://img.shields.io/badge/LICENSE-Apache2.0-blue)](https://www.apache.org/licenses/LICENSE-2.0)\n\u003c!-- ![](https://img.shields.io/github/stars/invictus717/UniDG?style=social) --\u003e\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-unified-and-effective-domain/domain-generalization-on-terraincognita)](https://paperswithcode.com/sota/domain-generalization-on-terraincognita?p=towards-unified-and-effective-domain)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-unified-and-effective-domain/domain-generalization-on-office-home)](https://paperswithcode.com/sota/domain-generalization-on-office-home?p=towards-unified-and-effective-domain)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-unified-and-effective-domain/domain-generalization-on-vlcs)](https://paperswithcode.com/sota/domain-generalization-on-vlcs?p=towards-unified-and-effective-domain)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-unified-and-effective-domain/domain-generalization-on-domainnet)](https://paperswithcode.com/sota/domain-generalization-on-domainnet?p=towards-unified-and-effective-domain)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-unified-and-effective-domain/domain-generalization-on-pacs-2)](https://paperswithcode.com/sota/domain-generalization-on-pacs-2?p=towards-unified-and-effective-domain)\n## Highlights ⭐⭐⭐\n- 🚀 ***UniDG* is an effective Test-Time Adaptation scheme. It brings out an average improvement to existing DG methods by `+5.0%` Accuracy on DomainBed benchmarks including PACS, VLCS, OfficeHome, and TerraInc datasets.**\n  \n\u003cp align=\"center\" width=\"100%\"\u003e\n\u003cimg src=\"assets/cmp.png\"  width=\"80%\" height=\"50%\"\u003e\n\u003c/p\u003e\n\n- 🚀 ***UniDG* is architecture-agnostic. Unified with 10+ visual backbones including CNN, MLP, and transformer-based models, UniDG can bring out consistent performance enhancement of `+5.4%` average on domain generalization.**\n\n\u003cp align=\"center\" width=\"100%\"\u003e\n\u003cimg src=\"assets/backbone.png\"  width=\"80%\" height=\"80%\"\u003e\n\u003c/p\u003e\n\n- 🏆 **Achieved `79.6 mAcc` on PACS , VLCS , OfficeHome, TerraIncognita , and DomainNet datasets.**\n\n### Abstract\n\nWe propose UniDG, a novel and Unified framework for Domain Generalization that is capable of significantly enhancing the out-of-distribution generalization\nperformance of foundation models regardless of their architectures. The core\nidea of UniDG is to finetune models during the inference stage, which saves\nthe cost of iterative training. Specifically, we encourage models to learn the\ndistribution of test data in an unsupervised manner and impose a penalty regarding the updating step of model parameters. The penalty term can effectively reduce the catastrophic forgetting issue as we would like to maximally preserve the valuable knowledge in the original model. Empirically, across 12 visual backbones, including CNN-, MLP-, and Transformer-based models, ranging\nfrom 1.89M to 303M parameters, UniDG shows an average accuracy improvement of +5.4% on DomainBed. \n\n## Features 🎉🎉🎉\n\n1. 🌟 ***Extensibility***:  we intergrate ***UniDG*** with Domainbed. More networks and algorithms can be built easily with our framework, and UniDG brings out an average improvement of `+5.0%` to existing methods including ERM, CORAL, and MIRO.\n\n2. 🌟 ***Reproducibility***: all implemented models are trained on various tasks at least three times. Mean±std is provided in the [UniDG paper](https://arxiv.org/abs/2310.10008). *Pretrained models and logs* are available.\n\n3. 🌟 ***Ease of Use***:  we develop tools to charge experimental logs with json files, which can transform results directly into latex:\n\n   ![logs](assets/logs.png)\n\n4. 🌟 ***Visualization Tools***: we provides scripts to easily visualize results by T-SNE and performance curves:\n    * Convergence Curves:\n   \n   ![curves](assets/curves.png)\n\n   * T-SNE Visualization Results:\n\n   ![T-SNE](assets/T-SNE.png)\n\n## Model Zoo for UniDG\n\nWe provide pretrained checkpoints with base `ERM` algorithm to reproduce our experimental results conveniently.\n\n*Note that `IID_best.pkl` is the pretrained source model.*\n\n* CORAL Source Models\n\n| Backbone         |                Dataset                | Algorithm |  Base Model  |   Adaptation |                       Google Drive                          |\n| ---------------- | :-----------------------------------: | :-------: | :-------: | :--------: | :----------------------------------------------------------: |\n| ResNet-50        | VLCS \\| PACS \\| OfficeHome \\| TerraInc\\| DomainNet |    CORAL    | 64.1 ± 0.1 | 69.3 ± 0.2 | [ckpt](https://drive.google.com/drive/folders/1-Vis3Ge_DhhU10YqsYUU_Y-9KAzKZ92w?usp=drive_link) |\n| Swin Transformer        | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    CORAL    | 77.2 ± 0.1 | 82.5 ± 0.2 | [ckpt](https://drive.google.com/drive/folders/1OKn_wd7pDjoIxhB_G38CkBMUr5MvEAdt?usp=sharing) |\n| ConvNeXt       | VLCS \\| PACS \\| OfficeHome \\| TerraInc\\| DomainNet |    CORAL    | 75.1 ± 0.1 | 79.6 ± 0.3 | [ckpt](https://drive.google.com/drive/folders/1xt3J_kfe8pEDZ8CjeX5CCexS5F5owLbP?usp=sharing) |\n\n* ERM Source Models \n\n| Backbone         |                Dataset                | Algorithm |  Base Model  |   Adaptation |                        Google Drive                          |\n| ---------------- | :-----------------------------------: | :-------: | :-------: | :--------: | :----------------------------------------------------------: | \n| ResNet-18        | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 63.0 ± 0.0 | 67.2 ± 0.2 |[ckpt](https://drive.google.com/drive/folders/1dF9TsKuLOIU_ii8jfDka4FitXlYGgO8D?usp=share_link) |\n| ResNet-50        | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 67.6 ± 0.0 | 73.1 ± 0.2 | [ckpt](https://drive.google.com/drive/folders/15d8EPsOMmN0q8NHKD6GQA1GfHaxY4jMN?usp=share_link) |\n| ResNet-101       | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 68.1 ± 0.1 | 72.3 ± 0.3 | [ckpt](https://drive.google.com/drive/folders/1TEA46H9JfDGBPhc7cGatYSDuBV63jCoQ?usp=share_link) |\n| Mobilenet V3     | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 58.9 ± 0.0 | 65.3 ± 0.2 | [ckpt](https://drive.google.com/drive/folders/1EJrwFIkihcIubznVeLNYp5UwCWSNqIdv?usp=share_link) |\n| EfficientNet V2  | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 67.2 ± 0.0 | 72.1 ± 0.3 | [ckpt](https://drive.google.com/drive/folders/1a3wwAo4LKiG9BrETw0lkn3mdXtnUQVHU?usp=share_link) |\n| ConvNeXt-B       | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 79.7 ± 0.0 | 83.7 ± 0.1 |[ckpt](https://drive.google.com/drive/folders/1p1b0ZUn8oT-tc9JCb1G8r5E0R7l2zVE6?usp=share_link) |\n| ViT-B16          | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 69.5 ± 0.0 | 75.4 ± 0.2 | [ckpt](https://drive.google.com/drive/folders/1okU78l0POKLg7m1q-c07Xhpd0tGa6Awl?usp=share_link) |\n| ViT-L16          | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 74.1 ± 0.0 | 79.9 ± 0.3 |[ckpt](https://drive.google.com/drive/folders/1EfNN4H1ZQCO6xnOvxRleMgJdMmDXLL-M?usp=share_link) |\n| DeiT             | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 73.5 ± 0.0 | 77.8 ± 0.2 | [ckpt](https://drive.google.com/drive/folders/1XwAfBPCTana7LP8wnp48wzzO-H66nEXB?usp=share_link) |\n| Swin Transformer | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 77.2 ± 0.0 | 81.5 ± 0.3 | [ckpt](https://drive.google.com/drive/folders/1fhp-jXJZU4VORguDupoGTEpKd6TeErS9?usp=share_link) |\n| Mixer-B16        | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 57.2 ± 0.1 | 65.6 ± 0.3 | [ckpt](https://drive.google.com/drive/folders/1myX-InLRTkKYC4akK1LmJeIfcAc5SZ6u?usp=share_link) |\n| Mixer-L16        | VLCS \\| PACS \\| OfficeHome \\| TerraInc |    ERM    | 67.4 ± 0.0 |  73.0 ± 0.2 |[ckpt](https://drive.google.com/drive/folders/1o-T93TscEEe1m9ZZBfpUfslMQ74Mhn9c?usp=share_link) |\n\n\n## 🔧 Get Started\n\nEnvironments Set up\n\n```sh\ngit clone https://github.com/invictus717/UniDG.git \u0026\u0026 cd UniDG\nconda env create -f UniDG.yaml \u0026\u0026  conda activate UniDG\n```\n\nDatasets Preparation\n\n```sh\npython -m domainbed.scripts.download \\\n       --data_dir=./data\n```\n\n## ⏳ Training \u0026 Test-time adaptation\n\nTrain a model:\n\n```sh\npython -m domainbed.scripts.train\\\n       --data_dir=./data \\\n       --algorithm ERM \\\n       --dataset OfficeHome \\\n       --test_env 2 \\\n       --hparams \"{\\\"backbone\\\": \\\"resnet50\\\"}\" \\\n       --output_dir my/pretrain/ERM/resnet50\n```\n\n*Note that you can download our pretrained checkpoints in the [Model Zoo](https://github.com/invictus717/UniDG#model-zoo-for-unidg).*\n\nThen you can perform self-supervised adaptation:\n\n```shell\npython -m domainbed.scripts.unsupervised_adaptation \\\n       --input_dir my/pretrain/ERM/resnet50 \\\n       --adapt_algorithm=UniDG\n```\n\n## 📆 Collect Experimental Results\n\nThen you can perform self-supervised adaptation:\n\n```shell\npython -m domainbed.scripts.collect_all_results\\\n       --input_dir=my/pretrain/ERM \\\n       --adapt_dir=results/ERM/resnet50 \\\n       --output_dir=log/UniDG/ \\\n       --adapt_algorithm=UniDG \\\n       --latex\n```\n\n## 📈 Visualization results\n\nFor T-SNE visualization: \n\n```bash\npython -m domainbed.scripts.visualize_tsne\\\n       --input_dir=my/pretrain/ERM \\\n       --adapt_dir=UniDG/results/ERM/resnet50 \\\n       --output_dir=log/UniDG/ \\\n       --adapt_algorithm=UniDG \\\n       --latex\n```\n\nFor performance curves visualization: \n\n```bash\npython -m domainbed.scripts.visualize_curves\\\n       --input_dir=my/pretrain/ERM \\\n       --adapt_dir=UniDG/results/ERM/resnet50 \\\n       --output_dir=log/UniDG/ \\\n       --adapt_algorithm=UniDG \\\n       --latex\n```\n\n\n## Citation\n\nIf this work is helpful for your research, please consider citing the following BibTeX entry.\n\n```\n@article{zhang2023unified,\n      title={Towards Unified and Effective Domain Generalization}, \n      author={Yiyuan Zhang and Kaixiong Gong and Xiaohan Ding and Kaipeng Zhang and Fangrui Lv and Kurt Keutzer and Xiangyu Yue},\n      year={2023},\n      eprint={2310.10008},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\n## Acknowledge\n\nThis repository is based on [DomainBed](https://github.com/facebookresearch/DomainBed), [T3A](https://github.com/matsuolab/T3A), [timm](https://github.com/fastai/timmdocs). Thanks a lot for their great works!\n\n## License\n\nThis repository is released under the Apache 2.0 license as found in the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) file.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finvictus717%2Funidg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finvictus717%2Funidg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finvictus717%2Funidg/lists"}