{"id":19839822,"url":"https://github.com/qdata/pgrad","last_synced_at":"2025-05-01T19:30:28.412Z","repository":{"id":159010908,"uuid":"629257398","full_name":"QData/PGrad","owner":"QData","description":null,"archived":false,"fork":false,"pushed_at":"2023-05-03T15:52:05.000Z","size":9392,"stargazers_count":9,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-06T17:05:12.333Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/QData.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-18T00:35:29.000Z","updated_at":"2025-02-12T17:25:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"a7d375eb-6e04-4319-bf25-feffeabefa77","html_url":"https://github.com/QData/PGrad","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QData%2FPGrad","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QData%2FPGrad/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QData%2FPGrad/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QData%2FPGrad/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/QData","download_url":"https://codeload.github.com/QData/PGrad/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251932522,"owners_count":21667158,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T12:24:29.790Z","updated_at":"2025-05-01T19:30:28.405Z","avatar_url":"https://github.com/QData.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Welcome to our code repo for PGrad\n\nSee the image below for an illustration.\n\u003cp align='center'\u003e\u003cimg src=\"https://github.com/QData/PGrad/blob/main/figures/Framework.png\" alt=\"Training Paradigm\" width=\"700\"/\u003e\u003c/p\u003e\n\nTo get a quick introduction of PGrad, please check out our ICLR [Slides] (https://github.com/QData/PGrad/blob/main/PGrad_ICLR.pdf)\n\nThis is the official PyTorch implementation of our ICLR 2023 paper [PGrad: Learning Principal Gradients for Domain Generalization](https://openreview.net/pdf?id=CgCmwcfgEdH).\n\nMachine learning models fail to perform when facing out-of-distribution (OOD) domains, a challenging task known as domain generalization (DG). In this work, we develop a novel DG training strategy, we call PGrad, to learn a robust gradient direction, improving models' generalization ability on unseen domains.  The proposed gradient aggregates the principal directions of a sampled roll-out optimization trajectory that measures the training dynamics across all training domains. PGrad's gradient design forces the DG training to ignore domain-dependent noise signals and updates all training domains with a robust direction covering main components of parameter dynamics.  We further improve PGrad via bijection-based computational refinement and directional plus length-based calibrations. Our theoretical proof connects PGrad to the spectral analysis of Hessian in training neural networks. \n\n## How to use our code\n\nThis repository contains code to reproduce the main results of our paper.\n\nOur code is largely based on the DomainBed(https://github.com/facebookresearch/DomainBed).\n\nTo train a model: \n\n```sh\npython -m domainbed.scripts.train\\\n       --data_dir=./domainbed/data/\\\n       --algorithm PGrad\\\n       --dataset PACS\\\n       --test_env 0\n```\n\nTo lunch a sweep and reproduce the main results:\n\n```sh\npython -m domainbed.scripts.sweep launch\\\n       --data_dir=./domainbed/data/ \\\n       --output_dir ./results\\\n       --command_launcher multi_gpu\\ \n       --algorithms  PGrad\\\n       --datasets OfficeHome TerraIncognita PACS VLCS\\ \n       --n_hparams 2\\\n       --n_trials 3\\\n       --single_test_envs\\\n```\n\n## Analysis code validating the effectiveness of the sequential training in PGrad\n\nPGrad learns robust update direcitions from those principal components of the sampled parameter trajectories. The trajectories are sampled by performing sequential training on each training domains. We analyze in the paper that sequential training will reinforce robust directions; parallel training will enlarge domain-specific noises and suppress robust directions. \n\nFor comparison, run PGradParallel with:\n\n```sh\npython -m domainbed.scripts.train\\\n       --data_dir=./domainbed/data/ \\\n       --algorithm PGradParallel\\\n       --dataset VLCS\\\n       --test_env 0\n```\n\nUsing the test domain accuracy as the indicator, PGrad will consistently outperform ERM, while PGradParallel has the worst generalization ability.\n\n## Reference\n\nIf you found our work helpful, we would appreciate if you considered citing the paper that is most relevant to your work:\n\n```\n@inproceedings{\n       wang2023pgrad,\n       title={{PG}rad: Learning Principal Gradients For Domain Generalization},\n       author={Zhe Wang and Jake Grigsby and Yanjun Qi},\n       booktitle={The Eleventh International Conference on Learning Representations },\n       year={2023},\n       url={https://openreview.net/forum?id=CgCmwcfgEdH}\n       }\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqdata%2Fpgrad","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqdata%2Fpgrad","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqdata%2Fpgrad/lists"}