{"id":13578117,"url":"https://github.com/moskomule/sam.pytorch","last_synced_at":"2025-04-14T22:34:20.118Z","repository":{"id":113718931,"uuid":"325422006","full_name":"moskomule/sam.pytorch","owner":"moskomule","description":"A PyTorch implementation of Sharpness-Aware Minimization for Efficiently Improving Generalization","archived":false,"fork":false,"pushed_at":"2021-03-16T03:05:13.000Z","size":12,"stargazers_count":136,"open_issues_count":0,"forks_count":10,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-28T10:47:35.825Z","etag":null,"topics":["optimizer","pytorch","sam"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moskomule.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-12-30T01:09:23.000Z","updated_at":"2024-12-26T16:13:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"4ffd79d2-de0d-4cc3-afd4-bc1f5178414c","html_url":"https://github.com/moskomule/sam.pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moskomule%2Fsam.pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moskomule%2Fsam.pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moskomule%2Fsam.pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moskomule%2Fsam.pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moskomule","download_url":"https://codeload.github.com/moskomule/sam.pytorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248972740,"owners_count":21191852,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["optimizer","pytorch","sam"],"created_at":"2024-08-01T15:01:27.650Z","updated_at":"2025-04-14T22:34:20.060Z","avatar_url":"https://github.com/moskomule.png","language":"Python","readme":"# sam.pytorch\n\nA PyTorch implementation of *Sharpness-Aware Minimization for Efficiently Improving Generalization* (\nForet+2020) [Paper](https://arxiv.org/abs/2010.01412), [Official implementation](https://github.com/google-research/sam)\n.\n\n## Requirements\n\n* Python\u003e=3.8\n* PyTorch\u003e=1.7.1\n\nTo run the example, you further need\n\n* `homura` by `pip install -U homura-core==2020.12.0`\n* `chika` by `pip install -U chika`\n\n## Example\n\n```commandline\npython cifar10.py [--optim.name {sam,sgd}] [--model {renst20, wrn28_2}] [--optim.rho 0.05]\n```\n\n### Results: Test Accuracy (CIFAR-10)\n\nModel       | SAM | SGD |\n---         | --- | --- |\nResNet-20   | 93.5| 93.2|\nWRN28-2     | 95.8| 95.4|\nResNeXT29   | 96.4| 95.8|\n\nSAM needs double forward passes per each update, thus training with SAM is slower than training with SGD. In case of\nResNet-20 training, 80 mins vs 50 mins on my environment. Additional options `--use_amp --jit_model` may slightly\naccelerates the training.\n\n## Usage\n\n`SAMSGD` can be used as a drop-in replacement of PyTorch optimizers by using a closure as follows. Also, it is compatible\nwith `lr_scheduler` and has `state_dict` and `load_state_dict`. Currently, this implementation does not support multiple parameter groups.\n\n```python\nfrom sam import SAMSGD\n\noptimizer = SAMSGD(model.parameters(), lr=1e-1, rho=0.05)\n\nfor input, target in dataset:\n    def closure():\n        optimizer.zero_grad()\n        output = model(input)\n        loss = loss_f(output, target)\n        loss.backward()\n        return loss\n\n\n    loss = optimizer.step(closure)\n```\n\n## Citation\n\n```bibtex\n@ARTICLE{2020arXiv201001412F,\n    author = {{Foret}, Pierre and {Kleiner}, Ariel and {Mobahi}, Hossein and {Neyshabur}, Behnam},\n    title = \"{Sharpness-Aware Minimization for Efficiently Improving Generalization}\",\n    year = 2020,\n    eid = {arXiv:2010.01412},\n    eprint = {2010.01412},\n}\n\n@software{sampytorch\n    author = {Ryuichiro Hataya},\n    titile = {sam.pytorch},\n    url    = {https://github.com/moskomule/sam.pytorch},\n    year   = {2020}\n}\n```\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoskomule%2Fsam.pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoskomule%2Fsam.pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoskomule%2Fsam.pytorch/lists"}