{"id":13564086,"url":"https://github.com/prigoyal/pytorch_memonger","last_synced_at":"2025-09-06T03:46:09.417Z","repository":{"id":44840015,"uuid":"113219145","full_name":"prigoyal/pytorch_memonger","owner":"prigoyal","description":"Experimental ground for optimizing memory of pytorch models","archived":false,"fork":false,"pushed_at":"2018-04-23T20:24:58.000Z","size":208,"stargazers_count":365,"open_issues_count":7,"forks_count":35,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-05-20T06:05:16.930Z","etag":null,"topics":["memory-management","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prigoyal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-05T18:37:15.000Z","updated_at":"2025-04-02T19:20:37.000Z","dependencies_parsed_at":"2022-09-07T20:23:13.313Z","dependency_job_id":null,"html_url":"https://github.com/prigoyal/pytorch_memonger","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/prigoyal/pytorch_memonger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prigoyal%2Fpytorch_memonger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prigoyal%2Fpytorch_memonger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prigoyal%2Fpytorch_memonger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prigoyal%2Fpytorch_memonger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prigoyal","download_url":"https://codeload.github.com/prigoyal/pytorch_memonger/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prigoyal%2Fpytorch_memonger/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273854594,"owners_count":25180011,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["memory-management","pytorch"],"created_at":"2024-08-01T13:01:26.425Z","updated_at":"2025-09-06T03:46:09.392Z","avatar_url":"https://github.com/prigoyal.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# PyTorch Memory optimizations via gradient checkpointing\n\nThis repository contains implementation of various PyTorch models using the\n**gradient checkpointing**[1] which allows trading compute for memory and hence allows\ntraining bigger/wider models and use large minibatch sizes.\n\nThe application of checkpointing is showcased on various models:\n\n- ResNet\n- DenseNet\n- LSTM model from pytorch examples [here](https://github.com/pytorch/examples/blob/master/word_language_model/model.py)\n- VNet model which is used in medical imaging applications, available [here](https://github.com/mattmacy/vnet.pytorch)\n\nResults of checkpointing on these models are showcased below:\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"tutorial/results.png\"  width=\"90%\" /\u003e\u003c/p\u003e\n\nIn order to use the models, you need to install PyTorch master following instructions from [here](https://github.com/pytorch/pytorch/#from-source)\n\nTo run checkpointed models and their baseline tests, follow the commands below:\n```\n# for checkpointed\npython test_memory_optimized.py\n\n# for baseline\npython test_memory_optimized.py\n```\n\n## Tutorial\n\nWe provide a [tutorial](https://github.com/prigoyal/pytorch_memonger/blob/master/tutorial/Checkpointing_for_PyTorch_models.ipynb) to describe how to use checkpointing for various kinds of\nmodels.\n\nThere are few special kinds of layers like Batch normalization, dropout that should\nbe handled carefully. The details for handling those are also available in the\ntutorial\n\n## References\n\n[1]. Siskind, Jeffrey Mark, and Barak A. Pearlmutter. \"Divide-and-Conquer Checkpointing for Arbitrary Programs with No User Annotation.\" arXiv preprint arXiv:1708.06799 (2017).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprigoyal%2Fpytorch_memonger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprigoyal%2Fpytorch_memonger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprigoyal%2Fpytorch_memonger/lists"}