{"id":15640033,"url":"https://github.com/rasbt/pytorch-memory-optim","last_synced_at":"2025-08-21T19:29:35.755Z","repository":{"id":177725448,"uuid":"659388576","full_name":"rasbt/pytorch-memory-optim","owner":"rasbt","description":"This code repository contains the code used for my \"Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch\" blog post.","archived":false,"fork":false,"pushed_at":"2023-07-14T20:11:55.000Z","size":1192,"stargazers_count":87,"open_issues_count":0,"forks_count":10,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-12T17:51:55.596Z","etag":null,"topics":["deep-learning","llm","memory-optimization","pytorch","vision","vit"],"latest_commit_sha":null,"homepage":"https://lightning.ai/pages/community/tutorial/pytorch-memory-vit-llm/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rasbt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-27T18:16:56.000Z","updated_at":"2024-12-26T17:59:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"b4639c4c-e689-477a-8995-241c72e58013","html_url":"https://github.com/rasbt/pytorch-memory-optim","commit_stats":null,"previous_names":["rasbt/pytorch-memory-optim"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rasbt%2Fpytorch-memory-optim","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rasbt%2Fpytorch-memory-optim/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rasbt%2Fpytorch-memory-optim/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rasbt%2Fpytorch-memory-optim/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rasbt","download_url":"https://codeload.github.com/rasbt/pytorch-memory-optim/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246155984,"owners_count":20732355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","llm","memory-optimization","pytorch","vision","vit"],"created_at":"2024-10-03T11:30:18.252Z","updated_at":"2025-03-30T21:31:46.446Z","avatar_url":"https://github.com/rasbt.png","language":"Python","readme":"# Optimizing PyTorch Memory Usage\n\n\nThis code repository contains the code used for my \"Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch\" blog post. \n\nYou can install the dependencies via\n\n\n```bash\npip install -r requirements.txt\n```\n\nThe scripts are all standalone scripts and can be run by executing \n\n```bash\npython 01_pytorch-vit.py\n```\n\nand so forth. The only requirement is to have the [`local_utilities.py`](local_utilities.py) in the same folder as the script as it contains some data loading utilities that are reused across all the scripts.\n\nI tracked the script outputs in the [`logs.md`](logs.md) file.\n\n\n\n![overview](figures/overview.png)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frasbt%2Fpytorch-memory-optim","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frasbt%2Fpytorch-memory-optim","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frasbt%2Fpytorch-memory-optim/lists"}