{"id":13738522,"url":"https://github.com/Oldpan/Pytorch-Memory-Utils","last_synced_at":"2025-05-08T16:34:31.244Z","repository":{"id":43537053,"uuid":"136622423","full_name":"Oldpan/Pytorch-Memory-Utils","owner":"Oldpan","description":"pytorch memory track code","archived":false,"fork":false,"pushed_at":"2021-05-04T14:11:27.000Z","size":20,"stargazers_count":1004,"open_issues_count":19,"forks_count":155,"subscribers_count":16,"default_branch":"master","last_synced_at":"2024-11-15T07:34:41.352Z","etag":null,"topics":["gpu","memory","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Oldpan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-08T13:27:20.000Z","updated_at":"2024-11-05T08:20:10.000Z","dependencies_parsed_at":"2022-07-10T16:31:48.996Z","dependency_job_id":null,"html_url":"https://github.com/Oldpan/Pytorch-Memory-Utils","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oldpan%2FPytorch-Memory-Utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oldpan%2FPytorch-Memory-Utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oldpan%2FPytorch-Memory-Utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oldpan%2FPytorch-Memory-Utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Oldpan","download_url":"https://codeload.github.com/Oldpan/Pytorch-Memory-Utils/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253105647,"owners_count":21855071,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gpu","memory","pytorch"],"created_at":"2024-08-03T03:02:25.134Z","updated_at":"2025-05-08T16:34:30.981Z","avatar_url":"https://github.com/Oldpan.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Pytorch-Memory-Utils\n\nThese codes can help you to detect your GPU memory during training with Pytorch.\n\nA blog about this tool and explain the details : https://oldpan.me/archives/pytorch-gpu-memory-usage-track\n\n# Usage:\n\nPut ``modelsize_estimate.py`` or ``gpu_mem_track.py`` under your current working directory and import them.\n\n## The following is the print content.\n\n- Calculate the memory usage of a single model\n```\nModel Sequential : params: 0.450304M\nModel Sequential : intermedite variables: 336.089600 M (without backward)\nModel Sequential : intermedite variables: 672.179200 M (with backward)\n```\n- Track the amount of GPU memory usage\n```markdown\n# 30-Apr-21-20:25:29-gpu_mem_track.txt\n\nGPU Memory Track | 30-Apr-21-20:25:29 | Total Tensor Used Memory:0.0    Mb Total Used Memory:0.0    Mb\n\n\nAt main.py line 10: \u003cmodule\u003e                          Total Tensor Used Memory:0.0    Mb Total Allocated Memory:0.0    Mb\n\n+ | 1 * Size:(64, 64, 3, 3)       | Memory: 0.1406 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 1 * Size:(128, 128, 3, 3)     | Memory: 0.5625 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 1 * Size:(256, 128, 3, 3)     | Memory: 1.125 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 1 * Size:(512, 256, 3, 3)     | Memory: 4.5 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 3 * Size:(256, 256, 3, 3)     | Memory: 6.75 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 8 * Size:(512,)               | Memory: 0.0156 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 2 * Size:(64,)                | Memory: 0.0004 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 7 * Size:(512, 512, 3, 3)     | Memory: 63.0 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 4 * Size:(256,)               | Memory: 0.0039 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 1 * Size:(128, 64, 3, 3)      | Memory: 0.2812 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 2 * Size:(128,)               | Memory: 0.0009 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n+ | 1 * Size:(64, 3, 3, 3)        | Memory: 0.0065 M | \u003cclass 'torch.nn.parameter.Parameter'\u003e | torch.float32\n\nAt main.py line 12: \u003cmodule\u003e                          Total Tensor Used Memory:76.4   Mb Total Allocated Memory:76.4   Mb\n\n+ | 1 * Size:(60, 3, 512, 512)    | Memory: 180.0 M | \u003cclass 'torch.Tensor'\u003e | torch.float32\n+ | 1 * Size:(40, 3, 512, 512)    | Memory: 120.0 M | \u003cclass 'torch.Tensor'\u003e | torch.float32\n+ | 1 * Size:(30, 3, 512, 512)    | Memory: 90.0 M | \u003cclass 'torch.Tensor'\u003e | torch.float32\n\nAt main.py line 18: \u003cmodule\u003e                          Total Tensor Used Memory:466.4  Mb Total Allocated Memory:466.4  Mb\n\n+ | 1 * Size:(120, 3, 512, 512)   | Memory: 360.0 M | \u003cclass 'torch.Tensor'\u003e | torch.float32\n+ | 1 * Size:(80, 3, 512, 512)    | Memory: 240.0 M | \u003cclass 'torch.Tensor'\u003e | torch.float32\n\nAt main.py line 23: \u003cmodule\u003e                          Total Tensor Used Memory:1066.4 Mb Total Allocated Memory:1066.4 Mb\n\n- | 1 * Size:(40, 3, 512, 512)    | Memory: 120.0 M | \u003cclass 'torch.Tensor'\u003e | torch.float32\n- | 1 * Size:(120, 3, 512, 512)   | Memory: 360.0 M | \u003cclass 'torch.Tensor'\u003e | torch.float32\n\nAt main.py line 29: \u003cmodule\u003e                          Total Tensor Used Memory:586.4  Mb Total Allocated Memory:586.4  Mb\n```\n\n## How to use\n\n### Track the amount of GPU memory usage\nsimple example:\n\n```python\nimport torch\n\nfrom torchvision import models\nfrom gpu_mem_track import MemTracker\n\ndevice = torch.device('cuda:0')\n\ngpu_tracker = MemTracker()         # define a GPU tracker\n\ngpu_tracker.track()                     # run function between the code line where uses GPU\ncnn = models.vgg19(pretrained=True).features.to(device).eval()\ngpu_tracker.track()                     # run function between the code line where uses GPU\n\ndummy_tensor_1 = torch.randn(30, 3, 512, 512).float().to(device)  # 30*3*512*512*4/1024/1024 = 90.00M\ndummy_tensor_2 = torch.randn(40, 3, 512, 512).float().to(device)  # 40*3*512*512*4/1024/1024 = 120.00M\ndummy_tensor_3 = torch.randn(60, 3, 512, 512).float().to(device)  # 60*3*512*512*4/1024/1024 = 180.00M\n\ngpu_tracker.track()\n\ndummy_tensor_4 = torch.randn(120, 3, 512, 512).float().to(device)  # 120*3*512*512*4/1024/1024 = 360.00M\ndummy_tensor_5 = torch.randn(80, 3, 512, 512).float().to(device)  # 80*3*512*512*4/1024/1024 = 240.00M\n\ngpu_tracker.track()\n\ndummy_tensor_4 = dummy_tensor_4.cpu()\ndummy_tensor_2 = dummy_tensor_2.cpu()\ngpu_tracker.clear_cache() # or torch.cuda.empty_cache()\n\ngpu_tracker.track()\n```\nThis will output a ``.txt`` to current dir and the content of output is above(print content).\n\n# FAQs\n\n1. Why Total Tensor Used Memory is much smaller than Total Allocated Memory?\n\n* Total Allocated Memory is the peak of the memory usage. When you delete some tensors, PyTorch will not release the space to the device, until you call ``gpu_tracker.clear_cache()`` like the example script.\n\n* The cuda kernel will take some space. See https://github.com/pytorch/pytorch/issues/12873\n\n2. Why does Total Allocated Memory stay unchanged?\n\n* See Q1.\n\n3. I deleted some tensors. Why are they not deleted in tracker's output?\n\n* Make sure that you have released all the references to the tensor object. Then you can call \"import gc; gc.collect()\" and tell python to collect the unreferenced tensor.\n\n# REFERENCE\nPart of the code is referenced from:\n\nhttp://jacobkimmel.github.io/pytorch_estimating_model_size/ \nhttps://gist.github.com/MInner/8968b3b120c95d3f50b8a22a74bf66bc\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOldpan%2FPytorch-Memory-Utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FOldpan%2FPytorch-Memory-Utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOldpan%2FPytorch-Memory-Utils/lists"}