{"id":18711123,"url":"https://github.com/rocm/pytorch-micro-benchmarking","last_synced_at":"2025-04-12T11:40:24.194Z","repository":{"id":38208776,"uuid":"297429497","full_name":"ROCm/pytorch-micro-benchmarking","owner":"ROCm","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-25T14:16:55.000Z","size":87,"stargazers_count":20,"open_issues_count":2,"forks_count":11,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-03-26T06:23:25.873Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ROCm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-21T18:39:37.000Z","updated_at":"2025-03-08T06:41:58.000Z","dependencies_parsed_at":"2024-11-06T16:44:30.106Z","dependency_job_id":null,"html_url":"https://github.com/ROCm/pytorch-micro-benchmarking","commit_stats":null,"previous_names":["rocm/pytorch-micro-benchmarking"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Fpytorch-micro-benchmarking","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Fpytorch-micro-benchmarking/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Fpytorch-micro-benchmarking/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Fpytorch-micro-benchmarking/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ROCm","download_url":"https://codeload.github.com/ROCm/pytorch-micro-benchmarking/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248562806,"owners_count":21125150,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T12:37:06.129Z","updated_at":"2025-04-12T11:40:24.188Z","avatar_url":"https://github.com/ROCm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pytorch-micro-benchmarking\nWe supply a small microbenchmarking script for PyTorch training on ROCm.\n\nTo execute:\n`python micro_benchmarking_pytorch.py --network \u003cnetwork name\u003e [--batch-size \u003cbatch size\u003e ] [--iterations \u003cnumber of iterations\u003e] [--fp16 \u003c0 or 1\u003e ] [--distributed_dataparallel] [--device_ids \u003ccomma separated list (no spaces) of GPU indices (0-indexed) to run distributed_dataparallel api on\u003e] `\n\nPossible network names are: `alexnet`, `densenet121`, `inception_v3`, `resnet50`, `resnet101`, `SqueezeNet`, `vgg16` etc.\n\nDefault are 10 training iterations, `fp16` off (i.e., 0), and a batch size of 64.\n\nFor mGPU runs, use one of the following methods.\n- `torchrun`: It will spawn multiple sub-processes for each of the GPUs and adjust `world_size` and `rank` accordingly. `torchrun` also defaults to using distributed dataparallel.\n- `--distributed_dataparallel`: Uses torch.nn.parallel.DistributedDataParallel to run multiple processes/node. However, the script only launches one process per GPU, multiple processes need to be launched manually. See example below.\n  \n_NOTE_: `--distributed_dataparallel` option will be deprecated in the future as this path can be exercised now with `torchrun`.\n_NOTE_: If comparing `--distributed_dataprallel` performance with `torchrun` one, you need to multiply the `--batch-size` with number of nodes in the `torchrun` command. `torchrun` will split the batch size into mini batches that run on each of the nodes. `--distributed_dataparallel` doesn't do that automatically, it run with whatever the user provides.\n\nExamples: \n- for a 1-GPU resnet50 run:\n```\npython3 micro_benchmarking_pytorch.py --network resnet50\n```\n\n- for a 2-GPU run on a single node using `torchrun`:\n```\ntorchrun --nproc-per-node 2 micro_benchmarking_pytorch.py --network resnet50 --batch-size 128\n\n```\n\n- for a 2-GPU run on a single node using `--distributed_dataparallel`:\n```\npython3 micro_benchmarking_pytorch.py --device_ids=0 --network resnet50 --distributed_dataparallel --rank 0 --world-size 2 --dist-backend nccl --dist-url tcp://127.0.0.1:4332 --batch-size 64 \u0026\npython3 micro_benchmarking_pytorch.py --device_ids=1 --network resnet50 --distributed_dataparallel --rank 1 --world-size 2 --dist-backend nccl --dist-url tcp://127.0.0.1:4332 --batch-size 64 \u0026\n```\n\n\nTo run FlopsProfiler (with deepspeed.profiling.flops_profiler imported):\n`python micro_benchmarking_pytorch.py --network resnet50 --amp-opt-level=2 --batch-size=256 --iterations=20 --flops-prof-step 10`\n\n## Performance tuning\nIf performance on a specific card and/or model is found to be lacking, typically some gains can be made by tuning MIOpen. For this, `export MIOPEN_FIND_ENFORCE=3` prior to running the model. This will take some time if untuned configurations are encountered and write to a local performance database. More information on this can be found in the [MIOpen documentation](https://rocm.github.io/MIOpen/doc/html/perfdatabase.html).\n\n## PyTorch 2.0\nAdded the `--compile` option opens up PyTorch 2.0 capabilities, which comes with several options. Here are some notes from upstream: \n```\n    Optimizes given model/function using TorchDynamo and specified backend.\n\n    Args:\n       model (Callable): Module/function to optimize\n       fullgraph (bool): Whether it is ok to break model into several subgraphs\n       dynamic (bool): Use dynamic shape tracing\n       backend (str or Callable): backend to be used\n       mode (str): Can be either \"default\", \"reduce-overhead\" or \"max-autotune\"\n       options (dict): A dictionary of options to pass to the backend.\n       disable (bool): Turn torch.compile() into a no-op for testing\n\n    Example::\n\n        @torch.compile(options={\"matmul-padding\": True}, fullgraph=True)\n        def foo(x):\n            return torch.sin(x) + torch.cos(x)\n```\n\nWith the required `--compile` option, these additional options are now available from the command line with the `--compileContext` flag. Here are a few examples:\n\n```bash\npython micro_benchmarking_pytorch.py --network resnet50 --compile # default run\n```\n\n```bash\npython micro_benchmarking_pytorch.py --network resnet50 --compile --compileContext \"{'mode': 'max-autotune', 'fullgraph': 'True'}\"\n```\n\n```bash\npython micro_benchmarking_pytorch.py --network resnet50 --compile --compileContext \"{'options': {'static-memory': 'True', 'matmul-padding': 'True'}}\"\n```\nNote: you cannot pass the `mode` and `options` options together.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frocm%2Fpytorch-micro-benchmarking","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frocm%2Fpytorch-micro-benchmarking","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frocm%2Fpytorch-micro-benchmarking/lists"}