{"id":13654269,"url":"https://github.com/paperswithcode/torchbench","last_synced_at":"2025-04-06T10:10:34.794Z","repository":{"id":55600178,"uuid":"198813889","full_name":"paperswithcode/torchbench","owner":"paperswithcode","description":"Easily benchmark machine learning models in PyTorch","archived":false,"fork":false,"pushed_at":"2024-03-20T15:45:34.000Z","size":2990,"stargazers_count":149,"open_issues_count":5,"forks_count":20,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-03-30T09:05:51.418Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paperswithcode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-25T10:54:15.000Z","updated_at":"2025-02-12T16:48:41.000Z","dependencies_parsed_at":"2024-08-02T02:10:53.235Z","dependency_job_id":"6b4412cd-f152-475f-9bf8-bc9ed3340afc","html_url":"https://github.com/paperswithcode/torchbench","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paperswithcode%2Ftorchbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paperswithcode%2Ftorchbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paperswithcode%2Ftorchbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paperswithcode%2Ftorchbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paperswithcode","download_url":"https://codeload.github.com/paperswithcode/torchbench/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247464220,"owners_count":20942970,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T02:01:25.981Z","updated_at":"2025-04-06T10:10:34.752Z","avatar_url":"https://github.com/paperswithcode.png","language":"Python","funding_links":[],"categories":["Tools","Python"],"sub_categories":["Simulation Frameworks"],"readme":"\u003cp align=\"center\"\u003e\u003cimg width=500 src=\"/docs/docs/img/torchbench.png\"\u003e\u003c/p\u003e\n\n--------------------------------------------------------------------------------\n\n[![PyPI version](https://badge.fury.io/py/torchbench.svg)](https://badge.fury.io/py/torchbench) [![Docs](https://img.shields.io/badge/Documentation-Here-\u003cCOLOR\u003e.svg)](https://paperswithcode.github.io/torchbench/)\n\n`torchbench` is a library that contains a collection of deep learning benchmarks you can use to benchmark your models, optimized for the PyTorch framework. It can be used in conjunction with the [sotabench](https://www.sotabench.com) service to record results for models, so the community can compare model performance on different tasks, as well as a continuous integration style service for your repository to benchmark your models on each commit.\n\n## Benchmarks Supported\n\n- [ImageNet](https://paperswithcode.github.io/torchbench/imagenet/) (Image Classification)\n- [COCO](https://paperswithcode.github.io/torchbench/coco/) (Object Detection) - *partial support*\n- [PASCAL VOC 2012](https://paperswithcode.github.io/torchbench/pascalvoc/) (Semantic Segmentation) - *partial support*\n\nPRs welcome for further benchmarks! \n\n## Installation\n\nRequires Python 3.6+. \n\n```bash\npip install torchbench\n```\n\n## Get Benching! 🏋️\n\nYou should read the [full documentation here](https://paperswithcode.github.io/torchbench/index.html), which contains guidance on getting started and connecting to [sotabench](https://www.sotabench.com).\n\nThe API is optimized for PyTorch implementations. For example, if you wanted to benchmark a [torchvision](https://github.com/pytorch/vision) model for ImageNet, you would write a `sotabench.py` file like this:\n\n```python\nfrom torchbench.image_classification import ImageNet\nfrom torchvision.models.resnet import resnext101_32x8d\nimport torchvision.transforms as transforms\nimport PIL\n\n# Define the transforms need to convert ImageNet data to expected model input\nnormalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], \n    std=[0.229, 0.224, 0.225])\ninput_transform = transforms.Compose([\n    transforms.Resize(256, PIL.Image.BICUBIC),\n    transforms.CenterCrop(224),\n    transforms.ToTensor(),\n    normalize,\n])\n\n# Run the benchmark\nImageNet.benchmark(\n    model=resnext101_32x8d(pretrained=True),\n    paper_model_name='ResNeXt-101-32x8d',\n    paper_arxiv_id='1611.05431',\n    input_transform=input_transform,\n    batch_size=256,\n    num_gpu=1\n)\n```\n\nSotabench will run this on each commit and record the results. For other tasks, such as object detection and semantic segmentation, implementations are much less standardized than for image classification. It is therefore recommended you use [sotabencheval](https://github.com/paperswithcode/sotabench-eval/) for these tasks - although there are experimental benchmarks for [COCO](https://paperswithcode.github.io/torchbench/coco/) and [PASCAL VOC](https://paperswithcode.github.io/torchbench/pascalvoc/).\n\n## Contributing\n\nAll contributions welcome!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaperswithcode%2Ftorchbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaperswithcode%2Ftorchbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaperswithcode%2Ftorchbench/lists"}