{"id":13869722,"url":"https://github.com/Chris-hughes10/pytorch-accelerated","last_synced_at":"2025-07-15T18:31:51.746Z","repository":{"id":37100799,"uuid":"416779882","full_name":"Chris-hughes10/pytorch-accelerated","owner":"Chris-hughes10","description":"A lightweight library designed to accelerate the process of training PyTorch models by providing a minimal, but extensible training loop which is flexible enough to handle the majority of use cases, and capable of utilizing different hardware options with no code changes required. Docs: https://pytorch-accelerated.readthedocs.io/en/latest/ ","archived":false,"fork":false,"pushed_at":"2025-06-10T15:25:59.000Z","size":252,"stargazers_count":187,"open_issues_count":1,"forks_count":19,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-12T14:57:43.925Z","etag":null,"topics":["deep-learning","huggingface-accelerate","huggingface-examples","pytorch","pytorch-tutorial","timm-examples","training"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Chris-hughes10.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-10-13T14:36:11.000Z","updated_at":"2025-07-10T11:13:39.000Z","dependencies_parsed_at":"2024-01-16T07:22:54.755Z","dependency_job_id":"31a4bbe0-94e5-42fc-a8d4-9f91f129a568","html_url":"https://github.com/Chris-hughes10/pytorch-accelerated","commit_stats":{"total_commits":109,"total_committers":4,"mean_commits":27.25,"dds":0.03669724770642202,"last_synced_commit":"a3f025fe2c36105d0f1986e4a533862dd47d3c3c"},"previous_names":[],"tags_count":44,"template":false,"template_full_name":null,"purl":"pkg:github/Chris-hughes10/pytorch-accelerated","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Chris-hughes10%2Fpytorch-accelerated","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Chris-hughes10%2Fpytorch-accelerated/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Chris-hughes10%2Fpytorch-accelerated/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Chris-hughes10%2Fpytorch-accelerated/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Chris-hughes10","download_url":"https://codeload.github.com/Chris-hughes10/pytorch-accelerated/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Chris-hughes10%2Fpytorch-accelerated/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265071750,"owners_count":23706843,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","huggingface-accelerate","huggingface-examples","pytorch","pytorch-tutorial","timm-examples","training"],"created_at":"2024-08-05T20:01:13.700Z","updated_at":"2025-07-15T18:31:51.458Z","avatar_url":"https://github.com/Chris-hughes10.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# pytorch-accelerated\n\n`pytorch-accelerated` is a lightweight library designed to accelerate the process of training PyTorch models\n by providing a minimal, but extensible training loop - encapsulated in a single `Trainer` \nobject - which is flexible enough to handle the majority of use cases, and capable of utilizing different hardware\n options with no code changes required.\n \n`pytorch-accelerated` offers a streamlined feature set, and places a huge emphasis on **simplicity** and **transparency**,\nto enable users to understand exactly what is going on under the hood, but without having to write and maintain the boilerplate themselves!\n   \nThe key features are:\n- A simple and contained, but easily customisable, training loop, which should work out of the box in straightforward cases;\n behaviour can be customised using inheritance and/or callbacks.\n- Handles device placement, mixed-precision, DeepSpeed integration, multi-GPU and distributed training with no code changes.\n- Uses pure PyTorch components, with no additional modifications or wrappers, and easily interoperates\n with other popular libraries such as [timm](https://github.com/rwightman/pytorch-image-models), \n [transformers](https://huggingface.co/transformers/) and [torchmetrics](https://torchmetrics.readthedocs.io/en/latest/).\n- A small, streamlined API ensures that there is a minimal learning curve for existing PyTorch users.\n\nSignificant effort has been taken to ensure that every part of the library - both internal and external components - is as clear and simple as possible, \nmaking it easy to customise, debug and understand exactly what is going on behind the scenes at each step; most of the \nbehaviour of the trainer is contained in a single class! \nIn the spirit of Python, nothing is hidden and everything is accessible.\n\n`pytorch-accelerated` is proudly and transparently built on top of \n[Hugging Face Accelerate](https://github.com/huggingface/accelerate), which is responsible for the \nmovement of data between devices and launching of training configurations. When customizing the trainer, or launching\ntraining, users are encouraged to consult the [Accelerate documentation](https://huggingface.co/docs/accelerate/) \nto understand all available options; Accelerate provides convenient functions for operations such gathering tensors \nand gradient clipping, usage of which can be seen in the `pytorch-accelerated` \n[examples](https://github.com/Chris-hughes10/pytorch-accelerated/tree/main/examples) folder! \n\nTo learn more about the motivations behind this library, along with a detailed getting started guide, check out [this blog post](https://medium.com/@chris.p.hughes10/introducing-pytorch-accelerated-6ba99530608c?source=friends_link\u0026sk=868c2d2ec5229fdea42877c0bf82b968).\n\n## Installation\n\n`pytorch-accelerated` can be installed from pip using the following command:\n```\npip install pytorch-accelerated\n```\n\nTo make the package as slim as possible, the packages required to run the examples are not included by default. To include these packages, you can use the following command:\n```\npip install pytorch-accelerated[examples]\n```\n\n## Quickstart\n\nTo get started, simply import and use the pytorch-accelerated `Trainer` ,as demonstrated in the following snippet,\nand then launch training using the \n[accelerate CLI](https://huggingface.co/docs/accelerate/quicktour.html#launching-your-distributed-script)\ndescribed below.\n\n```python\n# examples/core/train_mnist.py\nimport os\n\nfrom torch import nn, optim\nfrom torch.utils.data import random_split\nfrom torchvision import transforms\nfrom torchvision.datasets import MNIST\n\nfrom pytorch_accelerated import Trainer\n\nclass MNISTModel(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.main = nn.Sequential(\n            nn.Linear(in_features=784, out_features=128),\n            nn.ReLU(),\n            nn.Linear(in_features=128, out_features=64),\n            nn.ReLU(),\n            nn.Linear(in_features=64, out_features=10),\n        )\n\n    def forward(self, input):\n        return self.main(input.view(input.shape[0], -1))\n\ndef main():\n    dataset = MNIST(os.getcwd(), download=True, transform=transforms.ToTensor())\n    train_dataset, validation_dataset, test_dataset = random_split(dataset, [50000, 5000, 5000])\n    model = MNISTModel()\n    optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)\n    loss_func = nn.CrossEntropyLoss()\n\n    trainer = Trainer(\n            model,\n            loss_func=loss_func,\n            optimizer=optimizer,\n    )\n\n    trainer.train(\n        train_dataset=train_dataset,\n        eval_dataset=validation_dataset,\n        num_epochs=8,\n        per_device_batch_size=32,\n    )\n\n    trainer.evaluate(\n        dataset=test_dataset,\n        per_device_batch_size=64,\n    )\n    \nif __name__ == \"__main__\":\n    main()\n```\n\nTo launch training using the [accelerate CLI](https://huggingface.co/docs/accelerate/quicktour.html#launching-your-distributed-script)\n, on your machine(s), run:\n\n` accelerate config --config_file accelerate_config.yaml`\n\nand answer the questions asked. This will generate a config file that will be used to properly set the default options when doing\n\n`accelerate launch --config_file accelerate_config.yaml train.py [--training-args]`\n\n*Note*: Using the [accelerate CLI](https://huggingface.co/docs/accelerate/quicktour.html#launching-your-distributed-script) is completely optional, training can also be launched in the usual way using:\n\n`python train.py` / `python -m torch.distributed ...`\n\ndepending on your infrastructure configuration, for users who would like to maintain a more fine-grained control \nover the launch command.\n\nMore complex training examples can be seen in the examples folder \n[here](https://github.com/Chris-hughes10/pytorch-accelerated/tree/main/examples). \n\nAlternatively, if you would rather understand the core concepts first, this can be found in the [documentation](https://pytorch-accelerated.readthedocs.io/en/latest/).\n\n## Usage\n\n### Who is pytorch-accelerated aimed at?\n\n- Users that are familiar with PyTorch but would like to avoid having to write the common training loop boilerplate\nto focus on the interesting parts of the training loop.\n- Users who like, and are comfortable with, selecting and creating their own models, loss functions, optimizers and datasets.\n- Users who value a simple and streamlined feature set, where the behaviour is easy to debug, understand, and reason about!\n\n### When shouldn't I use pytorch-accelerated?\n\n- If you are looking for an end-to-end solution, encompassing everything from loading data to inference,\n  which helps you to select a model, optimizer or loss function, you would probably be better suited to\n  [fastai](https://github.com/fastai/fastai). `pytorch-accelerated` focuses only on the training process, with all other\n  concerns being left to the responsibility of the user.\n- If you would like to write the entire training loop yourself, just without all of the device management headaches, \nyou would probably be best suited to using [Accelerate](https://github.com/huggingface/accelerate) directly! Whilst it\nis possible to customize every part of the `Trainer`, the training loop is fundamentally broken up into a number of \ndifferent methods that you would have to override. But, before you go, is writing those `for` loops really important \nenough to warrant starting from scratch *again* 😉.\n- If you are working on a custom, highly complex, use case which does not fit the patterns of usual training loops\nand want to squeeze out every last bit of performance on your chosen hardware, you are probably best off sticking\n with vanilla PyTorch; any high-level API becomes an overhead in highly specialized cases!\n\n\n## Acknowledgements\n\nMany aspects behind the design and features of `pytorch-accelerated` were greatly inspired by a number of excellent \nlibraries and frameworks such as [fastai](https://github.com/fastai/fastai), [timm](https://github.com/rwightman/pytorch-image-models), \n[PyTorch-lightning](https://github.com/PyTorchLightning/pytorch-lightning) and [Hugging Face Accelerate](https://github.com/huggingface/accelerate). Each of these tools \nhave made an enormous impact on both this library and the machine learning community, and their influence can not be \nstated enough!\n\n`pytorch-accelerated` has taken only inspiration from these tools, and all of the functionality contained has been implemented\n from scratch in a way that benefits this library. The only exceptions to this are some of the scripts in the \n [examples](https://github.com/Chris-hughes10/pytorch-accelerated/tree/main/examples)\n folder in which existing resources were taken and modified in order to showcase the features of `pytorch-accelerated`;\n these cases are clearly marked, with acknowledgement being given to the original authors.\n \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FChris-hughes10%2Fpytorch-accelerated","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FChris-hughes10%2Fpytorch-accelerated","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FChris-hughes10%2Fpytorch-accelerated/lists"}