{"id":13572991,"url":"https://github.com/facebookresearch/fairscale","last_synced_at":"2025-05-13T19:17:56.378Z","repository":{"id":36993932,"uuid":"277899703","full_name":"facebookresearch/fairscale","owner":"facebookresearch","description":"PyTorch extensions for high performance and large scale training.","archived":false,"fork":false,"pushed_at":"2025-04-26T18:05:24.000Z","size":4685,"stargazers_count":3307,"open_issues_count":105,"forks_count":289,"subscribers_count":49,"default_branch":"main","last_synced_at":"2025-04-28T10:55:09.466Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facebookresearch.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-07-07T19:02:01.000Z","updated_at":"2025-04-28T09:44:26.000Z","dependencies_parsed_at":"2023-01-17T12:45:22.305Z","dependency_job_id":"b3497f1f-0563-4b82-96a0-3262cc3cb926","html_url":"https://github.com/facebookresearch/fairscale","commit_stats":{"total_commits":704,"total_committers":77,"mean_commits":9.142857142857142,"dds":0.7627840909090909,"last_synced_commit":"5f484b3545f27eddb19d970fbe1d361b9c5f2b07"},"previous_names":[],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Ffairscale","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Ffairscale/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Ffairscale/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Ffairscale/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facebookresearch","download_url":"https://codeload.github.com/facebookresearch/fairscale/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254010830,"owners_count":21999004,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:00:26.023Z","updated_at":"2025-05-13T19:17:56.344Z","avatar_url":"https://github.com/facebookresearch.png","language":"Python","readme":"![FairScale Logo](./docs/source/_static/img/fairscale-logo.png)\n\n[![Support Ukraine](https://img.shields.io/badge/Support-Ukraine-FFD500?style=flat\u0026labelColor=005BBB)](https://opensource.facebook.com/support-ukraine)\n![PyPI](https://img.shields.io/pypi/v/fairscale)\n[![Documentation Status](https://readthedocs.org/projects/fairscale/badge/?version=latest)](https://fairscale.readthedocs.io/en/latest/?badge=latest)\n[![CircleCI](https://circleci.com/gh/facebookresearch/fairscale.svg?style=shield)](https://app.circleci.com/pipelines/github/facebookresearch/fairscale/) ![PyPI - License](https://img.shields.io/pypi/l/fairscale) [![Downloads](https://pepy.tech/badge/fairscale)](https://pepy.tech/project/fairscale) [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/facebookresearch/fairscale/blob/main/CONTRIBUTING.md)\n--------------------------------------------------------------------------------\n\n## Description\nFairScale is a PyTorch extension library for high performance and large scale training.\nThis library extends basic PyTorch capabilities while adding new SOTA scaling techniques.\nFairScale makes available the latest distributed training techniques in the form of composable\nmodules and easy to use APIs. These APIs are a fundamental part of a researcher's toolbox as\nthey attempt to scale models with limited resources.\n\nFairScale was designed with the following values in mind:\n\n* **Usability** -  Users should be able to understand and use FairScale APIs with minimum cognitive overload.\n\n* **Modularity** - Users should be able to combine multiple FairScale APIs as part of their training loop seamlessly.\n\n* **Performance** - FairScale APIs provide the best performance in terms of scaling and efficiency.\n\n## Watch Introductory Video\n\n[![Explain Like I’m 5: FairScale](https://img.youtube.com/vi/oDt7ebOwWIc/0.jpg)](https://www.youtube.com/watch?v=oDt7ebOwWIc)\n\n## Installation\n\nTo install FairScale, please see the following [instructions](https://github.com/facebookresearch/fairscale/blob/main/docs/source/installation_instructions.rst).\nYou should be able to install a package with pip or conda, or build directly from source.\n\n## Getting Started\nThe full [documentation](https://fairscale.readthedocs.io/) contains instructions for getting started, deep dives and tutorials about the various FairScale APIs.\n\n## FSDP\n\nFullyShardedDataParallel (FSDP) is the recommended method for scaling to large NN models.\nThis library has been [upstreamed to PyTorch](https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/).\nThe version of FSDP here is for historical references as well as for experimenting with\nnew and crazy ideas in research of scaling techniques. Please see the following blog\nfor [how to use FairScale FSDP and how does it work](https://engineering.fb.com/2021/07/15/open-source/fsdp/).\n\n## Testing\n\nWe use circleci to test FairScale with the following PyTorch versions (with CUDA 11.2):\n* the latest stable release (e.g. 1.10.0)\n* the latest LTS release (e.g. 1.8.1)\n* a recent nightly release (e.g. 1.11.0.dev20211101+cu111)\n\nPlease create an [issue](https://github.com/facebookresearch/fairscale/issues) if you are having trouble with installation.\n\n## Contributors\n\nWe welcome contributions! Please see the [CONTRIBUTING](CONTRIBUTING.md) instructions for how you can contribute to FairScale.\n\n## License\n\nFairScale is licensed under the [BSD-3-Clause License](LICENSE).\n\nfairscale.nn.pipe is forked from [torchgpipe](https://github.com/kakaobrain/torchgpipe), Copyright 2019, Kakao Brain, licensed under [Apache License](http://www.apache.org/licenses/LICENSE-2.0).\n\nfairscale.nn.model_parallel is forked from [Megatron-LM](https://github.com/NVIDIA/Megatron-LM), Copyright 2020, NVIDIA CORPORATION, licensed under [Apache License](http://www.apache.org/licenses/LICENSE-2.0).\n\nfairscale.optim.adascale is forked from [AdaptDL](https://github.com/petuum/adaptdl), Copyright 2020, Petuum, Inc., licensed under [Apache License](http://www.apache.org/licenses/LICENSE-2.0).\n\nfairscale.nn.misc.flatten_params_wrapper is forked from [PyTorch-Reparam-Module](https://github.com/SsnL/PyTorch-Reparam-Module), Copyright 2018, Tongzhou Wang, licensed under [MIT License](https://github.com/SsnL/PyTorch-Reparam-Module/blob/master/LICENSE).\n\n\n## Citing FairScale\n\nIf you use FairScale in your publication, please cite it by using the following BibTeX entry.\n\n```BibTeX\n@Misc{FairScale2021,\n  author =       {{FairScale authors}},\n  title =        {FairScale:  A general purpose modular PyTorch library for high performance and large scale training},\n  howpublished = {\\url{https://github.com/facebookresearch/fairscale}},\n  year =         {2021}\n}\n```\n","funding_links":[],"categories":["Toolbox","Python","分布式机器学习","🌗 Model Scalability","Deep Learning Framework","LLM Pre-Training","PyTorch Tools, Libraries, and Frameworks","Open Source Projects"],"sub_categories":["Libraries","Deployment \u0026 Distribution","Memory Efficiency"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2Ffairscale","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacebookresearch%2Ffairscale","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2Ffairscale/lists"}