{"id":13691866,"url":"https://github.com/facebookresearch/fairseq2","last_synced_at":"2025-05-14T02:04:58.312Z","repository":{"id":190420962,"uuid":"581166973","full_name":"facebookresearch/fairseq2","owner":"facebookresearch","description":"FAIR Sequence Modeling Toolkit 2","archived":false,"fork":false,"pushed_at":"2025-05-12T18:37:55.000Z","size":27287,"stargazers_count":898,"open_issues_count":99,"forks_count":106,"subscribers_count":25,"default_branch":"main","last_synced_at":"2025-05-12T18:49:42.572Z","etag":null,"topics":["artificial-intelligence","deep-learning","machine-learning","python","pytorch"],"latest_commit_sha":null,"homepage":"https://facebookresearch.github.io/fairseq2/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facebookresearch.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-12-22T12:59:00.000Z","updated_at":"2025-05-09T09:30:27.000Z","dependencies_parsed_at":"2024-03-13T19:45:30.100Z","dependency_job_id":"77cf1eea-4fcc-4926-8e08-c849eb43f047","html_url":"https://github.com/facebookresearch/fairseq2","commit_stats":{"total_commits":1047,"total_committers":40,"mean_commits":26.175,"dds":"0.15759312320916907","last_synced_commit":"2d7a28197bda71c2b398c7fb9ff7f1342fa3ae0a"},"previous_names":["facebookresearch/fairseq2"],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Ffairseq2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Ffairseq2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Ffairseq2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Ffairseq2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facebookresearch","download_url":"https://codeload.github.com/facebookresearch/fairseq2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253805843,"owners_count":21967053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","deep-learning","machine-learning","python","pytorch"],"created_at":"2024-08-02T17:00:51.138Z","updated_at":"2025-05-14T02:04:58.297Z","avatar_url":"https://github.com/facebookresearch.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"doc/source/_static/img/logo.svg\" width=\"150\"\u003e\u003cbr /\u003e\n\u003c/p\u003e\n\n# fairseq2: FAIR Sequence Modeling Toolkit 2\n\n[![Nightly](https://github.com/facebookresearch/fairseq2/actions/workflows/nightly.yaml/badge.svg)](https://github.com/facebookresearch/fairseq2/actions/workflows/nightly.yaml)\n[![PyPI version](https://img.shields.io/pypi/v/fairseq2)](https://pypi.org/project/fairseq2/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n**Documentation: [Stable](https://facebookresearch.github.io/fairseq2/stable), [Nightly](https://facebookresearch.github.io/fairseq2/nightly)** | **Install: [Linux](#installing-on-linux), [macOS](#installing-on-macos), [Windows](#installing-on-windows), [From Source](INSTALL_FROM_SOURCE.md)** | **Contribute: [Guidelines](CONTRIBUTING.md)**\n\nfairseq2 is a sequence modeling toolkit that allows researchers to train custom models for content generation tasks.\n\n### Who uses it?\nMany FAIR teams utilize fairseq2 for a diverse set of projects, ranging from language model preference optimization to pretraining video diffusion models.\n\n### How is fairseq2 different from the original fairseq?\nfairseq2 is a start-from-scratch project that can be considered a reboot of the original [fairseq](https://github.com/facebookresearch/fairseq) to provide a clean, modular API. Notably, it differs from its predecessor in its design philosophy, moving from a monolithic framework to an extensible, much less intrusive architecture allowing researchers to independently own their project code base.\n\n\u003e As fairseq2 is a complete new project rather than an incremental update to the original fairseq, we intentionally avoided labeling it as fairseq version 2, reflecting its distinct and separate identity.\n\n## What's New?\n* February 2025: [Instruction finetuning](https://facebookresearch.github.io/fairseq2/stable/tutorials/end_to_end_fine_tuning.html) and [preference optimization](https://facebookresearch.github.io/fairseq2/stable/tutorials/preference_optimization.html) recipes with support for DPO, CPO, SimPO, and ORPO. Supports tensor parallelism and 70B+ scales.\n\n## Features\n* First-party recipes for language model [instruction finetuning](https://facebookresearch.github.io/fairseq2/stable/tutorials/end_to_end_fine_tuning.html) and [preference optimization](https://facebookresearch.github.io/fairseq2/stable/tutorials/preference_optimization.html)\n* Multi-GPU, multi-node [training](https://facebookresearch.github.io/fairseq2/stable/basics/trainer.html) using DDP, FSDP, and tensor parallelism. Supports 70B+ models.\n* Native support for vLLM along with built-in sampling and beam search sequence generators\n* Extensible with setuptools [extension mechanism](https://facebookresearch.github.io/fairseq2/stable/basics/runtime_extensions.html). Easily register new models, optimizers, lr schedulers, trainer units without forking/branching the library.\n* Modern PyTorch tooling. Uses composability (i.e. torch.compile), PyTorch FSDP, and other relevant features\n* Streaming-based, high throughput [data pipeline API](https://facebookresearch.github.io/fairseq2/stable/basics/data_pipeline.html) written in C++ with support for speech and (soon) video decoding\n* Programmatic [asset cards](https://facebookresearch.github.io/fairseq2/stable/basics/assets.html) for version controlled access to models, datasets, and tokenizers\n* Flexible, but deterministic configuration based on the built-in *structured* API\n\n## Getting Started\nVisit our [documentation website](https://facebookresearch.github.io/fairseq2/stable/) to learn more about fairseq2.\n\n## Models\nAs of today, the following models are available in fairseq2 for use in training and evaluation recipes:\n\n * [LLaMA 1 to 3.3](src/fairseq2/models/llama)\n * [Mistral 7B](src/fairseq2/mistral)\n * [NLLB-200](src/fairseq2/models/nllb)\n * [S2T Transformer + Conformer](src/fairseq2/models/s2t_transformer)\n * [V-JEPA](src/fairseq2/models/jepa)\n * [w2v-BERT](src/fairseq2/models/w2vbert)\n * [wav2vec 2.0](src/fairseq2/models/wav2vec2)\n * [wav2vec 2.0 ASR](src/fairseq2/models/wav2vec2/asr)\n\nfairseq2 is also used by various external projects such as:\n\n * [Seamless Communication](https://github.com/facebookresearch/seamless_communication)\n * [Large Concept Model](https://github.com/facebookresearch/large_concept_model)\n * [SONAR](https://github.com/facebookresearch/SONAR)\n\n\n## Installing on Linux\n\n### System Dependencies\nfairseq2 depends on [libsndfile](https://github.com/libsndfile/libsndfile),\nwhich can be installed via the system package manager on most Linux\ndistributions. For Ubuntu-based systems, run:\n\n```sh\nsudo apt install libsndfile1\n```\n\nSimilarly, on Fedora, run:\n\n```sh\nsudo dnf install libsndfile\n```\n\nFor other Linux distributions, please consult its documentation on how to\ninstall packages.\n\n### pip\nTo install fairseq2 on Linux x86-64, run:\n\n```sh\npip install fairseq2\n```\n\nThis command will install a version of fairseq2 that is compatible with PyTorch\nhosted on PyPI.\n\nAt this time, we do not offer a pre-built package for ARM-based systems such as\nRaspberry PI or NVIDIA Jetson. Please refer to\n[Install From Source](INSTALL_FROM_SOURCE.md) to learn how to build and install\nfairseq2 on those systems.\n\n### Variants\nBesides PyPI, fairseq2 also has pre-built packages available for different\nPyTorch and CUDA versions hosted on FAIR's package repository. The following\nmatrix shows the supported combinations.\n\n\u003ctable\u003e\n  \u003cthead\u003e\n    \u003cth\u003efairseq2\u003c/th\u003e\n    \u003cth\u003ePyTorch\u003c/th\u003e\n    \u003cth\u003ePython\u003c/th\u003e\n    \u003cth\u003eVariant*\u003c/th\u003e\n    \u003cth\u003eArch\u003c/th\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd rowspan=2\u003e\u003ccode\u003eHEAD\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e2.7.0\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e\u0026gt;=3.10\u003c/code\u003e, \u003ccode\u003e\u0026lt;=3.12\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ecpu\u003c/code\u003e, \u003ccode\u003ecu126\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ex86_64\u003c/code\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e2.6.0\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e\u0026gt;=3.10\u003c/code\u003e, \u003ccode\u003e\u0026lt;=3.12\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ecpu\u003c/code\u003e, \u003ccode\u003ecu124\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ex86_64\u003c/code\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd rowspan=3\u003e\u003ccode\u003e0.4\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e2.6.0\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e\u0026gt;=3.10\u003c/code\u003e, \u003ccode\u003e\u0026lt;=3.12\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ecpu\u003c/code\u003e, \u003ccode\u003ecu118\u003c/code\u003e, \u003ccode\u003ecu124\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ex86_64\u003c/code\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e2.5.0\u003c/code\u003e, \u003ccode\u003e2.5.1\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e\u0026gt;=3.10\u003c/code\u003e, \u003ccode\u003e\u0026lt;=3.12\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ecpu\u003c/code\u003e, \u003ccode\u003ecu118\u003c/code\u003e, \u003ccode\u003ecu121\u003c/code\u003e, \u003ccode\u003ecu124\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ex86_64\u003c/code\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e2.4.0\u003c/code\u003e, \u003ccode\u003e2.4.1\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e\u0026gt;=3.10\u003c/code\u003e, \u003ccode\u003e\u0026lt;=3.12\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ecpu\u003c/code\u003e, \u003ccode\u003ecu118\u003c/code\u003e, \u003ccode\u003ecu121\u003c/code\u003e, \u003ccode\u003ecu124\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003ex86_64\u003c/code\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n*\\* cuXYZ refers to CUDA XY.Z (e.g. cu118 means CUDA 11.8)*\n\nTo install a specific combination, first follow the installation instructions on\n[pytorch.org](https://pytorch.org/get-started/locally) for the desired PyTorch\nversion, and then use the following command (shown for PyTorch `2.7.0` and\nvariant `cu124`):\n\n```sh\npip install fairseq2\\\n  --extra-index-url https://fair.pkg.atmeta.com/fairseq2/whl/pt2.7.0/cu126\n```\n\n\u003e [!WARNING]\n\u003e fairseq2 relies on the C++ API of PyTorch which has no API/ABI compatibility\n\u003e between releases. This means **you have to install the fairseq2 variant that\n\u003e exactly matches your PyTorch version**. Otherwise, you might experience issues\n\u003e like immediate process crashes or spurious segfaults. For the same reason, if\n\u003e you upgrade your PyTorch version, you must also upgrade your fairseq2\n\u003e installation.\n\n### Nightlies\nFor Linux, we also host nightly builds on FAIR's package repository. The\nsupported variants are identical to the ones listed in *Variants* above. Once\nyou have installed the desired PyTorch version, you can use the following\ncommand to install the corresponding nightly package  (shown for PyTorch `2.7.0`\nand variant `cu124`):\n\n```sh\npip install fairseq2\\\n  --pre --extra-index-url https://fair.pkg.atmeta.com/fairseq2/whl/nightly/pt2.7.0/cu126\n```\n\n\n## Installing on macOS\n\n### System Dependencies\nfairseq2 depends on [libsndfile](https://github.com/libsndfile/libsndfile),\nwhich can be installed via Homebrew:\n\n```sh\nbrew install libsndfile\n```\n\n### pip\nTo install fairseq2 on ARM64-based (i.e. Apple silicon) Mac computers, run:\n\n```sh\npip install fairseq2\n```\n\nThis command will install a version of fairseq2 that is compatible with PyTorch\nhosted on PyPI.\n\nAt this time, we do not offer a pre-built package for Intel-based Mac computers.\nPlease refer to [Install From Source](INSTALL_FROM_SOURCE.md) to learn how to\nbuild and install fairseq2 on Intel machines.\n\n### Variants\nBesides PyPI, fairseq2 also has pre-built packages available for different\nPyTorch versions hosted on FAIR's package repository. The following matrix shows\nthe supported combinations.\n\n\u003ctable\u003e\n  \u003cthead\u003e\n    \u003cth\u003efairseq2\u003c/th\u003e\n    \u003cth\u003ePyTorch\u003c/th\u003e\n    \u003cth\u003ePython\u003c/th\u003e\n    \u003cth\u003eArch\u003c/th\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003eHEAD\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e2.7.0\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e\u0026gt;=3.10\u003c/code\u003e, \u003ccode\u003e\u0026lt;=3.12\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003earm64\u003c/code\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e0.4\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e2.6.0\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003e\u0026gt;=3.10\u003c/code\u003e, \u003ccode\u003e\u0026lt;=3.12\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003e\u003ccode\u003earm64\u003c/code\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\nTo install a specific combination, first follow the installation instructions on\n[pytorch.org](https://pytorch.org/get-started/locally) for the desired PyTorch\nversion, and then use the following command (shown for PyTorch `2.7.0`):\n\n```sh\npip install fairseq2\\\n  --extra-index-url https://fair.pkg.atmeta.com/fairseq2/whl/pt2.7.0/cpu\n```\n\n\u003e [!WARNING]\n\u003e fairseq2 relies on the C++ API of PyTorch which has no API/ABI compatibility\n\u003e between releases. This means **you have to install the fairseq2 variant that\n\u003e exactly matches your PyTorch version**. Otherwise, you might experience issues\n\u003e like immediate process crashes or spurious segfaults. For the same reason, if\n\u003e you upgrade your PyTorch version, you must also upgrade your fairseq2\n\u003e installation.\n\n### Nightlies\nFor macOS, we also host nightly builds on FAIR's package repository. The\nsupported variants are identical to the ones listed in *Variants* above. Once\nyou have installed the desired PyTorch version, you can use the following\ncommand to install the corresponding nightly package  (shown for PyTorch `2.7.0`):\n\n```sh\npip install fairseq2\\\n  --pre --extra-index-url https://fair.pkg.atmeta.com/fairseq2/whl/nightly/pt2.7.0/cpu\n```\n\n\n## Installing on Windows\nfairseq2 does not have native support for Windows and there are no plans to\nsupport it in the foreseeable future. However, you can use fairseq2 via the\n[Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/about)\n(a.k.a. WSL) along with full CUDA support introduced in WSL 2. Please follow the\ninstructions in the [Installing on Linux](#installing-on-linux) section for a\nWSL-based installation.\n\n\n## Installing from Source\nSee [here](INSTALL_FROM_SOURCE.md).\n\n\n## Contributing\nWe always welcome contributions to fairseq2! Please refer to\n[Contribution Guidelines](CONTRIBUTING.md) to learn how to format, test, and\nsubmit your work.\n\n\n## Citing fairseq2\nIf you use fairseq2 in your research and wish to refer to it, please use the\nfollowing BibTeX entry.\n\n```\n@software{balioglu2023fairseq2,\n  author = {Can Balioglu and Martin Gleize and Artyom Kozhevnikov and Ilia Kulikov and Tuan Tran and Julien Yao},\n  title = {fairseq2},\n  url = {http://github.com/facebookresearch/fairseq2},\n  year = {2023},\n}\n```\n\n\n## License\nThis project is MIT licensed, as found in the [LICENSE](LICENSE) file.\n","funding_links":[],"categories":["General","Open LLM Tools","📋 Contents"],"sub_categories":["🧬 1. Core Frameworks \u0026 Libraries"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2Ffairseq2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacebookresearch%2Ffairseq2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2Ffairseq2/lists"}