Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/facebookresearch/fairseq2

FAIR Sequence Modeling Toolkit 2
https://github.com/facebookresearch/fairseq2

artificial-intelligence deep-learning machine-learning python pytorch

Last synced: about 6 hours ago
JSON representation

FAIR Sequence Modeling Toolkit 2

Awesome Lists containing this project

README 

        


  





# fairseq2: FAIR Sequence Modeling Toolkit 2



[![Nightly](https://github.com/facebookresearch/fairseq2/actions/workflows/nightly.yaml/badge.svg)](https://github.com/facebookresearch/fairseq2/actions/workflows/nightly.yaml)

[![PyPI version](https://img.shields.io/pypi/v/fairseq2)](https://pypi.org/project/fairseq2/)

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)



**Documentation: [Stable](https://facebookresearch.github.io/fairseq2/stable), [Nightly](https://facebookresearch.github.io/fairseq2/nightly)** | **Install: [Linux](#installing-on-linux), [macOS](#installing-on-macos), [Windows](#installing-on-windows), [From Source](INSTALL_FROM_SOURCE.md)** | **Contribute: [Guidelines](CONTRIBUTING.md)**



fairseq2 is a sequence modeling toolkit that allows researchers and developers

to train custom models for translation, summarization, language modeling, and

other content generation tasks. It is also the successor of

[fairseq](https://github.com/facebookresearch/fairseq).



## Getting Started

Coming soon...



For recent changes, you can check out our [changelog](CHANGELOG.md).



## Models

As of today, the following models are available in fairseq2:



 * [LLaMA](src/fairseq2/models/llama)

 * [LLaMA 2](src/fairseq2/models/llama)

 * [LLaMA 3](src/fairseq2/models/llama)

 * [LLaMA 3.1](src/fairseq2/models/llama)

 * [Mistral 7B](src/fairseq2/mistral)

 * [NLLB-200](src/fairseq2/models/nllb)

 * [S2T Transformer + Conformer](src/fairseq2/models/s2t_transformer)

 * [w2v-BERT](src/fairseq2/models/w2vbert)

 * [wav2vec 2.0](src/fairseq2/models/wav2vec2)

 * [wav2vec 2.0 ASR](src/fairseq2/models/wav2vec2/asr)



fairseq2 is also used by various external projects such as:



 * [Seamless Communication](https://github.com/facebookresearch/seamless_communication)

 * [SONAR](https://github.com/facebookresearch/SONAR)



## Installing on Linux



### System Dependencies

fairseq2 depends on [libsndfile](https://github.com/libsndfile/libsndfile),

which can be installed via the system package manager on most Linux

distributions. For Ubuntu-based systems, run:



```sh

sudo apt install libsndfile1

```



Similarly, on Fedora, run:



```sh

sudo dnf install libsndfile

```



For other Linux distributions, please consult its documentation on how to

install packages.



### pip

To install fairseq2 on Linux x86-64, run:



```sh

pip install fairseq2

```



This command will install a version of fairseq2 that is compatible with PyTorch

hosted on PyPI.



At this time, we do not offer a pre-built package for ARM-based systems such as

Raspberry PI or NVIDIA Jetson. Please refer to

[Install From Source](INSTALL_FROM_SOURCE.md) to learn how to build and install

fairseq2 on those systems.



### Variants

Besides PyPI, fairseq2 also has pre-built packages available for different

PyTorch and CUDA versions hosted on FAIR's package repository. The following

matrix shows the supported combinations.



  

    fairseq2

    PyTorch

    Python

    Variant*

    Arch

  

  

    

      HEAD

      2.4.0

      >=3.10, <=3.12

      cpu, cu118, cu121

      x86_64

    

    

      2.3.0, 2.3.1

      >=3.10, <=3.12

      cpu, cu118, cu121

      x86_64

    

    

      2.2.0, 2.2.1, 2.2.2

      >=3.10, <=3.12

      cpu, cu118, cu121

      x86_64

    

    

      0.2.0

      2.1.1

      >=3.8, <=3.11

      cpu, cu118, cu121

      x86_64

    

    

      2.0.1

      >=3.8, <=3.11

      cpu, cu117, cu118

      x86_64

    

    

      1.13.1

      >=3.8, <=3.10

      cpu, cu116

      x86_64

    

  



*\* cuXYZ refers to CUDA XY.Z (e.g. cu118 means CUDA 11.8)*



To install a specific combination, first follow the installation instructions on

[pytorch.org](https://pytorch.org/get-started/locally) for the desired PyTorch

version, and then use the following command (shown for PyTorch `2.4.0` and

variant `cu121`):



```sh

pip install fairseq2\

  --extra-index-url https://fair.pkg.atmeta.com/fairseq2/whl/pt2.4.0/cu121

```



> [!WARNING]

> fairseq2 relies on the C++ API of PyTorch which has no API/ABI compatibility

> between releases. This means **you have to install the fairseq2 variant that

> exactly matches your PyTorch version**. Otherwise, you might experience issues

> like immediate process crashes or spurious segfaults. For the same reason, if

> you upgrade your PyTorch version, you must also upgrade your fairseq2

> installation.



### Nightlies

For Linux, we also host nightly builds on FAIR's package repository. The

supported variants are identical to the ones listed in *Variants* above. Once

you have installed the desired PyTorch version, you can use the following

command to install the corresponding nightly package  (shown for PyTorch `2.4.0`

and variant `cu121`):



```sh

pip install fairseq2\

  --pre --extra-index-url https://fair.pkg.atmeta.com/fairseq2/whl/nightly/pt2.4.0/cu121

```



## Installing on macOS



### System Dependencies

fairseq2 depends on [libsndfile](https://github.com/libsndfile/libsndfile),

which can be installed via Homebrew:



```sh

brew install libsndfile

```



### pip

To install fairseq2 on ARM64-based (i.e. Apple silicon) Mac computers, run:



```sh

pip install fairseq2

```



This command will install a version of fairseq2 that is compatible with PyTorch

hosted on PyPI.



At this time, we do not offer a pre-built package for Intel-based Mac computers.

Please refer to [Install From Source](INSTALL_FROM_SOURCE.md) to learn how to

build and install fairseq2 on Intel machines.



### Variants

Besides PyPI, fairseq2 also has pre-built packages available for different

PyTorch versions hosted on FAIR's package repository. The following matrix shows

the supported combinations.



  

    fairseq2

    PyTorch

    Python

    Arch

  

  

    

      HEAD

      2.4.0

      >=3.9, <=3.12

      arm64

    

  



To install a specific combination, first follow the installation instructions on

[pytorch.org](https://pytorch.org/get-started/locally) for the desired PyTorch

version, and then use the following command (shown for PyTorch `2.4.0`):



```sh

pip install fairseq2\

  --extra-index-url https://fair.pkg.atmeta.com/fairseq2/whl/pt2.4.0/cpu

```



> [!WARNING]

> fairseq2 relies on the C++ API of PyTorch which has no API/ABI compatibility

> between releases. This means **you have to install the fairseq2 variant that

> exactly matches your PyTorch version**. Otherwise, you might experience issues

> like immediate process crashes or spurious segfaults. For the same reason, if

> you upgrade your PyTorch version, you must also upgrade your fairseq2

> installation.



### Nightlies

For macOS, we also host nightly builds on FAIR's package repository. The

supported variants are identical to the ones listed in *Variants* above. Once

you have installed the desired PyTorch version, you can use the following

command to install the corresponding nightly package  (shown for PyTorch `2.4.0`):



```sh

pip install fairseq2\

  --pre --extra-index-url https://fair.pkg.atmeta.com/fairseq2/whl/nightly/pt2.4.0/cpu

```



## Installing on Windows

fairseq2 does not have native support for Windows and there are no plans to

support it in the foreseeable future. However, you can use fairseq2 via the

[Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/about)

(a.k.a. WSL) along with full CUDA support introduced in WSL 2. Please follow the

instructions in the [Installing on Linux](#installing-on-linux) section for a

WSL-based installation.



## Installing from Source

See [here](INSTALL_FROM_SOURCE.md).



## Contributing

We always welcome contributions to fairseq2! Please refer to

[Contribution Guidelines](CONTRIBUTING.md) to learn how to format, test, and

submit your work.



## Citing fairseq2

If you use fairseq2 in your research and wish to refer to it, please use the

following BibTeX entry.



```

@software{balioglu2023fairseq2,

  author = {Can Balioglu},

  title = {fairseq2},

  url = {http://github.com/facebookresearch/fairseq2},

  year = {2023},

}

```



## License

This project is MIT licensed, as found in the [LICENSE](LICENSE) file.