https://github.com/intellabs/fastrag

Efficient Retrieval Augmentation and Generation Framework
https://github.com/intellabs/fastrag

benchmark colbert diffusion generative-ai information-retrieval knowledge-graph llm multi-modal nlp question-answering semantic-search sentence-transformers summarization transformers

Last synced: 1 day ago
JSON representation

Efficient Retrieval Augmentation and Generation Framework

• Host: GitHub
• URL: https://github.com/intellabs/fastrag
• Owner: IntelLabs
• License: apache-2.0
• Created: 2023-01-23T16:25:35.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-05-22T15:13:25.000Z (6 months ago)
• Last Synced: 2024-05-22T16:29:22.077Z (6 months ago)
• Topics: benchmark, colbert, diffusion, generative-ai, information-retrieval, knowledge-graph, llm, multi-modal, nlp, question-answering, semantic-search, sentence-transformers, summarization, transformers
• Language: Python
• Homepage:
• Size: 20.3 MB
• Stars: 945
• Watchers: 10
• Forks: 75
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Citation: CITATION.cff
  • Security: SECURITY.md

Awesome Lists containing this project

README

        


    


---



    

Build and explore efficient retrieval-augmented generative models and applications


![PyPI - Version](https://img.shields.io/pypi/v/fastrag)

![PyPI - Downloads](https://img.shields.io/pypi/dm/fastrag)

:round_pushpin: Installation • :rocket: Components • :books: Examples • :red_car: Getting Started • :pill: Demos • :pencil2: Scripts • :bar_chart: Benchmarks



fast**RAG** is a research framework for ***efficient*** and ***optimized*** retrieval augmented generative pipelines,

incorporating state-of-the-art LLMs and Information Retrieval. fastRAG is designed to empower researchers and developers

with a comprehensive tool-set for advancing retrieval augmented generation.

Comments, suggestions, issues and pull-requests are welcomed! :heart:

> [!IMPORTANT]

> Now compatible with Haystack v2+. Please report any possible issues you find.

## :mega: Updates

- **2024-05**: fastRAG V3 is Haystack 2.0 compatible :fire:

- **2023-12**: Gaudi2 and ONNX runtime support; Optimized Embedding models; Multi-modality and Chat demos; [REPLUG](https://arxiv.org/abs/2301.12652) text generation.

- **2023-06**: ColBERT index modification: adding/removing documents; see [IndexUpdater](libs/colbert/colbert/index_updater.py).

- **2023-05**: [RAG with LLM and dynamic prompt synthesis example](examples/rag-prompt-hf.ipynb).

- **2023-04**: Qdrant `DocumentStore` support.

## Key Features

- **Optimized RAG**: Build RAG pipelines with SOTA efficient components for greater compute efficiency.

- **Optimized for Intel Hardware**: Leverage [Intel extensions for PyTorch (IPEX)](https://github.com/intel/intel-extension-for-pytorch), [🤗 Optimum Intel](https://github.com/huggingface/optimum-intel) and [🤗 Optimum-Habana](https://github.com/huggingface/optimum-habana) for *running as optimal as possible* on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators.

- **Customizable**: fastRAG is built using [Haystack](https://github.com/deepset-ai/haystack) and HuggingFace. All of fastRAG's components are 100% Haystack compatible.

## :rocket: Components

For a brief overview of the various unique components in fastRAG refer to the [Components Overview](components.md) page.



  

    LLM Backends

  

  

    Intel Gaudi Accelerators

    Running LLMs on Gaudi 2

  

  

    ONNX Runtime

    Running LLMs with optimized ONNX-runtime

  

  

    OpenVINO

    Running quantized LLMs using OpenVINO

  

  

    Llama-CPP

    Running RAG Pipelines with LLMs on a Llama CPP backend

  

  

    Optimized Components

  

  

    Embedders

    Optimized int8 bi-encoders

  

  

    Rankers

    Optimized/sparse cross-encoders

  

  

    RAG-efficient Components

  

  

    ColBERT

    Token-based late interaction

  

  

    Fusion-in-Decoder (FiD)

    Generative multi-document encoder-decoder

  

  

    REPLUG

    Improved multi-document decoder

  

  

    PLAID

    Incredibly efficient indexing engine

  



## :round_pushpin: Installation

Preliminary requirements:

- **Python** 3.8 or higher.

- **PyTorch** 2.0 or higher.

To set up the software, install from `pip` or clone the project for the bleeding-edge updates. Run the following, preferably in a newly created virtual environment:

```bash

pip install fastrag

```

### Extra Packages

There are additional dependencies that you can install based on your specific usage of fastRAG:

```bash

# Additional engines/components

pip install fastrag[intel]               # Intel optimized backend [Optimum-intel, IPEX]

pip install fastrag[openvino]            # Intel optimized backend using OpenVINO

pip install fastrag[elastic]             # Support for ElasticSearch store

pip install fastrag[qdrant]              # Support for Qdrant store

pip install fastrag[colbert]             # Support for ColBERT+PLAID; requires FAISS

pip install fastrag[faiss-cpu]           # CPU-based Faiss library

pip install fastrag[faiss-gpu]           # GPU-based Faiss library

```

To work with the latest version of fastRAG, you can install it using the following command:

```bash

pip install .

```

### Development tools

```bash

pip install .[dev]

```

## License

The code is licensed under the [Apache 2.0 License](LICENSE).

## Disclaimer

This is not an official Intel product.