{"id":13596980,"url":"https://github.com/rapidsai/raft","last_synced_at":"2025-05-14T00:09:34.627Z","repository":{"id":37057465,"uuid":"191779319","full_name":"rapidsai/raft","owner":"rapidsai","description":"RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.","archived":false,"fork":false,"pushed_at":"2025-05-07T05:44:37.000Z","size":15788,"stargazers_count":876,"open_issues_count":433,"forks_count":204,"subscribers_count":25,"default_branch":"branch-25.06","last_synced_at":"2025-05-07T06:19:09.076Z","etag":null,"topics":["anns","building-blocks","clustering","cuda","distance","gpu","information-retrieval","linear-algebra","llm","machine-learning","nearest-neighbors","neighborhood-methods","primitives","random-sampling","solvers","sparse","statistics","vector-search","vector-similarity","vector-store"],"latest_commit_sha":null,"homepage":"https://docs.rapids.ai/api/raft/stable/","language":"Cuda","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rapidsai.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-06-13T14:32:40.000Z","updated_at":"2025-05-06T20:59:16.000Z","dependencies_parsed_at":"2023-09-25T01:10:17.852Z","dependency_job_id":"d5e63138-d1d8-49be-9f0b-9774593da75f","html_url":"https://github.com/rapidsai/raft","commit_stats":null,"previous_names":[],"tags_count":56,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rapidsai%2Fraft","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rapidsai%2Fraft/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rapidsai%2Fraft/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rapidsai%2Fraft/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rapidsai","download_url":"https://codeload.github.com/rapidsai/raft/tar.gz/refs/heads/branch-25.06","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254044181,"owners_count":22005095,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anns","building-blocks","clustering","cuda","distance","gpu","information-retrieval","linear-algebra","llm","machine-learning","nearest-neighbors","neighborhood-methods","primitives","random-sampling","solvers","sparse","statistics","vector-search","vector-similarity","vector-store"],"created_at":"2024-08-01T17:00:17.014Z","updated_at":"2025-05-14T00:09:29.619Z","avatar_url":"https://github.com/rapidsai.png","language":"Cuda","readme":"# \u003cdiv align=\"left\"\u003e\u003cimg src=\"https://rapids.ai/assets/images/rapids_logo.png\" width=\"90px\"/\u003e\u0026nbsp;RAFT: Reusable Accelerated Functions and Tools for Vector Search and More\u003c/div\u003e\n\n\u003e [!IMPORTANT]\n\u003e The vector search and clustering algorithms in RAFT have been formally migrated to a new library dedicated to vector search called [cuVS](https://github.com/rapidsai/cuvs). The headers for the vector search and clustering algorithms in RAFT will remain for a bried period, but will no longer be tested, benchmarked, included in the pre-compiled libraft binary, or otherwise updated after the 24.12 (December 2024) release. We will be removing these headers altogether in a future release. It is strongly suggested to use cuVS for these routines, which include any headers in the `distance`, `neighbors`, `cluster` and `spatial` directories, and use the RAFT versions at your own risk.\n\n![RAFT tech stack](img/raft-tech-stack-vss.png)\n\n\n\n## Contents\n\u003chr\u003e\n\n1. [Useful Resources](#useful-resources)\n2. [What is RAFT?](#what-is-raft)\n2. [Use cases](#use-cases)\n3. [Is RAFT right for me?](#is-raft-right-for-me)\n4. [Getting Started](#getting-started)\n5. [Installing RAFT](#installing)\n6. [Codebase structure and contents](#folder-structure-and-contents)\n7. [Contributing](#contributing)\n8. [References](#references)\n\n\u003chr\u003e\n\n## Useful Resources\n\n- [RAFT Reference Documentation](https://docs.rapids.ai/api/raft/stable/): API Documentation.\n- [RAFT Getting Started](./docs/source/quick_start.md): Getting started with RAFT.\n- [Build and Install RAFT](./docs/source/build.md): Instructions for installing and building RAFT.\n- [RAPIDS Community](https://rapids.ai/community.html): Get help, contribute, and collaborate.\n- [GitHub repository](https://github.com/rapidsai/raft): Download the RAFT source code.\n- [Issue tracker](https://github.com/rapidsai/raft/issues): Report issues or request features.\n\n\n\n## What is RAFT?\n\nRAFT contains fundamental widely-used algorithms and primitives for machine learning and data mining. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.\n\nBy taking a primitives-based approach to algorithm development, RAFT\n- accelerates algorithm construction time\n- reduces the maintenance burden by maximizing reuse across projects, and\n- centralizes core reusable computations, allowing future optimizations to benefit all algorithms that use them.\n\nWhile not exhaustive, the following general categories help summarize the accelerated functions in RAFT:\n#####\n| Category              | Accelerated Functions in RAFT                                                                                                     |\n|-----------------------|-----------------------------------------------------------------------------------------------------------------------------------|\n| **Data Formats**      | sparse \u0026 dense, conversions, data generation                                                                                      |\n| **Dense Operations**  | linear algebra, matrix and vector operations, reductions, slicing, norms, factorization, least squares, svd \u0026 eigenvalue problems |\n| **Sparse Operations** | linear algebra, eigenvalue problems, slicing, norms, reductions, factorization, symmetrization, components \u0026 labeling             |\n| **Solvers**           | combinatorial optimization, iterative solvers                                                                                     |\n| **Statistics**        | sampling, moments and summary statistics, metrics, model evaluation                                                               |\n| **Tools \u0026 Utilities** | common tools and utilities for developing CUDA applications, multi-node multi-gpu infrastructure                                  |\n\n\nRAFT is a C++ header-only template library with an optional shared library that\n1) can speed up compile times for common template types, and\n2) provides host-accessible \"runtime\" APIs, which don't require a CUDA compiler to use\n\nIn addition being a C++ library, RAFT also provides 2 Python libraries:\n- `pylibraft` - lightweight Python wrappers around RAFT's host-accessible \"runtime\" APIs.\n- `raft-dask` - multi-node multi-GPU communicator infrastructure for building distributed algorithms on the GPU with Dask.\n\n![RAFT is a C++ header-only template library with optional shared library and lightweight Python wrappers](img/arch.png)\n\n\n## Is RAFT right for me?\n\nRAFT contains low-level primitives for accelerating applications and workflows. Data source providers and application developers may find specific tools -- like ANN algorithms -- very useful. RAFT is not intended to be used directly by data scientists for discovery and experimentation. For data science tools, please see the [RAPIDS website](https://rapids.ai/).\n\n## Getting started\n\n### RAPIDS Memory Manager (RMM)\n\nRAFT relies heavily on RMM which eases the burden of configuring different allocation strategies globally across the libraries that use it.\n\n### Multi-dimensional Arrays\n\nThe APIs in RAFT accept the [mdspan](https://arxiv.org/abs/2010.06474) multi-dimensional array view for representing data in higher dimensions similar to the `ndarray` in the Numpy Python library. RAFT also contains the corresponding owning `mdarray` structure, which simplifies the allocation and management of multi-dimensional data in both host and device (GPU) memory.\n\nThe `mdarray` forms a convenience layer over RMM and can be constructed in RAFT using a number of different helper functions:\n\n```c++\n#include \u003craft/core/device_mdarray.hpp\u003e\n\nint n_rows = 10;\nint n_cols = 10;\n\nauto scalar = raft::make_device_scalar\u003cfloat\u003e(handle, 1.0);\nauto vector = raft::make_device_vector\u003cfloat\u003e(handle, n_cols);\nauto matrix = raft::make_device_matrix\u003cfloat\u003e(handle, n_rows, n_cols);\n```\n\n### C++ Example\n\nMost of the primitives in RAFT accept a `raft::device_resources` object for the management of resources which are expensive to create, such CUDA streams, stream pools, and handles to other CUDA libraries like `cublas` and `cusolver`.\n\nThe example below demonstrates creating a RAFT handle and using it with `device_matrix` and `device_vector` to allocate memory, generating random clusters, and computing\npairwise Euclidean distances:\n```c++\n#include \u003craft/core/device_resources.hpp\u003e\n#include \u003craft/core/device_mdarray.hpp\u003e\n#include \u003craft/random/make_blobs.cuh\u003e\n#include \u003craft/distance/distance.cuh\u003e\n\nraft::device_resources handle;\n\nint n_samples = 5000;\nint n_features = 50;\n\nauto input = raft::make_device_matrix\u003cfloat, int\u003e(handle, n_samples, n_features);\nauto labels = raft::make_device_vector\u003cint, int\u003e(handle, n_samples);\nauto output = raft::make_device_matrix\u003cfloat, int\u003e(handle, n_samples, n_samples);\n\nraft::random::make_blobs(handle, input.view(), labels.view());\n\nauto metric = raft::distance::DistanceType::L2SqrtExpanded;\nraft::distance::pairwise_distance(handle, input.view(), input.view(), output.view(), metric);\n```\n\nIt's also possible to create `raft::device_mdspan` views to invoke the same API with raw pointers and shape information. Take this example from the [NVIDIA cuVS](https://github.com/rapidsai/cuvs) library:\n\n```c++\n#include \u003craft/core/device_resources.hpp\u003e\n#include \u003craft/core/device_mdspan.hpp\u003e\n#include \u003craft/random/make_blobs.cuh\u003e\n#include \u003ccuvs/distance/distance.hpp\u003e\n\nraft::device_resources handle;\n\nint n_samples = 5000;\nint n_features = 50;\n\nfloat *input;\nint *labels;\nfloat *output;\n\n...\n// Allocate input, labels, and output pointers\n...\n\nauto input_view = raft::make_device_matrix_view(input, n_samples, n_features);\nauto labels_view = raft::make_device_vector_view(labels, n_samples);\nauto output_view = raft::make_device_matrix_view(output, n_samples, n_samples);\n\nraft::random::make_blobs(handle, input_view, labels_view);\n\nauto metric = cuvs::distance::DistanceType::L2SqrtExpanded;\ncuvs::distance::pairwise_distance(handle, input_view, input_view, output_view, metric);\n```\n\n\n### Python Example\n\nThe `pylibraft` package contains a Python API for RAFT algorithms and primitives. `pylibraft` integrates nicely into other libraries by being very lightweight with minimal dependencies and accepting any object that supports the `__cuda_array_interface__`, such as [CuPy's ndarray](https://docs.cupy.dev/en/stable/user_guide/interoperability.html#rmm). The number of RAFT algorithms exposed in this package is continuing to grow from release to release.\n\nThe example below demonstrates computing the pairwise Euclidean distances between CuPy arrays using the [NVIDIA cuVS](https://github.com/rapidsai/cuvs) library. Note that CuPy is not a required dependency for `pylibraft`.\n\n```python\nimport cupy as cp\n\nfrom cuvs.distance import pairwise_distance\n\nn_samples = 5000\nn_features = 50\n\nin1 = cp.random.random_sample((n_samples, n_features), dtype=cp.float32)\nin2 = cp.random.random_sample((n_samples, n_features), dtype=cp.float32)\n\noutput = pairwise_distance(in1, in2, metric=\"euclidean\")\n```\n\nThe `output` array in the above example is of type `raft.common.device_ndarray`, which supports [__cuda_array_interface__](https://numba.pydata.org/numba-doc/dev/cuda/cuda_array_interface.html#cuda-array-interface-version-2) making it interoperable with other libraries like CuPy, Numba, PyTorch and RAPIDS cuDF that also support it. CuPy supports DLPack, which also enables zero-copy conversion from `raft.common.device_ndarray` to JAX and Tensorflow.\n\nBelow is an example of converting the output `pylibraft.device_ndarray` to a CuPy array:\n```python\ncupy_array = cp.asarray(output)\n```\n\nAnd converting to a PyTorch tensor:\n```python\nimport torch\n\ntorch_tensor = torch.as_tensor(output, device='cuda')\n```\n\nOr converting to a RAPIDS cuDF dataframe:\n```python\ncudf_dataframe = cudf.DataFrame(output)\n```\n\nWhen the corresponding library has been installed and available in your environment, this conversion can also be done automatically by all RAFT compute APIs by setting a global configuration option:\n```python\nimport pylibraft.config\npylibraft.config.set_output_as(\"cupy\")  # All compute APIs will return cupy arrays\npylibraft.config.set_output_as(\"torch\") # All compute APIs will return torch tensors\n```\n\nYou can also specify a `callable` that accepts a `pylibraft.common.device_ndarray` and performs a custom conversion. The following example converts all output to `numpy` arrays:\n```python\npylibraft.config.set_output_as(lambda device_ndarray: return device_ndarray.copy_to_host())\n```\n\n`pylibraft` also supports writing to a pre-allocated output array so any `__cuda_array_interface__` supported array can be written to in-place:\n\n```python\nimport cupy as cp\n\nfrom cuvs.distance import pairwise_distance\n\nn_samples = 5000\nn_features = 50\n\nin1 = cp.random.random_sample((n_samples, n_features), dtype=cp.float32)\nin2 = cp.random.random_sample((n_samples, n_features), dtype=cp.float32)\noutput = cp.empty((n_samples, n_samples), dtype=cp.float32)\n\npairwise_distance(in1, in2, out=output, metric=\"euclidean\")\n```\n\n\n## Installing\n\nRAFT's C++ and Python libraries can both be installed through Conda and the Python libraries through Pip.\n\n\n### Installing C++ and Python through Conda\n\nThe easiest way to install RAFT is through conda and several packages are provided.\n- `libraft-headers` C++ headers\n- `pylibraft` (optional) Python library\n- `raft-dask` (optional) Python library for deployment of multi-node multi-GPU algorithms that use the RAFT `raft::comms` abstraction layer in Dask clusters.\n\nUse the following command, depending on your CUDA version, to install all of the RAFT packages with conda (replace `rapidsai` with `rapidsai-nightly` to install more up-to-date but less stable nightly packages). `mamba` is preferred over the `conda` command.\n```bash\n# for CUDA 11.8\nmamba install -c rapidsai -c conda-forge -c nvidia raft-dask pylibraft cuda-version=11.8\n```\n\n```bash\n# for CUDA 12.5\nmamba install -c rapidsai -c conda-forge -c nvidia raft-dask pylibraft cuda-version=12.8\n```\n\nNote that the above commands will also install `libraft-headers` and `libraft`.\n\nYou can also install the conda packages individually using the `mamba` command above. For example, if you'd like to install RAFT's headers and pre-compiled shared library to use in your project:\n```bash\n# for CUDA 12.5\nmamba install -c rapidsai -c conda-forge -c nvidia libraft libraft-headers cuda-version=12.8\n```\n\n### Installing Python through Pip\n\n`pylibraft` and `raft-dask` both have experimental packages that can be [installed through pip](https://rapids.ai/pip.html#install):\n```bash\npip install pylibraft-cu11 --extra-index-url=https://pypi.nvidia.com\npip install raft-dask-cu11 --extra-index-url=https://pypi.nvidia.com\n```\n\nThese packages statically build RAFT's pre-compiled instantiations and so the C++ headers won't be readily available to use in your code.\n\nThe [build instructions](https://docs.rapids.ai/api/raft/nightly/build/) contain more details on building RAFT from source and including it in downstream projects. You can also find a more comprehensive version of the above CPM code snippet the [Building RAFT C++ and Python from source](https://docs.rapids.ai/api/raft/nightly/build/#building-c-and-python-from-source) section of the build instructions.\n\n\n## Contributing\n\nIf you are interested in contributing to the RAFT project, please read our [Contributing guidelines](docs/source/contributing.md). Refer to the [Developer Guide](docs/source/developer_guide.md) for details on the developer guidelines, workflows, and principals.\n\n## References\n\nWhen citing RAFT generally, please consider referencing this Github project.\n```bibtex\n@misc{rapidsai,\n  title={Rapidsai/raft: RAFT contains fundamental widely-used algorithms and primitives for data science, Graph and machine learning.},\n  url={https://github.com/rapidsai/raft},\n  journal={GitHub},\n  publisher={NVIDIA RAPIDS},\n  author={Rapidsai},\n  year={2022}\n}\n```\n","funding_links":[],"categories":["Open Sources","Multidimensional data / Vectors","statistics"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frapidsai%2Fraft","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frapidsai%2Fraft","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frapidsai%2Fraft/lists"}