{"id":20282358,"url":"https://github.com/redisai/vectorsimilarity","last_synced_at":"2025-08-21T07:32:40.637Z","repository":{"id":37757098,"uuid":"349954736","full_name":"RedisAI/VectorSimilarity","owner":"RedisAI","description":null,"archived":false,"fork":false,"pushed_at":"2025-08-20T09:37:08.000Z","size":13220,"stargazers_count":46,"open_issues_count":15,"forks_count":21,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-08-20T10:16:18.008Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RedisAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-03-21T09:40:38.000Z","updated_at":"2025-08-18T17:23:03.000Z","dependencies_parsed_at":"2024-04-24T10:46:41.162Z","dependency_job_id":"8c4c1d51-b20e-4646-8c33-d8940d841592","html_url":"https://github.com/RedisAI/VectorSimilarity","commit_stats":{"total_commits":355,"total_committers":13,"mean_commits":"27.307692307692307","dds":0.6929577464788732,"last_synced_commit":"1e55ba726e8723978db848833b5ffddd68b4f66f"},"previous_names":[],"tags_count":50,"template":false,"template_full_name":null,"purl":"pkg:github/RedisAI/VectorSimilarity","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisAI%2FVectorSimilarity","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisAI%2FVectorSimilarity/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisAI%2FVectorSimilarity/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisAI%2FVectorSimilarity/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RedisAI","download_url":"https://codeload.github.com/RedisAI/VectorSimilarity/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisAI%2FVectorSimilarity/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271444363,"owners_count":24760784,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-21T02:00:08.990Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T14:09:04.841Z","updated_at":"2025-08-21T07:32:40.630Z","avatar_url":"https://github.com/RedisAI.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![nightly](https://github.com/RedisAI/VectorSimilarity/actions/workflows/event-nightly.yml/badge.svg)](https://github.com/RedisAI/VectorSimilarity/actions/workflows/event-nightly.yml)\n[![codecov](https://codecov.io/gh/RedisAI/VectorSimilarity/branch/main/graph/badge.svg)](https://codecov.io/gh/RedisAI/VectorSimilarity)\n[![CodeQL](https://github.com/RedisAI/VectorSimilarity/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/RedisAI/VectorSimilarity/actions/workflows/codeql-analysis.yml)\n[![Known Vulnerabilities](https://snyk.io/test/github/RedisAI/VectorSimilarity/badge.svg)](https://snyk.io/test/github/RedisAI/VectorSimilarity)\n\n\n# VectorSimilarity\nStarting with version 8.0, RediSearch and this vector similarity library is an integral part of Redis. See https://github.com/redis/redis\n\nThis repo exposes a C API for using vector similarity search.\nAllows creating indices of vectors and searching for top K similar to some vector in two methods: brute force, and by using the hnsw algorithm (probabilistic).\n\nThe API header files are `vec_sim.h` and `query_results.h`, which are located in `src/VecSim`.\n\n## Algorithms\n\nAll of the algorithms in this library are designed to work inside RediSearch and support the following features:\n1. In place insert, delete, and update vectors in the index.\n2. KNN queries - results can be ordered by score or ID.\n3. Iterator interface for consecutive KNN queries.\n4. Range queries\n5. Multiple vector indexing for the same label (multi-value indexing)\n6. 3rd party allocators\n\n### Flat (Brute Force)\n\nBrute force comparison of the query vector `q` with the stored vectors. Vectors are stored in vector blocks, which are contiguous memory blocks with a configurable size.\n\n\n### HNSW\nModified implementation of [hnswlib](https://github.com/nmslib/hnswlib). Modified to accommodate the above feature set.\n\n## Metrics\nWe support three popular distance metrics to measure the degree of similarity between two vectors:\n\n| Distance metric | Description                                                    | Value range      |\n|-----------------|----------------------------------------------------------------|------------------|\n| L2              | Euclidean distance between two vectors.                        | [0, +∞)          |\n| IP              | Inner product distance (vectors are assumed to be normalized). | [0, 2]           |\n| COSINE          | Cosine distance of two vectors.                                | [0, 2]           |\n\nThe above metrics calculate the distance between two vectors, where smaller values indicate that the vectors are closer in the vector space.\n\n### Datatypes and SIMD support\nThe library supports the following data types for distance computation:\n* `float32`\n* `float64`\n* `bfloat16`\n* `float16`\n* `int8`\n* `uint8`\n\nTo accelerate performance, we leverage SIMD instructions on both x86 and ARM CPUs.\nThe tables below detail the supported SIMD instruction sets (CPU flags) used for each data type on each architecture.\n\n#### x86_64 SIMD Support\n| Operation          | CPU flags                                                   |\n|--------------------|---------------------------------------------------------------------|\n| FP32 IP \u0026 Cosine   | SSE, AVX, AVX512F                                                  |\n| FP32 L2 distance   | SSE, AVX, AVX512F                                                  |\n| FP64 IP \u0026 Cosine   | SSE, AVX, AVX512F                                                  |\n| FP64 L2 distance   | SSE, AVX, AVX512F                                                  |\n| FP16 IP \u0026 Cosine   | F16C+FMA+AVX, AVX512F, AVX512FP16+AVX512VL                         |\n| FP16 L2 distance   | F16C+FMA+AVX, AVX512F, AVX512FP16+AVX512VL                         |\n| BF16 IP \u0026 Cosine   | SSE3, AVX2, AVX512BW+AVX512VBMI2, AVX512BF16+AVX512VL              |\n| BF16 L2 distance   | SSE3, AVX2, AVX512BW+AVX512VBMI2           |\n| INT8 IP \u0026 Cosine   | AVX512F+AVX512BW+AVX512VL+AVX512VNNI                               |\n| INT8 L2 distance   | AVX512F+AVX512BW+AVX512VL+AVX512VNNI                               |\n| UINT8 IP \u0026 Cosine  | AVX512F+AVX512BW+AVX512VL+AVX512VNNI                               |\n| UINT8 L2 distance  | AVX512F+AVX512BW+AVX512VL+AVX512VNNI                               |\n\n#### ARM SIMD Support (arm64v8 \u0026 Apple Silicon)\n| Operation           | CPU flags                             |\n|---------------------|-------------------------------------------|\n| FP32 IP \u0026 Cosine    | NEON, SVE, SVE2                           |\n| FP32 L2 distance    | NEON, SVE, SVE2                           |\n| FP64 IP \u0026 Cosine    | NEON, SVE, SVE2                           |\n| FP64 L2 distance    | NEON, SVE, SVE2                           |\n| FP16 IP \u0026 Cosine    | NEON_HP, SVE, SVE2                        |\n| FP16 L2 distance    | NEON_HP, SVE, SVE2                        |\n| BF16 IP \u0026 Cosine    | NEON_BF16, SVE_BF16                       |\n| BF16 L2 distance    | NEON_BF16, SVE_BF16                       |\n| INT8 IP \u0026 Cosine    | NEON, NEON_DOTPROD, SVE, SVE2            |\n| INT8 L2 distance    | NEON, NEON_DOTPROD, SVE, SVE2            |\n| UINT8 IP \u0026 Cosine   | NEON, NEON_DOTPROD, SVE, SVE2            |\n| UINT8 L2 distance   | NEON, NEON_DOTPROD, SVE, SVE2            |\n\n## Build\nFor building you will need:\n1. Python 3 as `python` (either by creating a virtual environment or setting your system python to point to the right python distribution)\n2. gcc \u003e= 10\n3. cmake version \u003e= 3.10. (To build the `python bindings` you will need cmake \u003c 3.26 due to `pybind11` policy version handling).\n\nTo build the main library and unit tests in one command run\n\n```\nmake\n```\n\n## Unit tests\nTo execute unit tests run\n\n```\nmake unit_test\n```\n\n## Memory check\n\nTo run the unit tests with Valgrind run\n```\nmake unit_test VALGRIND=1\n```\n\n## Python bindings\n\nExamples of using the Python bindings to run vector similarity search can be found in `tests/flow`.\nTo build the Python wheel, first create a dedicated virtualenv using Python 3.7 and higher. Then, activate the environment, install the dependencies, and build the package. Please note, due to the way poetry generates a setup.py, you may have to erase it before re-running *poetry build*.\n\n```\npython -m venv venv\nsource venv/bin/activate\npip install poetry\npoetry install\npoetry build\n```\n\nTo run in debug mode, replace the last two lines with:\n\n```\nDEBUG=1 poetry install\nDEBUG=1 poetry build\n```\n\nAfter building the wheel, if you want to use the package you built, you will need to manually execute a *pip install dist/\u003cpackage\u003e.whl*. Remember to replace \u003cpackage\u003e with the complete package name.\n\n### Testing Python bindings\nThis will create a new virtual environment (if needed), install the wheel, and execute the Python bindings tests\n```\npoetry run pytest tests/flow\n```\nOr you can use the make command:\n```\nmake flow_test\n```\n\n## Benchmark\n\nTo benchmark the capabilities of this library, follow the instructions in the [benchmarks user guide](docs/benchmarks.md).\nIf you'd like to create your own benchmarks, you can find more information in the [developer guide](docs/benchmarks_developer.md).\n\n\n## License\n\nStarting with Redis 8, this library is licensed under your choice of: (i) Redis Source Available License 2.0 (RSALv2); (ii) the Server Side Public License v1 (SSPLv1); or (iii) the GNU Affero General Public License version 3 (AGPLv3). Please review the license folder for the full license terms and conditions. Prior versions remain subject to (i) and (ii).\n\n## Code contributions\n\n\nBy contributing code to this Redis module in any form, including sending a pull request via GitHub, a code fragment or patch via private email or public discussion groups, you agree to release your code under the terms of the Redis Software Grant and Contributor License Agreement. Please see the CONTRIBUTING.md file in this source distribution for more information. For security bugs and vulnerabilities, please see SECURITY.md.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredisai%2Fvectorsimilarity","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fredisai%2Fvectorsimilarity","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredisai%2Fvectorsimilarity/lists"}