https://github.com/specializedgeneralist/hnsw-grpc-server
gRPC server for hnswlib
https://github.com/specializedgeneralist/hnsw-grpc-server
hnsw hnswlib vector-search-engine vector-similarity-database
Last synced: 5 months ago
JSON representation
gRPC server for hnswlib
- Host: GitHub
- URL: https://github.com/specializedgeneralist/hnsw-grpc-server
- Owner: SpecializedGeneralist
- License: apache-2.0
- Archived: true
- Created: 2021-04-04T21:58:38.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2023-03-06T23:53:14.000Z (over 3 years ago)
- Last Synced: 2024-06-21T20:08:43.731Z (about 2 years ago)
- Topics: hnsw, hnswlib, vector-search-engine, vector-similarity-database
- Language: Go
- Homepage:
- Size: 217 KB
- Stars: 14
- Watchers: 3
- Forks: 0
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# hnsw-grpc-server
[](https://goreportcard.com/report/github.com/SpecializedGeneralist/hnsw-grpc-server)
[](https://codeclimate.com/github/SpecializedGeneralist/hnsw-grpc-server/maintainability)
[](https://codeclimate.com/github/SpecializedGeneralist/hnsw-grpc-server/test_coverage)
This is a gRPC server for [hnswlib](https://github.com/nmslib/hnswlib).
It provides more than just the core HNSW model: it is a tool that can be used
end-to-end, supporting TLS encryption, multiple persistent indices and batch
insertions.
This repository includes the relevant sources from the [hnswlib](https://github.com/nmslib/hnswlib),
so it doesn't require any external dependency. For more information please
refer to [hnswlib](https://github.com/nmslib/hnswlib) and
[Efficient and robust approximate nearest neighbor search using Hierarchical
Navigable Small World graphs](https://arxiv.org/abs/1603.09320).
## Features
A list of API methods now follows:
| Method | Description |
| -------------- | --------- |
| CreateIndex | Make a new index |
| DeleteIndex | Removes an index |
| InsertVector | Insert a new vector in the given index, letting the index generate an ID |
| InsertVectors | Insert new vectors in the given index, with generated ID, then flush the index |
| InsertVectorWithId | Insert a new vector with given ID in the given index |
| InsertVectorsWithId | Insert new vectors with given IDs in the given index, then flush the index |
| SearchKNN | Return the top k nearest neighbors to the query, searching on the given index |
| FlushIndex | Serialize the index to file |
| Indices | Return the list of indices |
| SetEf | Set the `ef` parameter for the given index |
## Build and run
You first need to compile the C++ `hnswlib` wrapper. Just run the following
script to compile it with `g++`:
```shell
./pkg/hnswgo/make.sh
```
Then, download Go dependencies and build the package (with `cgo` enabled):
```shell
go mod download
CGO_CXXFLAGS="-std=c++11" CGO_ENABLED=1 go build \
-ldflags="-extldflags=-static" \
-o hnsw-grpc-server \
cmd/main.go
```
You can finally run the executable. For example, you can get help running:
```shell
./hnsw-grpc-server -h
```
## Docker
The [Docker](https://www.docker.com/) image can be built like this:
```console
docker build -t hnsw-grpc-server:latest .
```
Pre-built images are available on Docker Hub, at [specializedgeneralist/hnsw-grpc-server](https://hub.docker.com/r/specializedgeneralist/hnsw-grpc-server).
For example, you can pull the image and run the server like this:
```shell
docker run -d \
--name hnsw-grpc-server \
-v /path/to/your/data/folder:/hnsw-grpc-server-data
-p 19530:19530 \
specializedgeneralist/hnsw-grpc-server:1.0.0
```
## Credits
- [hnswlib](https://github.com/nmslib/hnswlib) - Header-only C++ HNSW implementation.
- [hnswgo](https://github.com/evan176/hnswgo) - Go interface for hnswlib.