https://github.com/barakmich/bbqvec
Scalable Embedded Vector Index for Go and Rust
https://github.com/barakmich/bbqvec
aknn ann golang knn rust similarity-search vector vector-database vector-search
Last synced: 4 months ago
JSON representation
Scalable Embedded Vector Index for Go and Rust
- Host: GitHub
- URL: https://github.com/barakmich/bbqvec
- Owner: barakmich
- License: apache-2.0
- Created: 2024-09-13T17:14:58.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-10-31T18:19:37.000Z (11 months ago)
- Last Synced: 2025-05-16T17:05:17.016Z (5 months ago)
- Topics: aknn, ann, golang, knn, rust, similarity-search, vector, vector-database, vector-search
- Language: Rust
- Homepage:
- Size: 453 KB
- Stars: 36
- Watchers: 0
- Forks: 4
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](LICENSE)
[](https://godoc.org/github.com/barakmich/bbqvec)
[](https://crates.io/crates/bbqvec)
[](https://github.com/barakmich/bbqvec/actions/workflows/go.yml)
[](https://github.com/barakmich/bbqvec/actions/workflows/rust.yml)
BBQvec is an open-source, embedded vector database index for Go and Rust, providing approximate K-nearest-neighbors (aKNN).
# Getting Started
## Go
```go
package main
import (
"fmt"
bbq "github.com/barakmich/bbqvec"
)
func main() {
// Declare store parameters
dimensions := 200
nBasis := 10
// Initialize the store
backend := bbq.NewMemoryBackend(dimensions)
datastore, _ := bbq.NewVectorStore(backend, nBasis)
// Create some test data, 100K random vectors
vecs := bbq.NewRandVectorSet(100_000, dimensions, nil)
datastore.AddVectorsWithOffset(0, vecs)
/*
Equivalent to:
for i, v := range vecs {
datastore.AddVector(bbq.ID(i), v)
}
*/
// Run a query
targetVec := bbq.NewRandVector(dimensions, nil)
results, _ := datastore.FindNearest(targetVec, 10, 1000, 1)
// Inspect the results
top := results.ToSlice()[0]
vec, _ := backend.GetVector(top.ID)
fmt.Println(top.ID, vec, top.Similarity)
}
```
## Rust
```rust
use bbqvec::IndexIDIterator;
fn main() -> Result<()> {
// Declare store parameters
let dimensions = 200;
let n_basis = 10;
// Initialize the store
let mem = bbqvec::MemoryBackend::new(dimensions, n_basis)?;
let mut store = bbqvec::VectorStore::new(mem)?;
// Create some test data, 100K random vectors
let vecs = bbqvec::create_vector_set(dimensions, 100000);
store.add_vector_iter(vecs.enumerate_ids())?;
// Run a query
let target = bbqvec::create_random_vector(dimensions);
let results = store.find_nearest(&target, 10, 1000, 1)?;
// Inspect the results
for res in results.iter_results() {
println!("{} {}", res.id, res.similarity)
}
}
```
# TODOs
We're still early; Go is the more tried-and-true and suits the beta use-case, but Rust is a good deal faster. We welcome contributions.
## Go
- [ ] More benchmarks
- [ ] New Quantizations
- [ ] Hamming Distance (single-bit vectors)
- [ ] Novel quantizations
## Rust
- [ ] Finish disk backend to match Go (in progress, shortly)
- [ ] New Quantizations
### Acknowledgements
Thank you to MariaLetta for the [free-gophers-pack](https://github.com/MariaLetta/free-gophers-pack) and to [rustacean.net](https://rustacean.net) for the CC0 logo characters.