https://github.com/golbin/imvectordb
Super simple in-memory vector DB for Node.js
https://github.com/golbin/imvectordb
Last synced: 10 months ago
JSON representation
Super simple in-memory vector DB for Node.js
- Host: GitHub
- URL: https://github.com/golbin/imvectordb
- Owner: golbin
- Created: 2023-09-08T17:05:43.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2023-09-10T03:27:42.000Z (almost 3 years ago)
- Last Synced: 2024-04-22T12:41:50.749Z (about 2 years ago)
- Language: TypeScript
- Size: 85.9 KB
- Stars: 37
- Watchers: 2
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-vector-databases - imvectordb - Super simple and easy-to-use in-memory vector database for Node.js. Perfect for quickly building prototypes or small-scale applications with a compressed file size of just 3KB. ([Read more](/details/imvectordb.md)) `Javascript` `In Memory` `Lightweight` (SDKs & Libraries)
README
# In-memory VectorDB
Super simple and easy-to-use in-memory vector DB for Node.js
Perfect for quickly building prototypes or small-scale applications in Node.js.
With a compressed (ZIP) file size of just 3KB.
## Installation
```sh
npm install imvectordb
```
## Usage
Easily integrate it into your project with just a few lines of code:
```typescript
import { VectorDB } from 'imvectordb';
const db = new VectorDB();
db.addText("text for semantic search")
// ...add additional text as needed
db.queryText("text to search")
```
This library automatically uses OpenAI's "text-embedding-ada-002" model for embedding, so you'll need to set the `OPENAI_API_KEY` environment variable.
You can also add documents to the database using your own embedding models.
Here's a complete guide:
```typescript
import { VectorDB } from 'imvectordb';
const db = new VectorDB();
// Add a new document to the database
db.add({
id: "1",
embedding: [0.014970540, ...],
metadata: {
text: "original text",
... // additional metadata
}
})
// Perform a search and retrieve the top 10 similar documents
const queryVector = [0.014970540, ...]
const searchResults = await db.query(queryVector, 10)
// Access search result details
searchResults[0].similarity
searchResults[0].document.id
searchResults[0].document.embedding
searchResults[0].document.metadata.text
// Retrieve or delete a document by its ID
db.get("1")
db.del("1")
// Save to or load from a file
db.dumpFile("filename.json")
db.loadFile("filename.json")
// Terminate the Worker when it's no longer needed or when the server closes
db.terminate()
```
For more examples, please check the "samples" folder and the available types.
## Performance
Use up to 10,000 documents; going over is not recommended. A few thousand is ideal.
There's significant room for performance improvement. Patches and PRs are welcome.
```
Machine: MackBook Air M2
Dimensions: 1,536 (text-embedding-ada)
----------
Search in 100 documents, 10 times.
----------
Total: 57.577ms
Average: 5.758ms
----------
Search in 1000 documents, 10 times.
----------
Total: 541.979ms
Average: 54.198ms
----------
Search in 10000 documents, 10 times.
----------
Total: 13430.621ms
Average: 1343.062ms
```
## Family
- [LLM Chunk](https://github.com/golbin/llm-chunk)
- Super simple and easy-to-use text splitter for Node.js
## License
MIT