https://github.com/at15/faiss-s3
On-demand Faiss vector search from S3. Lazy-loads clusters without fetching entire index.
https://github.com/at15/faiss-s3
faiss faiss-backend faiss-vector-database rag
Last synced: 6 months ago
JSON representation
On-demand Faiss vector search from S3. Lazy-loads clusters without fetching entire index.
- Host: GitHub
- URL: https://github.com/at15/faiss-s3
- Owner: at15
- License: mit
- Created: 2025-10-29T02:14:06.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2025-10-29T04:17:46.000Z (6 months ago)
- Last Synced: 2025-10-29T04:25:36.961Z (6 months ago)
- Topics: faiss, faiss-backend, faiss-vector-database, rag
- Language: C++
- Homepage:
- Size: 13.7 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# faiss-s3
On-demand Faiss vector search from S3. Lazy-loads clusters without fetching entire index.
Original implementation: https://github.com/at15/faiss/tree/at15/faiss-on-s3/demos/s3_ivf
## TODO
- [ ] e2e test script in python
- [ ] github action (with cache on faiss and aws-sdk?)
- [ ] benchmark using larger dataset
## Usage
- [ ] Add how to build server
```bash
python3.13 -m venv .venv
source .venv/bin/activate
# Brings in boto3 and faiss-cpu
pip install -e ./client
# Generate embeddings
pip install sentence-transformers pandas
```
## License
MIT