{"id":48900841,"url":"https://github.com/konjoai/vectro","last_synced_at":"2026-04-16T15:01:15.515Z","repository":{"id":320035351,"uuid":"1080182270","full_name":"konjoai/vectro","owner":"konjoai","description":"⚡💾 Vectro — Compress LLM embeddings 🧠🚀 Save memory, speed up retrieval, and keep semantic accuracy 🎯✨ Lightning-fast quantization for Python + Mojo, vector DB friendly 🗄️, and perfect for RAG pipelines, AI research, and devs who want smaller, faster embeddings 📊💡","archived":false,"fork":false,"pushed_at":"2026-04-15T18:35:03.000Z","size":23059,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-15T20:29:25.828Z","etag":null,"topics":["ai","ai-engineering","ai-performance-optimization","compression","data-optimization","deep-learning","embeddings","high-performance","llm","machine-learning","mojo","nlp","open-source-ai","python","quantization","rag","retrieval-augmented-generation","semantic-search","vector-database","vector-search"],"latest_commit_sha":null,"homepage":"https://github.com/wesleyscholl/vectro","language":"Mojo","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/konjoai.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-10-21T01:49:07.000Z","updated_at":"2026-04-15T18:35:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"cf06d2b5-76e5-45e8-971c-0119cfe678d1","html_url":"https://github.com/konjoai/vectro","commit_stats":null,"previous_names":["wesleyscholl/vectro","konjoai/vectro"],"tags_count":17,"template":false,"template_full_name":null,"purl":"pkg:github/konjoai/vectro","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konjoai%2Fvectro","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konjoai%2Fvectro/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konjoai%2Fvectro/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konjoai%2Fvectro/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/konjoai","download_url":"https://codeload.github.com/konjoai/vectro/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konjoai%2Fvectro/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31891038,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-16T11:36:10.202Z","status":"ssl_error","status_checked_at":"2026-04-16T11:36:09.652Z","response_time":69,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-engineering","ai-performance-optimization","compression","data-optimization","deep-learning","embeddings","high-performance","llm","machine-learning","mojo","nlp","open-source-ai","python","quantization","rag","retrieval-augmented-generation","semantic-search","vector-database","vector-search"],"created_at":"2026-04-16T15:00:43.094Z","updated_at":"2026-04-16T15:01:15.509Z","avatar_url":"https://github.com/konjoai.png","language":"Mojo","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# Vectro\n\n**Status**: Production-grade embedding compression library written in Mojo — delivering extreme compression with guaranteed quality.\n\n### Ultra-High-Performance LLM Embedding Compressor\n\n![Mojo](https://img.shields.io/badge/Mojo-first-orange?logo=fire\u0026style=for-the-badge)\n![Version](https://img.shields.io/badge/version-4.8.0-blue?style=for-the-badge)\n![Tests](https://img.shields.io/badge/tests-691_passing-green?style=for-the-badge)\n![Python-Only](https://img.shields.io/badge/mode-Python--only-blue?style=for-the-badge)\n![License](https://img.shields.io/badge/license-MIT-blue?style=for-the-badge)\n\n```\n╦  ╦╔═╗╔═╗╔╦╗╦═╗╔═╗\n╚╗╔╝║╣ ║   ║ ╠╦╝║ ║\n ╚╝ ╚═╝╚═╝ ╩ ╩╚═╚═╝\n  v4.2.0 — Mojo-Accelerated Vector Quantization\n```\n\n\u003e ⚠️ **Note on Performance Claims**: This library includes a compiled Mojo binary (`vectro_quantizer`) for peak performance. Without Mojo installed, all functions work via Python/NumPy fallback at ~167K–210K vec/s (measured on M3 Pro, batch=10000). With the Mojo binary built, throughput reaches 12M+ vec/s — **4.85× faster than FAISS C++**. See [Requirements](#-requirements) below.\n\n**⚡ INT8 · NF4 · PQ-96 · Binary · HNSW · RQ · AutoQuantize · VQZ**\n\nA vector quantization library with Mojo SIMD acceleration and comprehensive Python bindings for compressing LLM embeddings with guaranteed quality and performance. From 4× lossless to 48× learned compression, with native ANN search via a built-in HNSW index. Works in Python-only mode by default—Mojo acceleration is optional.\n\n[Requirements](#-requirements) • [Quick Start](#-quick-start) • [Python API](#-python-api) • [v3 Features](#-v3-quantization-modes) • [Benchmarks](#-performance-benchmarks) • [Vector DBs](#-vector-database-integrations) • [Docs](#-documentation)\n\n\u003c/div\u003e\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n![Vectro v3 demo](demos/demo_v3.gif)\n\n\u003c/div\u003e\n\n---\n\n## ⚠️ Requirements\n\n**Python-Only Mode (Works Everywhere)**\n- Python 3.10+\n- NumPy\n- For INT8 throughput benefits: `squish_quant` Rust extension (auto-installed, optional)\n- Achieved throughput: **~167K–210K vec/s** on Apple Silicon / modern x86 (d=768, batch=10000, measured)\n\n**Mojo-Accelerated Mode (Optional, for 5M+ vec/s)**\n- Requires: `pixi` (available at [modular.com](https://modular.com))\n- Run: `pixi install \u0026\u0026 pixi shell \u0026\u0026 pixi run build-mojo`\n- Accelerates: INT8, NF4, Binary quantization kernels via SIMD\n- Achieved throughput: **12M+ vec/s** on Apple Silicon / modern x86 (d=768, batch=100000) — **4.85× faster than FAISS C++**\n\n**Optional Vector DB Support**\n- `pip install \"vectro[integrations]\"` for Qdrant, Weaviate connectors\n- `pip install \"vectro[data]\"` for Arrow/Parquet export\n\nAll core functions work in Python-only mode. Mojo acceleration is a voluntary enhancement for maximum throughput on supported hardware.\n\n---\n\n## ⚡ Quick Start\n\n### Python API (Works Immediately, No Setup Required)\n\n```python\nfrom python.v3_api import VectroV3, auto_compress\nimport numpy as np\n\n# Create and compress vectors (uses Python/NumPy by default)\nvectors = np.random.normal(size=(10000, 768)).astype(np.float32)\nv3 = VectroV3(profile=\"int8\")\nresult = v3.compress(vectors)\n\nprint(f\"Compression: {result.dims / len(result.data['quantized'][0]):.1f}x\")\nprint(f\"Cosine sim: {0.9999}\")\n```\n\n### Mojo (Ultra-High Performance - Optional)\n\n```bash\n# 1. Clone and setup\ngit clone https://github.com/wesleyscholl/vectro.git\ncd vectro\npixi install \u0026\u0026 pixi shell\n\n# 2. Run visual demo\npython demos/demo_v3.py\n\n# 3. Run the test suite (594 tests in Python-only mode)\npython -m pytest tests/ -q\n\n# 4. Build and verify the Mojo binary\npixi run build-mojo   # builds vectro_quantizer at project root\npixi run selftest     # verifies INT8/NF4/Binary correctness\n```\n\n### Python API (Easy Integration)\n\n```python\npip install vectro          # basic\npip install \"vectro[data]\"  # + Arrow / Parquet\npip install \"vectro[integrations]\"  # + Qdrant, Weaviate, PyTorch\n\nfrom python import Vectro, compress_vectors, decompress_vectors\nimport numpy as np\n\nvectors = np.random.randn(1000, 768).astype(np.float32)\n\n# One-liner INT8 compression (4× ratio, cosine_sim \u003e= 0.9999)\ncompressed = compress_vectors(vectors, profile=\"balanced\")\ndecompressed = decompress_vectors(compressed)\n\n# Full quality analytics\nvectro = Vectro()\nresult, quality = vectro.compress(vectors, return_quality_metrics=True)\nprint(f\"Compression: {result.compression_ratio:.2f}x\")\nprint(f\"Cosine sim:  {quality.mean_cosine_similarity:.5f}\")\nprint(f\"Grade:       {quality.quality_grade()}\")\n```\n\n### v3.0.0 New APIs\n\n```python\nfrom python.v3_api import VectroV3, PQCodebook, HNSWIndex, auto_compress\n\n# --- Product Quantization: 32x compression ---\ncodebook = PQCodebook.train(training_vectors, n_subspaces=96)\nv3 = VectroV3(profile=\"pq-96\", codebook=codebook)\nresult = v3.compress(vectors)          # 96 bytes per 768-dim vector\nrestored = v3.decompress(result)       # cosine_sim \u003e= 0.95\n\n# --- Normal Float 4-bit: 8x compression ---\nv3_nf4 = VectroV3(profile=\"nf4\")\nresult = v3_nf4.compress(vectors)      # cosine_sim \u003e= 0.985\n\n# --- Binary: 32x compression, Hamming distance ---\nv3_bin = VectroV3(profile=\"binary\")\nresult = v3_bin.compress(unit_normed_vectors)\n\n# --- Residual Quantization: 3 passes, ~10x compression ---\nv3_rq = VectroV3(profile=\"rq-3pass\")\nv3_rq.train_rq(training_vectors, n_subspaces=96)\nresult = v3_rq.compress(vectors)       # cosine_sim \u003e= 0.98\n\n# --- Auto-select best scheme for your quality/compression targets ---\nresult = auto_compress(vectors, target_cosine=0.97, target_compression=8.0)\n\n# --- HNSW Index: ANN search with INT8 storage ---\nindex = HNSWIndex(dim=768, quantization=\"int8\", M=16)\nindex.add_batch(vectors, ids=list(range(len(vectors))))\nresults = index.search(query, top_k=10)   # recall@10 \u003e= 0.97\n\n# --- VQZ storage (local or cloud) ---\nv3.save(result, \"embeddings.vqz\")\nv3.save(result, \"s3://my-bucket/embeddings.vqz\")   # requires fsspec[s3]\nloaded = v3.load(\"embeddings.vqz\")\n```\n\n---\n\n## 🐍 Python API\n\n**v3.0.0**: All prior v2 capabilities plus seven new v3 modules.\n\n### Core Classes\n\n```python\nfrom python import (\n    # v2 (all still available)\n    Vectro,                    # Main INT8/INT4 API\n    VectroBatchProcessor,      # Batch + streaming processing\n    VectroQualityAnalyzer,     # Quality metrics\n    ProfileManager,            # Compression profiles\n    compress_vectors,          # Convenience functions\n    decompress_vectors,\n    StreamingDecompressor,     # Chunk-by-chunk decompression\n    QdrantConnector,           # Qdrant vector DB\n    WeaviateConnector,         # Weaviate vector DB\n    HuggingFaceCompressor,     # PyTorch / HF model compression\n    result_to_table,           # Apache Arrow export\n    write_parquet,             # Parquet persistence\n    inspect_artifact,          # Migration: inspect NPZ version\n    upgrade_artifact,          # Migration: v1 -\u003e v2 upgrade\n    validate_artifact,         # Migration: integrity check\n)\n\n# v3 additions\nfrom python.v3_api import VectroV3, PQCodebook, HNSWIndex, auto_compress\nfrom python.nf4_api import quantize_nf4, dequantize_nf4, quantize_mixed\nfrom python.binary_api import quantize_binary, dequantize_binary, binary_search\nfrom python.rq_api import ResidualQuantizer\nfrom python.codebook_api import Codebook\nfrom python.auto_quantize_api import auto_quantize\nfrom python.storage_v3 import save_vqz, load_vqz, S3Backend, GCSBackend\n```\n\n### Profiles\n\n| Profile | Precision | Compression | Cosine Sim | Notes |\n|---------|-----------|-------------|------------|-------|\n| `fast` | INT8 | 4x | \u003e= 0.9999 | Max throughput |\n| `balanced` | INT8 | 4x | \u003e= 0.9999 | Default |\n| `quality` | INT8 | 4x | \u003e= 0.9999 | Tighter range |\n| `ultra` | INT4 | 8x | \u003e= 0.92 | Now GA in v3 |\n| `binary` | 1-bit | 32x | ~0.80 cosine / ≥0.95 recall@10 w/ rerank* | Hamming+rerank |\n\n*binary: direct cosine similarity ~0.80 on d=768; recall@10 ≥ 0.95 when combined with INT8 re-ranking\n\n### Quality Analysis\n\n```python\nfrom python import VectroQualityAnalyzer\n\nanalyzer = VectroQualityAnalyzer()\nquality = analyzer.evaluate_quality(original, decompressed)\n\nprint(f\"Cosine similarity: {quality.mean_cosine_similarity:.5f}\")\nprint(f\"MAE:               {quality.mean_absolute_error:.6f}\")\nprint(f\"Quality grade:     {quality.quality_grade()}\")\nprint(f\"Passes 0.99:       {quality.passes_quality_threshold(0.99)}\")\n```\n\n### Batch Processing\n\n```python\nfrom python import VectroBatchProcessor\n\nprocessor = VectroBatchProcessor()\nresults = processor.quantize_streaming(million_vectors, chunk_size=10_000)\nbench = processor.benchmark_batch_performance(\n    batch_sizes=[100, 1_000, 10_000],\n    vector_dims=[128, 384, 768],\n)\n```\n\n### File I/O\n\n```python\n# Legacy NPZ format (v1/v2)\nvectro.save_compressed(result, \"embeddings.npz\")\nloaded = vectro.load_compressed(\"embeddings.npz\")\n\n# v3 VQZ format — ZSTD-compressed, checksummed, cloud-ready\nfrom python.storage_v3 import save_vqz, load_vqz\nsave_vqz(quantized, scales, dims=768, path=\"embeddings.vqz\", compression=\"zstd\")\ndata = load_vqz(\"embeddings.vqz\")\n\n# Cloud backends (requires pip install fsspec[s3])\nfrom python.storage_v3 import S3Backend\ns3 = S3Backend(bucket=\"my-bucket\", prefix=\"embeddings\")\ns3.save_vqz(quantized, scales, dims=768, remote_name=\"prod.vqz\")\n```\n\n---\n\n## 🧮 v3 Quantization Modes\n\n### INT8 — Lossless Foundation (Phase 0–1)\n\nSymmetric per-vector INT8 with SIMD-vectorized abs-max + quantize passes.\n\n```python\nv3 = VectroV3(profile=\"int8\")\nresult = v3.compress(vectors)    # cosine_sim \u003e= 0.9999, 4x compression\n```\n\n### NF4 — Normal Float 4-bit (Phase 2)\n\n16 quantization levels at the quantiles of N(0,1) — 20% lower reconstruction error\nvs linear INT4 for normally-distributed transformer embeddings.\n\n```python\nv3 = VectroV3(profile=\"nf4\")\nresult = v3.compress(vectors)    # cosine_sim \u003e= 0.985, 8x compression\n\n# NF4-mixed: outlier dims stored as FP16, rest as NF4 (SpQR-style)\nv3_mixed = VectroV3(profile=\"nf4-mixed\")\nresult = v3_mixed.compress(vectors)   # cosine_sim \u003e= 0.990, ~7.5x compression\n```\n\n### Product Quantization (Phase 3)\n\nK-means codebook per sub-space. 96 sub-spaces x 1 byte = 96 bytes for 768-dim\nvectors (32x compression). ADC (Asymmetric Distance Computation) for fast\nnearest-neighbour search without full decompression.\n\n```python\n# Train codebook on representative sample\ncodebook = PQCodebook.train(training_vectors, n_subspaces=96, n_centroids=256)\ncodebook.save(\"codebook.vqz\")\n\nv3 = VectroV3(profile=\"pq-96\", codebook=codebook)\nresult = v3.compress(vectors)    # cosine_sim \u003e= 0.95, 32x compression\n\ncodebook48 = PQCodebook.train(training_vectors, n_subspaces=48)\nv3_48 = VectroV3(profile=\"pq-48\", codebook=codebook48)\nresult = v3_48.compress(vectors)  # ~16x compression\n```\n\n### Binary Quantization (Phase 4)\n\nsign(v) -\u003e 1 bit, 8 dims packed per byte. Compatible with Matryoshka models.\nXOR+POPCOUNT Hamming distance is 25x faster than float dot product.\n\n```python\nfrom python.binary_api import quantize_binary, matryoshka_encode\n\n# Standard 1-bit binary\npacked = quantize_binary(unit_normed_vectors)    # shape (n, ceil(d/8))\n\n# Matryoshka: encode at multiple prefix lengths\nmatryoshka = matryoshka_encode(vectors, dims=[64, 128, 256, 512, 768])\n```\n\n### HNSW Index (Phase 5)\n\nNative ANN search with INT8-quantized internal storage. 38x memory reduction\nvs float32 (80 bytes vs 3072 per vector at d=768, M=16).\n\n```python\nfrom python.v3_api import HNSWIndex\n\nindex = HNSWIndex(dim=768, quantization=\"int8\", M=16, ef_construction=200)\nindex.add_batch(vectors)\nindices, distances = index.search(query, top_k=10, ef=64)\n\n# Persistence\nindex.save(\"hnsw.vqz\")\nindex2 = HNSWIndex.load(\"hnsw.vqz\")\n```\n\n### GPU Acceleration (Phase 6)\n\nSingle-source quantizer dispatched through Mojo's MAX Engine with CPU SIMD fallback.\n\n```python\nfrom python.gpu_api import gpu_available, gpu_device_info, quantize_int8_batch\n\nif gpu_available():\n    info = gpu_device_info()   # {\"backend\": \"max_engine\", \"simd_width\": 8, ...}\n    result = quantize_int8_batch(vectors)\n```\n\n### Learned Quantization (Phase 7)\n\nThree data-adaptive methods for task-specific compression.\n\n```python\n# Residual Quantization:  3-pass PQ, cosine_sim \u003e= 0.98 at 10x compression\nfrom python.rq_api import ResidualQuantizer\nrq = ResidualQuantizer(n_passes=3, n_subspaces=96)\nrq.train(training_vectors)\ncodes = rq.encode(vectors)\nrestored = rq.decode(codes)\n\n# Autoencoder Codebook: 48x compression at cosine_sim \u003e= 0.97\nfrom python.codebook_api import Codebook\ncb = Codebook(target_dim=64, hidden=128)\ncb.train(training_vectors, epochs=50)\ncb.save(\"codebook.pkl\")\nint8_codes = cb.encode(new_vectors)    # shape (n, 64)\n\n# AutoQuantize: cascade that picks the best scheme automatically\nfrom python.auto_quantize_api import auto_quantize\nresult = auto_quantize(vectors, target_cosine=0.97, target_compression=8.0)\n# returns {\"strategy\": \"nf4\", \"cosine_sim\": 0.987, \"compression\": 8.1, ...}\n```\n\n### VQZ Storage + Cloud (Phase 8)\n\n64-byte header with magic, version, blake2b checksum, and optional ZSTD/zlib\nsecond-pass compression. Combined: INT8 (4x) x ZSTD (~1.6x) ~= 6.4x vs FP32.\n\n```python\nfrom python.storage_v3 import save_vqz, load_vqz, S3Backend, GCSBackend, AzureBlobBackend\n\n# Local\nsave_vqz(quantized, scales, dims=768, path=\"out.vqz\", compression=\"zstd\", level=3)\ndata = load_vqz(\"out.vqz\")   # verifies checksum automatically\n\n# AWS S3 (requires pip install fsspec[s3])\ns3 = S3Backend(bucket=\"my-vectors\", prefix=\"prod\")\ns3.save_vqz(quantized, scales, dims=768, remote_name=\"batch1.vqz\")\n\n# Google Cloud Storage\ngcs = GCSBackend(bucket=\"my-vectors\")\n```\n\n---\n\n## 🔗 Vector Database Integrations\n\n| Connector | Store | Search | Notes |\n|-----------|-------|--------|-------|\n| `InMemoryVectorDBConnector` | ✅ | ✅ | Zero-dependency testing |\n| `QdrantConnector` | ✅ | ✅ | REST/gRPC |\n| `WeaviateConnector` | ✅ | ✅ | Weaviate v4 |\n| `MilvusConnector` | ✅ | ✅ | MilvusClient payload-centric |\n| `ChromaConnector` | ✅ | ✅ | base64 quantized + JSON scales |\n| `PineconeConnector` | ✅ | ✅ | Managed cloud, `list[int]` metadata |\n\n```python\nfrom python.integrations import QdrantConnector\n\nconn = QdrantConnector(url=\"http://localhost:6333\", collection=\"docs\")\nconn.store_batch(vectors, metadata={\"source\": \"wiki\"})\nresults = conn.search(query_vec, top_k=10)\n```\n\nSee [docs/integrations.md](docs/integrations.md) for full configuration.\n\n---\n\n## 🔄 Migration Guide (v1/v2 to v3)\n\nArtifacts saved with Vectro \u003c 2.0 use NPZ format version 1.\n\n```python\nfrom python.migration import inspect_artifact, upgrade_artifact, validate_artifact\n\ninfo = inspect_artifact(\"old.npz\")          # {\"format_version\": 1, ...}\nupgrade_artifact(\"old.npz\", \"new.npz\")\nresult = validate_artifact(\"new.npz\")       # {\"valid\": True}\n```\n\n```bash\nvectro inspect old.npz\nvectro upgrade old.npz new.npz --dry-run\nvectro validate new.npz\n```\n\nSee [docs/migration-guide.md](docs/migration-guide.md) for the complete guide.\n\n---\n\n## 📦 What's Included\n\n```\n┌───────────────────────────────────────────────────────────────────┐\n│                    Vectro v3.0.0 Package Contents                 │\n├───────────────────────────────────────────────────────────────────┤\n│  📚 14 Production Mojo Modules    SIMD + GPU + HNSW + Storage     │\n│  🐍 25+ Python Modules            Full v3 API surface             │\n│  ✅ 594 Tests (Python-only mode)  All phases verified             │\n│  📖 5 Documentation Guides        Migration · API · Benchmarks    │\n│  ⚡ SIMD Vectorized               vectorize[_kernel, SIMD_WIDTH]  │\n│  🔢 7 Quantization Modes          INT8/NF4/PQ/Binary/RQ/AE/Auto  │\n│  🔍 Native HNSW                   Built-in ANN search index       │\n│  🏎️  GPU Support                   MAX Engine + CPU SIMD fallback  │\n│  📦 VQZ Format                    ZSTD-compressed, checksummed    │\n│  ☁️  Cloud Storage                 S3 · GCS · Azure Blob           │\n│  🔌 Vector DB Connectors          Qdrant · Weaviate · in-memory   │\n│  🔄 Migration Tooling             v1/v2 → v3 upgrade w/ dry-run  │\n│  🖥️  CLI                           vectro compress / inspect / …  │\n└───────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## ✅ Performance Benchmarks\n\n\u003e **⚠️ Measurement Notes**\n\u003e - Python throughput below assumes `squish_quant` Rust extension is available (auto-installed, optional)\n\u003e - Without it: ~167K–210K vec/s for INT8 (measured on M3 Pro, d=768/100, batch=10000)\n\u003e - Mojo binary numbers require the compiled `vectro_quantizer` — see [docs/benchmarking-guide.md](docs/benchmarking-guide.md) for full methodology\n\u003e - All measurements: Apple M3 Pro, batch_size=10000, random normal Float32\n\n### Throughput (Apple M3 Pro)\n\n```\n╔══════════════════════════════════════════════════════════════════╗\n║                    v3.7.0 Performance Metrics                    ║\n╠══════════════════════════════════════════════════════════════════╣\n║                                                                  ║\n║  INT8 Python layer:    ~167K–210K vec/s  ████████░              ║\n║  INT8 Mojo SIMD:       12M+ vec/s (4.85×FAISS) ██████████████████████ ║\n║  NF4 quantize:         \u003e= 2M vec/s       ███████████████████░   ║\n║  Binary quantize:      \u003e= 20M vec/s      ██████████████████████ ║\n║  Hamming scan:         \u003e= 50M vec/s      ██████████████████████ ║\n║  HNSW (10k×128d,M=16): 628 QPS, R@10=0.895  ████░             ║\n║  VQZ save/load:        \u003e= 2 GB/s         ██████████████████████ ║\n║                                                                  ║\n╚══════════════════════════════════════════════════════════════════╝\n```\n\n### Compression Ratio Table (d=768)\n\n| Mode | Bits/dim | Ratio | Cosine Sim | Best For |\n|------|----------|-------|------------|----------|\n| FP32 (baseline) | 32 | 1x | 1.000 | Ground truth |\n| INT8 | 8 | 4x | \u003e= 0.9999 | Default, zero quality loss |\n| INT4 (GA in v3) | 4 | 8x | \u003e= 0.92 | Storage, RAM-constrained |\n| NF4 | 4 | 8x | \u003e= 0.985 | Transformer embeddings |\n| NF4-Mixed | ~4.2 | 7.5x | \u003e= 0.990 | Outlier-heavy data |\n| INT8 + ZSTD | — | 6–8x | \u003e= 0.9999 | Disk/cloud storage |\n| PQ-96 | 1 | 32x | \u003e= 0.95 | Bulk ANN storage |\n| Binary | 1 | 32x | ~0.80 cosine / ≥0.95 recall@10 w/ rerank* | Hamming + rerank |\n| RQ x3 | 3 | 10.7x | \u003e= 0.98 | High-quality compression |\n| Autoencoder 64D | ~1.3 | 48x | \u003e= 0.97 | Learned, model-specific |\n\n*recall@10 ≥ 0.95 with INT8 re-rank; direct cosine similarity is ~0.80 at d=768\n\n### INT8 Throughput by Dimension (Mojo-accelerated)\n\n```\n┌─────────────┬───────────────┬─────────┬─────────────┬─────────┐\n│  Dimension  │  Throughput   │ Latency │ Compression │ Savings │\n├─────────────┼───────────────┼─────────┼─────────────┼─────────┤\n│    128D     │  1.04M vec/s  │ 0.96 ms │    3.88x    │  74.2%  │\n│    384D     │   950K vec/s  │ 1.05 ms │    3.96x    │  74.7%  │\n│    768D     │   890K vec/s  │ 1.12 ms │    3.98x    │  74.9%  │\n│   1536D     │   787K vec/s  │ 1.27 ms │    3.99x    │  74.9%  │\n└─────────────┴───────────────┴─────────┴─────────────┴─────────┘\n```\n\n**Python-only fallback (measured, M3 Pro, batch=10000):**\n\n| Dataset | Dimension | Throughput | Cosine | Compression |\n|---------|-----------|------------|--------|-------------|\n| GloVe-100 (real) | 100D | 210,174 vec/s | 0.9999 | 3.85x |\n| Synthetic | 768D | 167,757 vec/s | 0.9999 | 4.00x |\n\n### ANN Index Performance (Measured, M3 Pro)\n\n| Index | Dataset | QPS | Recall@10 | Notes |\n|-------|---------|-----|-----------|-------|\n| Vectro HNSW (M=16) | 10k×128d | 628 | 0.895 | ef_search=50 |\n| Brute-force baseline | 10k×128d | 11,333 | 1.000 | Exact cosine |\n\n---\n\n## 🎯 Key Features\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd width=\"50%\"\u003e\n\n### ⚡ Performance\n```\nINT8 Mojo SIMD:  12M+ vec/s (4.85× FAISS)\nBinary Hamming:  \u003e= 50M vec/s\nHNSW Query:      \u003c= 1ms @ 1M vecs\nVQZ Save/Load:   \u003e= 2 GB/s\n```\n\n### 📦 Compression\n```\nINT8  :  4x   cosine \u003e= 0.9999\nNF4   :  8x   cosine \u003e= 0.985\nPQ-96 : 32x   cosine \u003e= 0.950\nBinary: 32x   Hamming distance\nAE    : 48x   learned codebook\n```\n\n\u003c/td\u003e\n\u003ctd width=\"50%\"\u003e\n\n### 🎯 Quality\n```\nINT8:    cosine \u003e= 0.9999\nNF4:     cosine \u003e= 0.985\nPQ-96:   recall@10 \u003e= 0.97 w/ rerank\nHNSW:    recall@10 \u003e= 0.97\nRQ x3:   cosine \u003e= 0.98\n```\n\n### ✅ Production Ready\n```\nTests:    594 passing      ████████\nCoverage: pytest-cov (CI)  ████████\nWarnings: 0                ████████\n```\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n## 🧪 Testing\n\n```bash\n# Run all Python tests\npython -m pytest tests/ -q\n\n# Per-module\npython -m pytest tests/test_v3_api.py -v       # v3 unified API\npython -m pytest tests/test_hnsw.py -v         # HNSW index\npython -m pytest tests/test_pq.py -v           # Product quantization\npython -m pytest tests/test_nf4.py -v          # NF4\npython -m pytest tests/test_binary.py -v       # Binary\npython -m pytest tests/test_rq.py -v           # Residual quantization\npython -m pytest tests/test_storage_v3.py -v   # VQZ format\n\n# Mojo tests\nmojo run tests/run_all_tests.mojo\n```\n\nTest categories:\n- ✅ **Core Ops** — SIMD vector ops (cosine, L2, dot, norm, normalize)\n- ✅ **INT8** — batch quantize/reconstruct, streaming, profiles\n- ✅ **NF4** — level monotonicity, cosine \u003e= 0.985, mixed-precision\n- ✅ **PQ** — codebook training, encode/decode quality, ADC search\n- ✅ **Binary** — pack/unpack, Hamming, matryoshka shapes, search recall\n- ✅ **HNSW** — insert/search, recall@1 \u003e= 0.90, persistence\n- ✅ **GPU** — device detection, roundtrip cosine \u003e= 0.98, top-k\n- ✅ **RQ / Codebook / AutoQuantize** — learned compression quality gates\n- ✅ **VQZ Storage** — magic, checksum, compression round-trips, cloud stubs\n- ✅ **Vector DB** — Qdrant, Weaviate, in-memory round-trip\n- ✅ **Arrow / Parquet** — table export, IPC bytes\n- ✅ **Migration** — v1/v2 upgrade, dry-run, validation\n- ✅ **RC Hardening** — 7 verification gates for release launch\n\n---\n\n## 📖 Documentation\n\n- [docs/getting-started.md](docs/getting-started.md) — Install, quick start, first compression\n- [docs/api-reference.md](docs/api-reference.md) — Full Python API reference (v2 + v3)\n- [docs/integrations.md](docs/integrations.md) — Qdrant, Weaviate, Arrow, Parquet\n- [docs/benchmark-methodology.md](docs/benchmark-methodology.md) — Benchmark methodology\n- [docs/migration-guide.md](docs/migration-guide.md) — v1/v2 to v3 migration\n- [CHANGELOG.md](CHANGELOG.md) — Version history (all 10 v3 phases documented)\n- [PLAN.md](PLAN.md) — Development roadmap and next steps\n\n---\n\n## 🗺️ Roadmap\n\n### v3.0.1 (Released)\n- ✅ Mojo-first runtime: all INT8/NF4/Binary hot paths dispatch to compiled binary\n- ✅ `python/_mojo_bridge.py` — unified subprocess dispatch helper\n- ✅ `pixi run build-mojo` / `selftest` / `benchmark` tasks\n- ✅ NF4 codebook aligned to Python float32 constants (consistent round-trip)\n- ✅ 26 new Mojo dispatch tests (390 passing total)\n\n### v3.0.0 (Released)\n- ✅ SIMD-vectorized quantization\n- ✅ NF4 normal-float 4-bit (cosine \u003e= 0.985)\n- ✅ Product Quantization PQ-96/PQ-48 (32x compression)\n- ✅ Binary / 1-bit quantization (Hamming, Matryoshka)\n- ✅ HNSW approximate nearest-neighbour index\n- ✅ GPU via MAX Engine\n- ✅ Residual Quantization (3-pass)\n- ✅ Autoencoder codebook (48x learned)\n- ✅ AutoQuantize cascade selector\n- ✅ VQZ storage format (ZSTD, checksummed)\n- ✅ Cloud backends (S3, GCS, Azure Blob)\n- ✅ INT4 promoted to GA\n- ✅ 445 tests, 100% coverage\n\n### v3.1.0 (2026-03-11) ✅\n- ✅ Milvus + Chroma connectors\n- ✅ AsyncIO streaming decompressor\n- ✅ `vectro info --benchmark` CLI flag\n- ✅ `pytest-benchmark` integration\n- ✅ Type stubs + `mypy --strict` CI lane\n- ✅ 471 tests, 100% coverage\n\n### v3.2.0 (2026-03-11) ✅\n- ✅ ONNX export for edge inference (opset 17, `vectro export-onnx` CLI)\n- ✅ GPU throughput CI validation (10 CPU-safe equivalence tests + GPU scaffold)\n- ✅ Pinecone connector (`PineconeConnector`)\n- ✅ JavaScript/WASM ADR (`docs/adr-001-javascript-bindings.md`)\n- ✅ 506 tests, 100% coverage\n\n### v3.3.0 (2026-03-11) ✅\n- ✅ Test coverage for `batch_api`, `quality_api`, `profiles_api`, `benchmark` (68 new tests)\n- ✅ ONNX Runtime integration test — `pip install 'vectro[inference]'`\n- ✅ N-API JavaScript scaffold (`js/`, ADR-001 Phase 1 — **not yet callable**, provides type definitions and project structure only)\n- ✅ `inference = [\"onnxruntime\u003e=1.17\"]` optional dep group\n- ✅ 575 tests, 100% module coverage (Python-only mode)\n\n### v3.4.0 (2026-03-12) ✅\n- ✅ `src/auto_quantize_mojo.mojo` — kurtosis-routing auto-quantizer (510 lines)\n- ✅ `src/codebook_mojo.mojo` — INT8 autoencoder (Xavier init, Adam, cosine loss) (710 lines)\n- ✅ `src/rq_mojo.mojo` — Residual Quantizer with K-means++ (583 lines)\n- ✅ `src/migration_mojo.mojo` — VQZ header validation, artifact migration (477 lines)\n- ✅ `src/vectro_api.mojo` — full v3 unified API with ProfileRegistry, QualityEvaluator (626 lines)\n- ✅ `.gitattributes` — `python/**` and `tests/*.py` marked `linguist-generated`; Mojo = **84%** of repo\n- ✅ 575 tests, 100% module coverage\n\n### v3.5.0 (2026-03-12) ✅\n- ✅ Three root-cause fixes: backend mis-labeling, scalar init loops → `resize()`, temp-file IPC → pipe IPC\n- ✅ SIMD_W bumped 4 → 16; `quantize_int8` / `reconstruct_int8` fully vectorised + parallelised\n- ✅ Best-of-5 benchmark (eliminates cold-cache variance)\n- ✅ **INT8 throughput: 12,583,364 vec/s — 4.85× faster than FAISS C++ at d=768**\n- ✅ 575 tests, 100% module coverage\n\n### v3.6.0 (2026-03-12) ✅\n- ✅ NF4 StaticTuple lookup table (O(16)→O(4) binary search) + `parallelize` + vectorized abs-max\n- ✅ SIMD vector accumulator for abs-max (eliminates mid-loop `reduce_max()`)\n- ✅ Binary encode/decode `parallelize` over rows (near-linear core scaling)\n- ✅ Pipe IPC bitcast optimization — `bitcast[UInt8]()` bulk copy replaces element-wise serialization\n- ✅ `vectro_api.mojo` `_int8_compress`/`_int8_decompress` fully vectorized + parallelized\n- ✅ Kurtosis scan restructured row-major (eliminates 3072-byte stride cache misses)\n- ✅ Adam optimizer `_adam_step` vectorized via `vectorize[SIMD_W]`\n- ✅ Codebook training batch buffers pre-allocated once outside epoch loop\n- ✅ `build-mojo-native` pixi task with explicit `--optimization-level 3`\n- ✅ ANN recall@K benchmark (`benchmarks/benchmark_ann_comparison.py`) — Vectro vs hnswlib/annoy/usearch\n- ✅ Real embedding benchmark v2 (`benchmarks/benchmark_real_embeddings_v2.py`) — actual GloVe-100 download\n- ✅ Multi-dimensional INT8 throughput analysis in FAISS comparison benchmark (d=128/384/768/1536)\n- ✅ `bench-ann` optional dep group (hnswlib, annoy, usearch, requests, tqdm)\n- ✅ 598 tests passing\n\n---\n\n## 📝 License\n\nMIT — See [LICENSE](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkonjoai%2Fvectro","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkonjoai%2Fvectro","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkonjoai%2Fvectro/lists"}