{"id":46897494,"url":"https://github.com/tot-ra/vector-bible","last_synced_at":"2026-03-10T23:34:20.818Z","repository":{"id":253459290,"uuid":"843462083","full_name":"tot-ra/vector-bible","owner":"tot-ra","description":"Comparison of popular vector DBs","archived":false,"fork":false,"pushed_at":"2024-09-18T11:06:29.000Z","size":286,"stargazers_count":3,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-09-18T17:09:12.838Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tot-ra.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-16T15:09:31.000Z","updated_at":"2024-09-18T11:06:32.000Z","dependencies_parsed_at":"2024-08-29T14:45:03.931Z","dependency_job_id":"4e7e9d9d-39e7-40dc-a2aa-89ee503f0b52","html_url":"https://github.com/tot-ra/vector-bible","commit_stats":null,"previous_names":["tot-ra/vector-dbs-comparison"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tot-ra/vector-bible","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tot-ra%2Fvector-bible","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tot-ra%2Fvector-bible/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tot-ra%2Fvector-bible/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tot-ra%2Fvector-bible/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tot-ra","download_url":"https://codeload.github.com/tot-ra/vector-bible/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tot-ra%2Fvector-bible/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30362120,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T21:41:54.280Z","status":"ssl_error","status_checked_at":"2026-03-10T21:40:59.357Z","response_time":106,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-03-10T23:34:20.613Z","updated_at":"2026-03-10T23:34:20.796Z","avatar_url":"https://github.com/tot-ra.png","language":"Python","funding_links":[],"categories":["Benchmarks \u0026 Evaluation"],"sub_categories":[],"readme":"## Vector DBs comparison sandbox on a Biblical multilingual verses\n\nGoal of this repo is to compare different vector databases in terms of performance, load,\nease of use and features.\n\nhttps://github.com/user-attachments/assets/a622727e-deb7-4b55-95e2-0642bd6f4763\n\n(above - similarity visualization of 4 books of the New Testament of 21k verses in rus_syn)\n\n## Candidates \u0026 Results\n\nResults are for 1.4M rows dataset if not specified otherwise.\n\n| Nr  | Engine                                                                 | Ports                                                                           | UI  | Stability | Precision (\u003e=0.9 recall) | Insert speed\u003cbr\u003e(avg on 1k batch) | Search (21k rows)    | Search                                                                                  | HDD           | RAM        | Ease of integration 🤯 |\n| --- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------- | --- | --------- | ------------------------ | --------------------------------- | -------------------- | --------------------------------------------------------------------------------------- | ------------- | ---------- | ---------------------- |\n| 2   | [Qdrant 1.11.0](https://github.com/qdrant/qdrant)                      | 6334 [6333](http://localhost:6333/dashboard#/collections)                       | 🟢  | 🟢        | 🟢 8/10                  | 🟢 0.129 sec =\u003e 0.4 sec           | 🟢 0.008 sec         | 🟢 0.031 sec                                                                            | 🟢 4.8 GB     | 🟢 4.73 GB | ★★★★☆                  |\n| 5   | [Weaviate 1.24.22](https://github.com/weaviate/weaviate)               | [8080](http://localhost:8080/v1/schema/Collection_768?_with_meta_count=1) 50051 | 🔴  | 🟢        | 🟡 4/10                  | 🟡 0.411 sec =\u003e 2 sec             | 🟢 0.006 sec         | 🟢 0.010 sec                                                                            | 🟢 8.41 GB    | 🟡 8.16 GB | ★★★☆☆                  |\n| 6   | [Elastic 8.15](https://github.com/elastic/elasticsearch)               | [5601](http://localhost:5601/app/home#/) 9200                                   | 🟢  | 🟢        | 🟢 10/10                 | 🔴 2.917 sec                      | 🟢 0.008 sec         | 🟡 0.20 sec -\u003e 🟢0.011                                                                  | 🔴 23.46 GB   | 🟢 5.1 GB  | ★★★☆☆                  |\n| 7   | [ChromaDB 0.5.5](https://github.com/chroma-core/chroma)                | 8000                                                                            | 🔴  | 🟢        | 🟡 4/10                  | 🔴 1.21 sec =\u003e 4 sec              | 🟢 0.018 sec         | 🟡 1.26 sec -\u003e 🟢0.022 sec                                                              | 🟡 12.37 GB   | 🟢 4.86 GB | ★★★★☆                  |\n| 3   | [Milvus 2.4.8](https://github.com/milvus-io/milvus)                    | 9091 19530 [8000](http://localhost:8000)                                        | 🟢  | 🟡        | 🔴 0/10                  | 🟢 0.118 sec =\u003e 0.4 sec           | 🔴 0.234 sec         | 🟡 0.358 sec                                                                            | 🔴 15 GB      | 🟢 4.59 GB | ★★★☆☆                  |\n| 4   | [Redis stack 7.4](https://github.com/redis/redis)                      | 6379 [8001](http://localhost:8001/)                                             | 🟢  | 🟡        | 🟡 3/10                  | 🔴 0.8 =\u003e 7 (avg 6.482 sec)       | 🟢 0.002 sec         | 🟢 0.003 sec (1.1M dataset, knn), 🟡 0.04 sec (range)                                   | 🟢 6 GB       | 🔴 10.5 GB | ★★★☆☆                  |\n| 1   | Postgres 16.4 + [pgvector 0.7.4](https://github.com/pgvector/pgvector) | 5432                                                                            | 🟡  | 🟢        | 🟡 5/10                  | --                                | 🟡 0.069 sec         | 🔴 6.39 (L1)\u003cbr /\u003e 🔴 5.92 (L2)\u003cbr /\u003e 🔴 6.608 sec (COS)\u003cbr /\u003e 🔴 22.566 sec (no index) | 🟡 11.2 GB \\* | 63 MB      | ★★☆☆☆                  |\n| 8   | [Marqo 2.11](https://github.com/marqo-ai/marqo)                        | 8882                                                                            | 🔴  | -         | N/A                      | 🔴 4.14 sec                       | 🟡 0.19 -\u003e 0.030 sec | N/A                                                                                     | N/A           | 🟢 5 GB    | ★★☆☆☆                  |\n\nI don't take into account cloud-only solutions like\n[Pinecone](https://docs.pinecone.io/guides/get-started/quickstart), [MongoDB Atlas](https://www.mongodb.com/docs/atlas/getting-started/), [SingleStore](https://docs.singlestore.com/cloud/reference/sql-reference/vector-functions/vector-indexing/), [Rockset](https://docs.rockset.com/documentation/docs/vector-search). Maybe [https://benchmark.vectorview.ai](https://benchmark.vectorview.ai/vectordbs.html) helps\n\nI did not have time/energy to also test:\n[Vespa](https://github.com/vespa-engine/vespa),\n[LanceDB](https://github.com/lancedb/lancedb),\n[Clickhouse](https://github.com/ClickHouse/ClickHouse),\n[Cassandra](https://github.com/apache/cassandra)\n\nSee also benchmarks published by each engine (mostly saying they are the best) - [Qdrant](https://qdrant.tech/benchmarks/), [Milvus](https://zilliz.com/vector-database-benchmark-tool?database=Milvus%2CPgVector%2CPinecone\u0026dataset=medium\u0026filter=none%2Clow%2Chigh\u0026tab=3), [Redis](https://redis.io/blog/benchmarking-results-for-vector-databases/)\n\n### Testing Approach\n\nUsed:\n\n- 💻 Mac M3 36GB RAM (Nov 2023), Sonoma 14.1. On your machine, you likely will get different result. Goal is to compare engines between each other on the same machine.\n- 🐍 python 3.11 was used to run test scripts that move the data\n  - I tried to measure API time, not the time it takes to generate embeddings\n- 🐳 docker with 6 CPU and 12.8GB RAM global limits, no per-container limits\n  - single-container dockerized vector databases\n  - While testing, only postgres container (as source) and vector-DB-under-test containers were running to reduce potential CPU and I/O interference\n- Basic test is to load bible text data and compare search performance\n  - I did not use external (OpenAI) APIs for embeddings, but even so, multilingual model for embedding generation was very slow. Thats why I stored it in postgres to not do it in runtime while doing inserts\n  - For 21k dataset, I used `WHERE translationId = 'rus_syn'` to filter out data from postgres that had embeddings. Goal is to see how engines perform initially\n  - For 1.4M dataset I just used rows that had embeddings in postgres by that time. Goal is to test larger scales of data that engine may see in production and how it degrades\n- Most of time is spent on embedding generation (days)\n- Insertion speed - note that it is impacted by md5 hash generation for `id`s if engine needs it. Also, reading a batch from postgres takes time.\n  - `=\u003e` notation means that as dataset grows, insertion speed gets slower\n- Search - I tried to use Cosine similarity with HNSW index\n  - `-\u003e` notation means that new query or if server just started, then speed is \"cold\", but on subsequent queries it is \"warm\" and faster\n  - I tested search after starting a container (cold) and then after it was running for a while (warm)\n  - When testing multiple times, I tried to use the fastest time\n- Storage - I looked at filesystem, not at what engine itself reports in UI (if its able)\n- - Note that for postgres, evaluation in addition to 1.4M rows with embeddings, also had 8M other verses\n- Ease of integration - Subjective metric based on how easy it was to set up based on guide, docs and example code; How intuitive is API, docker setup and UI.\n- RAM - I looked at docker stats memory usage at 1.4M dataset while doing a search.\n\n### Data preparation\n\nFor your tests, I would recommend getting dataset from https://huggingface.co/datasets\n\nI used custom dataset:\n\n- Download SQLite data for bible in different languages\n  https://bible.helloao.org/bible.db (8.4GB)\n- Export `ChapterVerse` from SQLIte to Postgres for better performance. You will need some tool like IntelliJ DataGrip.\n  You could use sqlite but it would be very slow.\n- Add column `ChapterVerse.embedding` with `store.vector(768)` type\n- Create index in postgres for faster updates\n\n```mermaid\nflowchart LR\nsqlite -- \" 0 - import manually in IDE \" --\u003e postgres\npostgres -- \" text \" --\u003e 1-pgvector.py -- \" generate embeddings \" --\u003e postgres[( postgres )]\n```\n\n```\nalter table store.\"ChapterVerse\"\n    add embedding store.vector(768);\n\ncreate index ChapterVerse_translationid_bookid_chapternumber_number_index\n  on store.\"ChapterVerse\" (translationid, bookid, chapternumber, number);\n```\n\n- Pre-Generate embeddings and store them in same `ChapterVerse`.\n  Use multilingual embed model with **768 dim**.\n  https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2\n\n- Spin up vector database you want to test\n\n```\n# https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2\npip install -U sentence-transformers\n\n\n# Generate embeddings into SQLite\n# This will take a while\n# You can re-run it to continue from where it stopped\npython 0-generate-embeddings.py\n```\n\n### 1. Postgres + pgvector\n\n- ✅ Data is stored in Postgres, so no need to sync data between databases\n- 🟡 No specialized UI, but you can reuse postgres-specific ones. I relied on Jetbrains built-in DB plugin, VSCode DB plugin or neovim plugin\n- 🟡 Mediocre search on small dataset\n- 🟡 Operators are not the most intuitive\n- 🟡 Limited activity / community\n- ❌ Slowest search on large dataset\n- ❌ could not install pgvector on Postgres 14 and 15, only version 16 worked\n- ❌ faced `psycopg2.errors.UndefinedFunction: operator does not exist: text \u003c-\u003e vector` when installing extension because\n  operators were installed into public schema instead of `store`. Had to reset the image and set extension installation under `store` schema.\n\n```bash\ndocker-compose -f docker-compose.pgvector.yml up postgres --build\npython -m pip install \"psycopg[binary]\"\n\n# generate embeddings\npython 1-pgvector.py\n```\n\n```sql\nalter table store.\"ChapterVerse\"\n           add embedding vector\n\nalter table store.\"ChapterVerse\"\nalter column embedding type store.vector(768) using embedding::store.vector(768)\n\nCREATE INDEX ON store.\"ChapterVerse\"\n   USING hnsw (embedding store.vector_l2_ops)\n-- [2024-08-23 02:32:41] completed in 36 m 26 s 299 ms\n```\n\n\u003cdetails\u003e\n\u003csummary\u003ePostgres similarity results on 24k dataset\u003c/summary\u003e\n\n```\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226886199645554\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.8717796943695277\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.8707684267530202\nText: Но Христос воскрес из мертвых, первенец из умерших.; Similarity: 0.86272182337587\nText: Так и при воскресении мертвых: сеется в тлении, восстает в нетлении;; Similarity: 0.8626047520415614\nText: и что Он погребен был, и что воскрес в третий день, по Писанию,; Similarity: 0.8371098014647679\nText: Ибо как смерть через человека, так через человека и воскресение мертвых.; Similarity: 0.8319413838804383\nText: которою Он воздействовал во Христе, воскресив Его из мертвых и посадив одесную Себя на небесах,; Similarity: 0.8282566644099042\nText: и гробы отверзлись; и многие тела усопших святых воскресли; Similarity: 0.8217248023128517\nText: быв погребены с Ним в крещении, в Нем вы и совоскресли верою в силу Бога, Который воскресил Его из мертвых,; Similarity: 0.8162701932003219\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ePostgres similarity results on 1.4M dataset, no index\u003c/summary\u003e\n\n```\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.942152166206366\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226886199645554\nText: që në ndonjë mënyrë të mund t’ia arrij ringjalljes prej së vdekurish.; Similarity: 0.9156135526648216\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.9023653865460769\nText: щоб таким чином якось досягти воскресіння з мертвих.; Similarity: 0.9002284663817843\nText: si en alguna manera llegase a la resurrección de los muertos.; Similarity: 0.8967561370447131\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.8730900101269639\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.8717796943695277\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.8707684267530202\nText: abych [tak] snad dospěl ke vzkříšení z mrtvých.; Similarity: 0.8700215089349997\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ePostgres similarity results on 1.4M dataset, HNSW index and Cosine distance (`\u003c=\u003e`)\u003c/summary\u003e\n\n```\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.942152166206366\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226886199645554\nText: që në ndonjë mënyrë të mund t’ia arrij ringjalljes prej së vdekurish.; Similarity: 0.9156135526648216\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.9023653865460769\nText: щоб таким чином якось досягти воскресіння з мертвих.; Similarity: 0.9002284663817843\nText: si en alguna manera llegase a la resurrección de los muertos.; Similarity: 0.8967561370447131\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.8730900101269639\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.8717796943695277\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.8707684267530202\nText: abych [tak] snad dospěl ke vzkříšení z mrtvých.; Similarity: 0.8700215089349997\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ePostgres similarity results on 1.4M dataset, HNSW index and L1 distance (`\u003c+\u003e`)\u003c/summary\u003e\n\n```\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: -15.984977722167969\nText: чтобы достигнуть воскресения мертвых.; Similarity: -17.258892059326172\nText: që në ndonjë mënyrë të mund t’ia arrij ringjalljes prej së vdekurish.; Similarity: -17.903274536132812\nText: щоб таким чином якось досягти воскресіння з мертвих.; Similarity: -19.21749496459961\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: -19.965681076049805\nText: si en alguna manera llegase a la resurrección de los muertos.; Similarity: -22.477447509765625\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: -22.769420623779297\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: -22.937271118164062\nText: а Пачынальніка Жыцьця забілі; Гэтага Бог уваскрэсіў зь мёртвых, і мы сьведкі таго.; Similarity: -23.431766510009766\nText: hogy így eljuthassak a halottak feltámadására.; Similarity: -23.694473266601562\n```\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003ePostgres similarity results on 1.4M dataset, HNSW index and L2 distance (`\u003c-\u003e`)\u003c/summary\u003e\n\n```\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.2030089967677845\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.14195527821053566\nText: që në ndonjë mënyrë të mund t’ia arrij ringjalljes prej së vdekurish.; Similarity: 0.10919599491552112\nText: щоб таким чином якось досягти воскресіння з мертвих.; Similarity: 0.03218261941693723\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.0027483321380983305\nText: si en alguna manera llegase a la resurrección de los muertos.; Similarity: -0.08676286247228848\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: -0.10283884546121591\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: -0.1064100807014805\nText: а Пачынальніка Жыцьця забілі; Гэтага Бог уваскрэсіў зь мёртвых, і мы сьведкі таго.; Similarity: -0.15120845409069927\nText: Так и при воскресении мертвых: сеется в тлении, восстает в нетлении;; Similarity: -0.15640871144354795\n```\n\n\u003c/details\u003e\n\n#### How to visualize embeddings\n\nYou can use [cosmograph](https://cosmograph.app/run/) online tool to visualize nodes and edges.\n\n\u003cdetails\u003e\n\u003csummary\u003eExporting data for visualization\u003c/summary\u003e\n\nExport nodes into CSV:\n\n```sql\nSELECT CONCAT(translationid, '-', bookid, '-', chapternumber, '-', number) as id,\n       text                                                                as label\nFROM store.\"ChapterVerse\"\nWHERE embedding IS NOT NULL\n  AND translationId = 'rus_syn'\n  AND bookid IN ('JHN', 'LUK', 'MRK', 'MAT') LIMIT 10000;\n```\n\nExport edges into CSV:\n\n```sql\nSET\nsearch_path TO store;\nWITH pairwise_similarity\n         AS (SELECT CONCAT(t1.translationid, '-', t1.bookid, '-', t1.chapternumber, '-', t1.number) AS source,\n                    CONCAT(t2.translationid, '-', t2.bookid, '-', t2.chapternumber, '-', t2.number) AS target,\n                    1 - (t1.embedding \u003c=\u003e t2.embedding)                                             AS similarity\n             FROM store.\"ChapterVerse\" t1,\n                  store.\"ChapterVerse\" t2\n             WHERE t1.bookid IN ('JHN', 'LUK', 'MRK', 'MAT')\n               AND t2.bookid IN ('JHN', 'LUK', 'MRK', 'MAT')\n               AND t1.translationId = 'rus_syn'\n               AND t2.translationId = 'rus_syn'\n               AND t1.embedding IS NOT NULL\n               AND t2.embedding IS NOT NULL\n               AND CONCAT(t1.translationid, '-', t1.bookid, '-', t1.chapternumber, '-',\n                          t1.number) != CONCAT(t2.translationid, '-', t2.bookid, '-', t2.chapternumber, '-', t2.number)\n    )\nSELECT source,\n       target,\n       similarity AS weight\nFROM pairwise_similarity\nWHERE similarity \u003e 0.95\nORDER BY weight DESC LIMIT 10000;\n\n```\n\n\u003c/details\u003e\n\n### 2. Qdrant\n\n- ✅ very clear API and docs\n- ✅ fast search\n- ✅ built-in index creation at collection setup\n- ✅ has no-wait / async indexing\n- ✅ good community \u0026 PR activity\n- 🟡 has built-in UI with, but you need to type to search; has embedding visualization\n- 🟡 required entry to have `id`\n- ❌ Failing on starting `ERROR qdrant::startup: Panic occurred in file /qdrant/lib/collection/src/shards/replica_set/mod.rs at line 277: Failed to load local shard \"./storage/collections/collection_768/0\": Service internal error: Not a directory (os error 20)`\n- ❌ Failed at 920k, possibly related to docker?\n\n```\ngrpc._channel._InactiveRpcError: \u003c_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses; last error: UNKNOWN: ipv4:0.0.0.0:6334: Failed to connect to remote host: connect: Connection refused (61)\"\n\tdebug_error_string = \"UNKNOWN:Error received from peer  {grpc_message:\"failed to connect to all addresses; last error: UNKNOWN: ipv4:0.0.0.0:6334: Failed to connect to remote host: connect: Connection refused (61)\", grpc_status:14, created_time:\"2024-08-22T04:34:22.315805+03:00\"}\"\n\u003e\n```\n\n```mermaid\nflowchart LR\n2-qdrant.py -- \" read embeddings \" --\u003e postgres\n2-qdrant.py -- \" write over grpc API \" --\u003e qdrant[( qdrant )]\n```\n\n```\ndocker-compose -f docker-compose.qdrant.yml up qdrant\n\npython -m pip install 'qdrant-client'\n\n# Test Qdrant\npython 2-qdrant.py\n```\n\n\u003cimg width=\"600\" alt=\"Screenshot 2024-08-17 at 02 46 07\" src=\"https://github.com/user-attachments/assets/29068c19-1a2c-41ab-a15f-a0eeb92d3a2a\"\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eQdrant similarity results on 21k dataset\u003c/summary\u003e\n\n```\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226888418197632\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.8717798590660095\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.8707683682441711\nText: Но Христос воскрес из мертвых, первенец из умерших.; Similarity: 0.8627219200134277\nText: Так и при воскресении мертвых: сеется в тлении, восстает в нетлении;; Similarity: 0.8626047968864441\nText: и что Он погребен был, и что воскрес в третий день, по Писанию,; Similarity: 0.8371099233627319\nText: Ибо как смерть через человека, так через человека и воскресение мертвых.; Similarity: 0.8319414258003235\nText: которою Он воздействовал во Христе, воскресив Его из мертвых и посадив одесную Себя на небесах,; Similarity: 0.8282566666603088\nText: и гробы отверзлись; и многие тела усопших святых воскресли; Similarity: 0.8217248916625977\nText: быв погребены с Ним в крещении, в Нем вы и совоскресли верою в силу Бога, Который воскресил Его из мертвых,; Similarity: 0.8162703514099121\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eQdrant similarity results on 1.4M dataset\u003c/summary\u003e\n\n```\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.9421520829200745\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.9421520829200745\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.9421520829200745\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226888418197632\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226888418197632\nText: që në ndonjë mënyrë të mund t’ia arrij ringjalljes prej së vdekurish.; Similarity: 0.9156136512756348\nText: që në ndonjë mënyrë të mund t’ia arrij ringjalljes prej së vdekurish.; Similarity: 0.9156136512756348\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.9023655652999878\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.8730900287628174\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.8730900287628174\n```\n\n\u003c/details\u003e\n\n### 3. Milvus\n\n- ✅ Docs look impressive\n- ✅ good community \u0026 PR activity\n- 🟡 Has concept of \"loading\" and unloading a collection (from memory)\n- 🟡 Takes ~20 sec to start up (compared to others)\n- 🟡 Milvus does not come with built-in UI, so we use `attu` for that.\n- 🟡 Has extra containers\n- ❌ Large usage of Storage. minio/insert_log looks suspicious.\n- ❌ Search was slow, even though it used an index (maybe I did something wrong?)\n- ❌ Failed at insertion @ 683k and second attempt at @ 989k. No errors in milvus logs. UI and client just loose connection and server is not responsive. Becomes accessible after restart. Maybe related to out of memory?\n- Succeeded on third attempt\n\n```pymilvus.exceptions.MilvusException: \u003cMilvusException: (code=\u003cbound method _MultiThreadedRendezvous.code of \u003c_MultiThreadedRendezvous of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:19530: Failed to connect to remote host: connect: Connection refused (61)\"\n\tdebug_error_string = \"UNKNOWN:Error received from peer  {grpc_message:\"failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:19530: Failed to connect to remote host: connect: Connection refused (61)\", grpc_status:14, created_time:\"2024-08-22T04:07:48.480463+03:00\"}\"\n\u003e\u003e, message=Retry run out of 75 retry times, message=failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:19530: Failed to connect to remote host: connect: Connection refused (61))\u003e\n\npymilvus.exceptions.MilvusException: \u003cMilvusException: (code=2, message=Fail connecting to server on localhost:19530, illegal connection params or server unavailable)\u003e\n```\n\nSome suspicious logs:\n\n```\nmilvus-standalone  | [2024/08/22 19:51:46.460 +00:00] [WARN] [checkers/index_checker.go:139] [\"failed to get indexInfo for segment\"] [collectionID=452028396578800432] [segmentID=452028396580013227] [error=\"index not found[segmentID=452028396580013227]\"]\nmilvus-standalone  | [2024/08/22 19:51:46.919 +00:00] [INFO] [datacoord/task_scheduler.go:214] [\"there is no idle indexing node, wait a minute...\"]\n```\n\n```mermaid\nflowchart LR\n3-milvus.py -- \" write over grpc API \" --\u003e milvus[( milvus )]\n3-milvus.py -- \" read embeddings \" --\u003e postgres\natto[\"atto\\nlocalhost:8000\"] --\u003e milvus\n```\n\n```bash\npython -m pip install pymilvus\ndocker-compose -f docker-compose.milvus.yml up\n```\n\n\u003cimg width=\"1420\" alt=\"Screenshot 2024-08-24 at 14 06 47\" src=\"https://github.com/user-attachments/assets/253e285a-9434-46eb-96d8-8799517e48b8\"\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eMilvus similarity results on 21k dataset\u003c/summary\u003e\n\n```\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226888418197632\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.8717797994613647\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.8707685470581055\nText: Но Христос воскрес из мертвых, первенец из умерших.; Similarity: 0.8627219796180725\nText: Так и при воскресении мертвых: сеется в тлении, восстает в нетлении;; Similarity: 0.8626042604446411\nText: и что Он погребен был, и что воскрес в третий день, по Писанию,; Similarity: 0.8371096253395081\nText: Ибо как смерть через человека, так через человека и воскресение мертвых.; Similarity: 0.831940770149231\nText: которою Он воздействовал во Христе, воскресив Его из мертвых и посадив одесную Себя на небесах,; Similarity: 0.8282568454742432\nText: и гробы отверзлись; и многие тела усопших святых воскресли; Similarity: 0.8217248320579529\nText: быв погребены с Ним в крещении, в Нем вы и совоскресли верою в силу Бога, Который воскресил Его из мертвых,; Similarity: 0.8162701725959778\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eMilvus similarity results on 1.4M dataset\u003c/summary\u003e\n\n```\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.8730900883674622\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.8730900883674622\nText: मृतानां पुनरुत्थिति र्व्यतीतेति वदन्तौ केषाञ्चिद् विश्वासम् उत्पाटयतश्च।; Similarity: 0.8565160632133484\nText: אשר פעל במשיח בהעיר אתו מן המתים ויושיבנו לימינו במרומים׃; Similarity: 0.8483075499534607\nText: του ορισθεντος υιου θεου εν δυναμει κατα πνευμα αγιωσυνης εξ αναστασεως νεκρων ιησου χριστου του κυριου ημων; Similarity: 0.8435214757919312\nText: του ορισθεντος υιου θεου εν δυναμει κατα πνευμα αγιωσυνης εξ αναστασεως νεκρων ιησου χριστου του κυριου ημων; Similarity: 0.8435214757919312\nText: και οντας ημας νεκρους τοις παραπτωμασιν συνεζωοποιησεν τω χριστω χαριτι εστε σεσωσμενοι; Similarity: 0.8367748260498047\nText: ευλογητος ο θεος και πατηρ του κυριου ημων ιησου χριστου ο κατα το πολυ αυτου ελεος αναγεννησας ημας εις ελπιδα ζωσαν δι αναστασεως ιησου χριστου εκ νεκρων; Similarity: 0.8299326300621033\nText: которою Он воздействовал во Христе, воскресив Его из мертвых и посадив одесную Себя на небесах,; Similarity: 0.8282567262649536\nText: 死而复生所展现的惊人能力。上帝在天上将基督安置在他的右手边，; Similarity: 0.8202930092811584\n```\n\n\u003c/details\u003e\n\n### 4. Redis\n\n- ✅ Fastest search. But its KNN based, so not that precise.\n- ✅ As we use redis-stack, it came with redis-insight UI bundled. UI is nice, but not vector-specific. Can't see indexes or visualize embeddings.\n  - Open UI at `http://localhost:8001/`, use `pass` as password\n  - Use bulk actions to clean up data if needed\n- ✅ Insertion batching using `pipeline`\n- 🟡 Single CPU bound seems like a bottleneck for inserts\n- 🟡 High RAM usage\n- 🟡 API/Command syntax was not intuitive, had to spend too much time reverse-engineering it from docs and examples.\n  `redis.exceptions.ResponseError: Property vector_score not loaded nor in schema` while trying to search - index and query need to match\n- 🟡 `unknown command 'JSON.SET'` while using `redis` image, likely related to JSON extension, had to switch to `redis-stack` image.\n- 🟡 custom license\n- 🟡 docs are confusing\n- 🟡 `MISCONF Redis is configured to save RDB snapshots, but it's currently unable to persist to disk` while deleting keys\n- ❌ slow insert speed, gets worse as amount of data grows 1.3 sec -\u003e 7 sec per minibatch (1k rows). Maybe it has to do with the way embeddings are passed?\n- note that JSON -\u003e HSET switch and pipeline fix did not improve the speed much\n- CPU on redis side was 100% loaded as observed from docker\n- ❌ On first Redis failed to ingest all rows (maybe I did some misconfiguration?).\n  `redis.exceptions.BusyLoadingError: Redis is loading the dataset in memory` random error while loading dataset at 336K rows and 8.6GB of memory;\n- ❌ Second attempt - failed with `SERVER_CLOSED_CONNECTION_ERROR` in pipeline.execute(). UI cannot connect to server anymore even if it looks running\n- ❌ Search was slow, even though it used an index (maybe I did something wrong?)\n\nImprovements and third attempt:\n\n- ✅ Storage: JSON.set was changed -\u003e HSET\n- Insertion: Reusing pipeline to insert data\n- ✅ Search: Range query was changed -\u003e KNN.\n- ✅ New attempt- succeeded with 1M rows. Insert speeds at times reached 10 sec per 1k rows (bad).\n- 🟡 I also did experience \u003e 5 insert disruptions while inserting after 1M rows, effectively breaking insertion, but there were no error logs or exceptions that would explain why.\n  The script just ended insertion but still performed search (both operations are in the same script). Likely related to something with my script or setup again. Maybe missing error catching or pipeline handling.\n  After encountering this, I adjusted offset to continue insertion from position that was left off.\n- ❌ Failed at 1.33M rows, likely due to memory limits (see logs below). After restarting, I couldn't connect to the server again even after restarting container.\n\n\u003cdetails\u003e\n\u003csummary\u003eLogs\u003c/summary\u003e\n\ndocker logs before failure:\n\n```bash\nredis-1 | 498:C 20 Sep 2024 09:51:56.499 _ DB saved on disk\nredis-1 | 498:C 20 Sep 2024 09:51:56.641 _ Fork CoW for RDB: current 1108 MB, peak 1108 MB, average 609 MB\nredis-1 | 9:M 20 Sep 2024 09:51:56.935 _ Background saving terminated with success\nredis-1 | 9:M 20 Sep 2024 09:52:57.046 _ 10000 changes in 60 seconds. Saving...\nredis-1 | 9:M 20 Sep 2024 09:52:57.136 \\* Background saving started by pid 501\nredis-1 | 9:M 20 Sep 2024 09:53:43.928 # Can't fork for module: File exists\nredis-1 | 9:M 20 Sep 2024 09:53:43.928 # \u003cmodule\u003e fork failed - got errno 17, aborting fork GC\nredis-1 | 9:M 20 Sep 2024 09:53:55.054 # Can't fork for module: File exists\nredis-1 | 9:M 20 Sep 2024 09:53:55.055 # \u003cmodule\u003e fork failed - got errno 17, aborting fork GC\nredis-1 | 9:M 20 Sep 2024 09:54:07.963 # Can't fork for module: File exists\nredis-1 | 9:M 20 Sep 2024 09:54:07.963 # \u003cmodule\u003e fork failed - got errno 17, aborting fork GC\nredis-1 | Killed\nredis-1 | 501:signal-handler (1726826059) Received SIGTERM scheduling shutdown...\nredis-1 exited with code 0\n```\n\npython client failures:\n```python\nTraceback (most recent call last):\n  File \"/Users/artjom/git/vector-dbs-comparison/4-redis.py\", line 121, in \u003cmodule\u003e\n    read_verses(\n  File \"/Users/artjom/git/vector-dbs-comparison/common.py\", line 83, in read_verses\n    elapsed_time += handler(chunk, pipeline)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/artjom/git/vector-dbs-comparison/4-redis.py\", line 71, in redis_inserts\n    pipeline.execute()\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/client.py\", line 1503, in execute\n    return conn.retry.call_with_retry(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/retry.py\", line 65, in call_with_retry\n    fail(error)\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/client.py\", line 1505, in \u003clambda\u003e\n    lambda error: self._disconnect_raise_reset(conn, error),\n                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/client.py\", line 1481, in _disconnect_raise_reset\n    raise error\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/retry.py\", line 62, in call_with_retry\n    return do()\n           ^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/client.py\", line 1504, in \u003clambda\u003e\n    lambda: execute(conn, stack, raise_on_error),\n            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/client.py\", line 1412, in _execute_pipeline\n    response.append(self.parse_response(connection, args[0], **options))\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/client.py\", line 1435, in parse_response\n    result = Redis.parse_response(self, connection, command_name, **options)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/client.py\", line 565, in parse_response\n    response = connection.read_response()\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/connection.py\", line 512, in read_response\n    response = self._parser.read_response(disable_decoding=disable_decoding)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/_parsers/resp2.py\", line 15, in read_response\n    result = self._read_response(disable_decoding=disable_decoding)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/_parsers/resp2.py\", line 25, in _read_response\n    raw = self._buffer.readline()\n          ^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/_parsers/socket.py\", line 115, in readline\n    self._read_from_socket()\n  File \"/opt/homebrew/lib/python3.11/site-packages/redis/_parsers/socket.py\", line 68, in _read_from_socket\n    raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)\nredis.exceptions.ConnectionError: Connection closed by server.\n```\n\n\u003c/details\u003e\n\n```bash\ndocker-compose -f docker-compose.redis.yml up\npython 4-redis.py\n```\n\nDocs:\nhttps://redis-py.readthedocs.io/en/stable/examples/search_vector_similarity_examples.html\n\n\u003cimg width=\"600\" alt=\"Screenshot 2024-08-21 at 16 19 02\" src=\"https://github.com/user-attachments/assets/1c1c97c6-aba4-4282-bf06-f9e6dcdcd85e\"\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eRedis similarity results on 21k dataset with KNN query 0.002 sec\u003c/summary\u003e\n\n```\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.92\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.87\nText: которою Он воздействовал во Христе, воскресив Его из мертвых и посадив одесную Себя на небесах,; Similarity: 0.83\nText: быв погребены с Ним в крещении, в Нем вы и совоскресли верою в силу Бога, Который воскресил Его из мертвых,; Similarity: 0.82\nText: Посему сказано: “встань, спящий, и воскресни из мертвых, и осветит тебя Христос”.; Similarity: 0.79\nText: и открылся Сыном Божиим в силе, по духу святыни, через воскресение из мертвых, о Иисусе Христе Господе нашем,; Similarity: 0.79\nText: и воскресил с Ним, и посадил на небесах во Христе Иисусе,; Similarity: 0.78\nText: Который предан за грехи наши и воскрес для оправдания нашего.; Similarity: 0.77\nText: Благословен Бог и Отец Господа нашего Иисуса Христа, по великой Своей милости возродивший нас воскресением Иисуса Христа из мертвых к упованию живому,; Similarity: 0.76\nText: умершего за нас, чтобы мы бодрствуем ли, или спим, жили вместе с Ним.; Similarity: 0.75\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eRedis similarity results on 21k dataset with vector range query (0.04 sec)\u003c/summary\u003e\n\n```\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.92\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.87\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.87\nText: Но Христос воскрес из мертвых, первенец из умерших.; Similarity: 0.86\nText: Так и при воскресении мертвых: сеется в тлении, восстает в нетлении;; Similarity: 0.86\nText: и что Он погребен был, и что воскрес в третий день, по Писанию,; Similarity: 0.84\nText: Ибо как смерть через человека, так через человека и воскресение мертвых.; Similarity: 0.83\nText: которою Он воздействовал во Христе, воскресив Его из мертвых и посадив одесную Себя на небесах,; Similarity: 0.83\nText: и гробы отверзлись; и многие тела усопших святых воскресли; Similarity: 0.82\nText: быв погребены с Ним в крещении, в Нем вы и совоскресли верою в силу Бога, Который воскресил Его из мертвых,; Similarity: 0.82\n```\n\n\u003c/details\u003e\n\n\n\n\u003cimg width=\"1656\" alt=\"Screenshot 2024-09-20 at 12 54 29\" src=\"https://github.com/user-attachments/assets/4a36d308-89da-41f2-b88c-2f3d6a707263\"\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eRedis similarity results on 1.1/1.3M dataset with KNN query\u003c/summary\u003e\n\n```\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.94\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.92\nText: щоб таким чином якось досягти воскресіння з мертвих.; Similarity: 0.9\nText: per giungere in qualche modo alla risurrezione dei morti.; Similarity: 0.89\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.87\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.87\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.87\nText: if any way I arrive at the resurrection from among [the] dead.; Similarity: 0.86\nText: येन केनचित् प्रकारेण मृतानां पुनरुत्थितिं प्राप्तुं यते।; Similarity: 0.86\nText: যীশু খ্রীষ্টকে স্মরণ করো, যিনি মৃতলোক থেকে পুনরুত্থিত হয়েছেন এবং যিনি দাউদের বংশজাত। এই হল আমার সুসমাচার।; Similarity: 0.83\n```\n\u003c/details\u003e\n\n\n\n### 5. Weaviate\n\n```bash\ndocker-compose -f docker-compose.weaviate.yml up weaviate\n````\n\n- ✅ Fast search\n- ✅ Lots of docs, Multitenancy, Replication\n- 🟡 But Docs are confusing, emphasize cloud or older client versions, emphasizes OpenAI embeddings and configs instead of custom ones\n- 🟡 API at times confusing - `Failed to send 20 objects in a batch of 20. Please inspect client.batch.failed_objects or collection.batch.failed_objects for the failed objects` instead of showing errors.\n  - Did not like `id` or `vector` in properties\n- ❌ slow insert speed gets worse as amount of data grows 0.4 sec -\u003e 2.5 sec\n- ❌ Has no management UI, only API\n\n\u003cimg width=\"600\" alt=\"Screenshot 2024-08-22 at 13 02 23\" src=\"https://github.com/user-attachments/assets/0ae745e2-8c33-43e6-a60d-88d0d546b654\"\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eWeaviate similarity results on 21k dataset\u003c/summary\u003e\n\n```\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226889610290527\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.8717796802520752\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.8707684278488159\nText: Но Христос воскрес из мертвых, первенец из умерших.; Similarity: 0.862721860408783\nText: Так и при воскресении мертвых: сеется в тлении, восстает в нетлении;; Similarity: 0.8626049160957336\nText: и что Он погребен был, и что воскрес в третий день, по Писанию,; Similarity: 0.8371100425720215\nText: Ибо как смерть через человека, так через человека и воскресение мертвых.; Similarity: 0.8319416046142578\nText: которою Он воздействовал во Христе, воскресив Его из мертвых и посадив одесную Себя на небесах,; Similarity: 0.8282566666603088\nText: и гробы отверзлись; и многие тела усопших святых воскресли; Similarity: 0.8217248320579529\nText: быв погребены с Ним в крещении, в Нем вы и совоскресли верою в силу Бога, Который воскресил Его из мертвых,; Similarity: 0.8162702322006226\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eWeaviate similarity results on 1.4M dataset\u003c/summary\u003e\n\n```\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.9421523809432983\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.9421523809432983\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226889610290527\nText: që në ndonjë mënyrë të mund t’ia arrij ringjalljes prej së vdekurish.; Similarity: 0.9156137108802795\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.8730896711349487\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.8730896711349487\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.8717796802520752\nText: abych [tak] snad dospěl ke vzkříšení z mrtvých.; Similarity: 0.8700212836265564\nText: abych [tak] snad dospěl ke vzkříšení z mrtvých.; Similarity: 0.8700212836265564\nText: abych [tak] snad dospěl ke vzkříšení z mrtvých.; Similarity: 0.8700212836265564\n```\n\n\u003c/details\u003e\n\n### 6. Elastic\n\n- ✅ Has very nice kibana UI, highlighting indexes and properties used\n- 🟡 Custom license, had to use `basic` for testing\n- ❌ Has strict security `fatal exception while booting Elasticsearch: cannot read configured PEM certificate_authorities`, had to disable SSL and other security checks\n\n```bash\ndocker-compose -f docker-compose.elastic.yml up\n```\n\n```mermaid\nflowchart LR\nkibana --\"render ui\"--\u003e elasticsearch\n6-elastic.py -- \" read \" --\u003e postgres\n6-elastic.py -- \" insert \u0026 search \" --\u003e elasticsearch\n```\n\n\u003cimg width=\"1698\" alt=\"Screenshot 2024-08-22 at 22 57 10\" src=\"https://github.com/user-attachments/assets/d4c3a883-9acb-4ff4-b31c-bc9b5e5e5c1e\"\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eElastic similarity results on 21k dataset\u003c/summary\u003e\n\n```\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.95788\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.9358952\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.9356406\nText: Но Христос воскрес из мертвых, первенец из умерших.; Similarity: 0.93155193\nText: Так и при воскресении мертвых: сеется в тлении, восстает в нетлении;; Similarity: 0.9310014\nText: и что Он погребен был, и что воскрес в третий день, по Писанию,; Similarity: 0.9184923\nText: Ибо как смерть через человека, так через человека и воскресение мертвых.; Similarity: 0.91571283\nText: которою Он воздействовал во Христе, воскресив Его из мертвых и посадив одесную Себя на небесах,; Similarity: 0.914495\nText: и гробы отверзлись; и многие тела усопших святых воскресли; Similarity: 0.91006994\nText: быв погребены с Ним в крещении, в Нем вы и совоскресли верою в силу Бога, Который воскресил Его из мертвых,; Similarity: 0.9080174\n\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eElastic similarity results on 1.4M dataset\u003c/summary\u003e\n\n```\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9605174\nText: që në ndonjë mënyrë të mund t’ia arrij ringjalljes prej së vdekurish.; Similarity: 0.95751595\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.95071626\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.95071626\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.95071626\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.95071626\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.95071626\nText: щоб таким чином якось досягти воскресіння з мертвих.; Similarity: 0.9491937\nText: щоб таким чином якось досягти воскресіння з мертвих.; Similarity: 0.9491937\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.9359567\n\n```\n\n\u003c/details\u003e\n\n### 7. ChromaDB\n\n- ✅ Very straightforward quickstart guide\n- 🟡 API is confusing as it looks like column DB\n- ❌ Because of column API, one-by-one insertion speed is very slow. Likely could be improved though if multiple columns are first changed.\n- ❌ `TypeError: Descriptors cannot be created directly in chromadb.telemetry.opentelemetry `, had to set `PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python` env var\n- ❌ [No batching support](https://cookbook.chromadb.dev/strategies/batching/#creating-batches)\n- ❌ `chromadb.api.configuration.InvalidConfigurationError: batch_size must be less than or equal to sync_threshold` when trying to `client.get_or_create_collection` due to mismatching client and server versions\n\n```bash\ndocker-compose -f docker-compose.chromadb.yml up\n```\n\n\u003cimg width=\"600\" alt=\"Screenshot 2024-08-22 at 13 17 23\" src=\"https://github.com/user-attachments/assets/dc5daa48-4ff6-448c-aac7-3e3b3925c115\"\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eChromaDB similarity results on 21k dataset\u003c/summary\u003e\n\n```\nText: чтобы достигнуть воскресения мертвых.; Similarity: 0.9226889610290527\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.8717796206474304\nText: а Начальника жизни убили. Сего Бог воскресил из мертвых, чему мы свидетели.; Similarity: 0.8707686066627502\nText: Но Христос воскрес из мертвых, первенец из умерших.; Similarity: 0.8627220392227173\nText: Так и при воскресении мертвых: сеется в тлении, восстает в нетлении;; Similarity: 0.8626042008399963\nText: и что Он погребен был, и что воскрес в третий день, по Писанию,; Similarity: 0.8371098637580872\nText: Ибо как смерть через человека, так через человека и воскресение мертвых.; Similarity: 0.8319410681724548\nText: которою Он воздействовал во Христе, воскресив Его из мертвых и посадив одесную Себя на небесах,; Similarity: 0.8282561898231506\nText: и гробы отверзлись; и многие тела усопших святых воскресли; Similarity: 0.8217248320579529\nText: быв погребены с Ним в крещении, в Нем вы и совоскресли верою в силу Бога, Который воскресил Его из мертвых,; Similarity: 0.8162698745727539\n```\n\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eChromaDB similarity results on 1.4M dataset\u003c/summary\u003e\n\n```\nText: a fin de llegar a la resurrección de entre los muertos.; Similarity: 0.9421521425247192\nText: që në ndonjë mënyrë të mund t’ia arrij ringjalljes prej së vdekurish.; Similarity: 0.9156137108802795\nText: om eenmaal te kunnen komen tot de opstanding uit de doden.; Similarity: 0.9023656249046326\nText: щоб таким чином якось досягти воскресіння з мертвих.; Similarity: 0.9002282023429871\nText: si en alguna manera llegase a la resurrección de los muertos.; Similarity: 0.8967559337615967\nText: अपरं स्मुर्णास्थसमिते र्दूतं प्रतीदं लिख; य आदिरन्तश्च यो मृतवान् पुनर्जीवितवांश्च तेनेदम् उच्यते,; Similarity: 0.8730899095535278\nText: Но Бог воскресил Его из мертвых.; Similarity: 0.8717796206474304\nText: abych [tak] snad dospěl ke vzkříšení z mrtvých.; Similarity: 0.870021402835846\nText: hogy így eljuthassak a halottak feltámadására.; Similarity: 0.8691048622131348\nText: hogy így eljuthassak a halottak feltámadására.; Similarity: 0.8691048622131348\n```\n\n\u003c/details\u003e\n\n### 8. Marqo\n\n- Comes with built-in models like `hf/e5-base-v2` and `open_clip/ViT-B-32/laion2b_s34b_b79k`\n- 🟡 No management UI, only API\n- 🟡 API and docs are confusing, emphasize built-in embedding models, was hard to inject **custom** embeddings, hard to create index with proper schema and use search. Had issues like:\n  `marqo.errors.MarqoWebError: MarqoWebError: MarqoWebError Error message: {'message': '[{\"loc\": [\"__root__\"], \"msg\": \"Field \\'custom\\' has type \\'custom_vector\\' and must be a tensor field.`\n  `marqo.errors.MarqoWebError: MarqoWebError: MarqoWebError Error message: {'message': \"Cannot specify 'tensorFields' when adding documents to a structured index. 'tensorFields' must be defined in structured index schema at index creation time\",`\n\n- ❌ does not support 1k batch, only 128, had to lower it but it impacts insert speed. `marqo.errors.MarqoWebError: MarqoWebError: MarqoWebError Error message: {'message': 'Number of docs in add documents request (1000) exceeds limit of 128. If using the Python client, break up your `add_documents`request into smaller batches using its`client_batch_size` parameter.`\n  - I used CPU-based indexing. Docs and other guides say that GPU-based is faster\n- ❌ was unable to use docker volumes, got: `marqo.errors.MarqoWebError: MarqoWebError: MarqoWebError Error message: {'message': \"Marqo cannot connect to Zookeeper`\n- ❌ with pure docker image, I couldn't go past 24k, I received errors like:\n  - `Marqo vector store is out of memory or disk space`, also may be because of some docker configs?\n  - `marqo-1  | marqo.vespa.exceptions.VespaStatusError: 507: {\"pathId\":\"/document/v1/marqo__settings/marqo__settings/docid/collection_768\",\"id\":\"id:marqo__settings:marqo__settings::collection_768\",\"message\":\"[UNKNOWN(251009) @ tcp/9cbb296c4e04:19112/default]: ReturnCode(NO_SPACE, External feed is blocked due to resource exhaustion: in content cluster 'content_default': disk on node 0 [9cbb296c4e04] is 75.9% full (the configured limit is 75.0%). See https://docs.vespa.ai/en/operations/feed-block.html) \"}`\n\n\u003cimg width=\"1226\" alt=\"Screenshot 2024-08-24 at 14 50 31\" src=\"https://github.com/user-attachments/assets/d4f83f01-5047-4d66-a1f1-2fd8279725ca\"\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eMarqo similarity results on 21k dataset\u003c/summary\u003e\n\n```\nText: Посему сказано: “встань, спящий, и воскресни из мертвых, и осветит тебя Христос”.; Similarity: 1.0\nText: Ибо, как Отец воскрешает мертвых и оживляет, так и Сын оживляет, кого хочет.; Similarity: 1.0\nText: Ибо если мы соединены с Ним подобием смерти Его, то должны быть соединены и подобием воскресения,; Similarity: 1.0\nText: И они удержали это слово, спрашивая друг друга, что значит: воскреснуть из мертвых.; Similarity: 1.0\nText: Но Бог воскресил Его из мертвых.; Similarity: 1.0\nText: и сказал служащим при нем: это Иоанн Креститель; он воскрес из мертвых, и потому чудеса делаются им.; Similarity: 1.0\nText: Если же о Христе проповедуется, что Он воскрес из мертвых, то как некоторые из вас говорят, что нет воскресения мертвых?; Similarity: 1.0\nText: Если нет воскресения мертвых, то и Христос не воскрес;; Similarity: 1.0\nText: Но Христос воскрес из мертвых, первенец из умерших.; Similarity: 1.0\nText: Но скажет кто-нибудь: как воскреснут мертвые? И в каком теле придут?; Similarity: 1.0\n```\n\n\u003c/details\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftot-ra%2Fvector-bible","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftot-ra%2Fvector-bible","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftot-ra%2Fvector-bible/lists"}