{"id":13504390,"url":"https://github.com/firstbatchxyz/hollowdb-vector","last_synced_at":"2025-05-03T17:33:53.186Z","repository":{"id":214615849,"uuid":"624505911","full_name":"firstbatchxyz/hollowdb-vector","owner":"firstbatchxyz","description":"A decentralized vector database for building vector search applications","archived":false,"fork":false,"pushed_at":"2024-02-16T09:05:16.000Z","size":893,"stargazers_count":88,"open_issues_count":1,"forks_count":6,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-05-21T06:05:06.854Z","etag":null,"topics":["ai","ann","arweave","blockchain","decentralized","embedding","embeddings","nearest-neighbor-search","vector-database"],"latest_commit_sha":null,"homepage":"https://firstbatch.xyz","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/firstbatchxyz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-06T16:07:45.000Z","updated_at":"2024-08-01T02:25:14.378Z","dependencies_parsed_at":"2024-01-03T15:51:00.421Z","dependency_job_id":"790a09df-35c0-4f87-9005-ffed470976e0","html_url":"https://github.com/firstbatchxyz/hollowdb-vector","commit_stats":null,"previous_names":["firstbatchxyz/danny","firstbatchxyz/hollowdb-vector"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firstbatchxyz%2Fhollowdb-vector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firstbatchxyz%2Fhollowdb-vector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firstbatchxyz%2Fhollowdb-vector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firstbatchxyz%2Fhollowdb-vector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/firstbatchxyz","download_url":"https://codeload.github.com/firstbatchxyz/hollowdb-vector/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252227204,"owners_count":21714967,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ann","arweave","blockchain","decentralized","embedding","embeddings","nearest-neighbor-search","vector-database"],"created_at":"2024-08-01T00:00:36.343Z","updated_at":"2025-05-03T17:33:52.797Z","avatar_url":"https://github.com/firstbatchxyz.png","language":"TypeScript","readme":"\u003cp align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003e\n    HollowDB Vector\n  \u003c/h1\u003e\n\n  \u003cdiv align=\"center\"\u003e\n    \u003ca href=\"https://www.npmjs.com/package/hollowdb-vector\" target=\"_blank\"\u003e\n      \u003cimg alt=\"NPM\" src=\"https://img.shields.io/npm/v/hollowdb-vector?logo=npm\u0026color=CB3837\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"./.github/workflows/tests.yml\" target=\"_blank\"\u003e\n        \u003cimg alt=\"Workflow: Tests\" src=\"https://github.com/firstbatchxyz/hollowdb-vector/actions/workflows/tests.yml/badge.svg?branch=master\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://opensource.org/license/apache-2-0/\" target=\"_blank\"\u003e\n        \u003cimg alt=\"License: Apache 2.0\" src=\"https://img.shields.io/badge/license-Apache%202.0-7CB9E8.svg\"\u003e\n    \u003c/a\u003e\n  \u003c/div\u003e\n  \n  \u003cp align=\"center\"\u003e\n    \u003ci\u003eImplementation of Hierarchical Navigable Small Worlds (HNSW) index over \u003ca href=\"https://hollowdb.xyz/\" target=\"_blank\"\u003eHollowDB\u003c/a\u003e.\u003c/i\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\n## Installation\n\nInstall the package:\n\n```sh\nyarn add hollowdb-vector\npnpm add hollowdb-vector\nnpm install hollowdb-vector\n```\n\n## Usage\n\nYou can create the VectorDB as follows:\n\n```ts\nimport HollowDBVector from \"hollowdb-vector\";\nimport { WarpFactory, defaultCacheOptions } from \"warp-contracts\";\nimport { SetSDK } from \"hollowdb\";\nimport { Redis } from \"ioredis\";\nimport { RedisCache } from \"warp-contracts-redis\";\n\n// connect to Redis\nconst redis = new Redis();\n\n// create Warp instance with Redis cache\nconst warp = WarpFactory.forMainnet().useKVStorageFactory(\n  (contractTxId: string) =\u003e\n    new RedisCache({ ...defaultCacheOptions, dbLocation: `${contractTxId}` }, { client: redis }),\n);\n\n// create HollowDB SDK\nconst wallet = JSON.parse(readFileSync(\"./path/to/wallet.json\", \"utf-8\"));\nconst contractTxId = \"your-contract-tx-id\";\nconst hollowdb = new SetSDK\u003cstring\u003e(wallet, contractTxId, warp);\n\n// create HollowDB Vector\nconst vectordb = new HollowDBVector(hollowdb);\n```\n\n### Inserting a Vector\n\nWith this, you can insert a new point:\n\n```ts\nconst point = [-0.28571999073028564 /* and many more... */, 0.13964000344276428];\n\n// any object\nconst metadata = {\n  name: \"My favorite vector!\",\n};\n\n// insert a point\nawait vectordb.insert(point, metadata);\n```\n\nMetadata is optional, and you can leave it out during `insert`. If you would like to set it a later time, you can always do:\n\n```ts\nvectordb.db.set_metadata(index, metadata);\n```\n\n\u003e [!NOTE]\n\u003e\n\u003e The complexity of inserting a point may increase with more points in the DB.\n\n### Fetching a Vector\n\nYou can get a vector by its index, which returns its point value and metadata:\n\n```ts\nconst { point, metadata } = await vectordb.get_vector(index);\n```\n\n### Querying a Vector\n\nYou can make a query and return top K relevant results:\n\n```ts\n// a query point\nconst query = [-0.28571999073028564 /* and many more... */, 0.13964000344276428];\n\n// number of top results to return\nconst K = 10;\n\n// make a KNN search\nconst results = await vectordb.knn_search(query, K);\n\n// each result contains the vector id, its distance to query, and metadata\nconst { id, distance, metadata } = results[0];\n```\n\n### Deploying your own Contract\n\nHollowDB Vector exports a static function that allows you two deploy a new contract that you own. Assuming that you have a `wallet` and a `warp` instance as described [above](#usage), you can create a new contract with:\n\n```ts\nconst { contractTxId } = await HollowDBVector.deploy(wallet, warp);\nconsole.log(\"Deployed at:\", contractTxId);\n```\n\n## Setup\n\nFor local setup of this repo, first clone it.\n\n```sh\ngit clone https://github.com/firstbatchxyz/hollowdb-vector\n```\n\nThen, install packages:\n\n```sh\npnpm install\n```\n\nPeer dependencies should be installed automatically.\n\n### Protobuffers\n\nWe include the pre-compiled protobuf within the repo, but if you were to change the protobuf later, you can generate the compiled code as follows:\n\n```sh\n# HNSW protobufs\npnpm proto:code:hnsw # generate js code\npnpm proto:type:hnsw # generate types\n\n# Request protobufs\npnpm proto:code:req # generate js code\npnpm proto:type:req # generate types\n```\n\n## Testing\n\nTests are ran over a few cases for a fixed set of $N, K$ parameters that are prepared in [Python](./test/python/main.py), and are compared in Typescript. Run the tests via:\n\n```sh\npnpm test\n```\n\n\u003e [!WARNING]\n\u003e\n\u003e You need a live Redis server for the HollowDB test to work. Furthermore, the HollowDB tests may take some time.\n\n## Styling\n\nCheck the formatting with:\n\n```sh\npnpm format\n```\n\nLint everything with:\n\n```sh\npnpm lint\n```\n\nYou can also check types with:\n\n```sh\npnpm check\n```\n\n## Legacy\n\nHollowDB Vector replaces DANNY, for the legacy code please [refer to this branch](https://github.com/firstbatchxyz/danny/tree/legacy).\n\n## License\n\nHollowDB Vector is licensed under [Apache 2.0](./LICENSE).\n","funding_links":[],"categories":["Databases","Vector Database Engines"],"sub_categories":["Vectors"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirstbatchxyz%2Fhollowdb-vector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffirstbatchxyz%2Fhollowdb-vector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirstbatchxyz%2Fhollowdb-vector/lists"}