{"id":28493376,"url":"https://github.com/qdrant/qdrant-kafka","last_synced_at":"2025-07-08T05:30:46.666Z","repository":{"id":245794880,"uuid":"819253164","full_name":"qdrant/qdrant-kafka","owner":"qdrant","description":"Kafka Sink connector for Qdrant. Stream vector data into a Qdrant collection. Supports named/unnamed dense/sparse vectors.","archived":false,"fork":false,"pushed_at":"2024-12-27T08:19:10.000Z","size":96,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-01T17:42:54.559Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/qdrant.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-24T06:31:39.000Z","updated_at":"2024-12-27T07:57:51.000Z","dependencies_parsed_at":"2024-06-24T07:50:28.712Z","dependency_job_id":"2713f6d7-af94-4e8b-a7bf-a1a998435cab","html_url":"https://github.com/qdrant/qdrant-kafka","commit_stats":null,"previous_names":["anush008/qdrant-kafka"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qdrant%2Fqdrant-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qdrant%2Fqdrant-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qdrant%2Fqdrant-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qdrant%2Fqdrant-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/qdrant","download_url":"https://codeload.github.com/qdrant/qdrant-kafka/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qdrant%2Fqdrant-kafka/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":258688287,"owners_count":22741603,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-08T09:08:34.697Z","updated_at":"2025-07-08T05:30:46.661Z","avatar_url":"https://github.com/qdrant.png","language":"Java","funding_links":[],"categories":["Vector Databases"],"sub_categories":["Qdrant"],"readme":"# Qdrant Sink Connector\n\nUse Qdrant as a sink destination in [Kafka Connect](https://docs.confluent.io/platform/current/connect/index.html). Supports streaming dense/sparse/multi vectors into Qdrant collections.\n\n## Usage\n\n\u003e [!IMPORTANT]\n\u003e Qdrant collections have to be [created](https://qdrant.tech/documentation/concepts/collections/#create-a-collection) in advance with the appropriate vector dimensions and configurations.\n\nLearn to use the connector with\n\n- [Kafka on Confluent Cloud](https://github.com/qdrant/qdrant-kafka/blob/main/CONFLUENT.md)\n\n- [Self-hosted Kafka](https://github.com/qdrant/qdrant-kafka/blob/main/KAFKA.md)\n\n## Message Formats\n\nThis sink connector supports messages with multiple dense/sparse vectors.\n\n_Click each to expand._\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003cb\u003eUnnamed/Default vector\u003c/b\u003e\u003c/summary\u003e\n\nReference: [Creating a collection with a default vector](https://qdrant.tech/documentation/concepts/collections/#create-a-collection).\n\n```json\n{\n    \"collection_name\": \"{collection_name}\",\n    \"id\": 1,\n    \"vector\": [\n        0.1,\n        0.2,\n        0.3,\n        0.4,\n        0.5,\n        0.6,\n        0.7,\n        0.8\n    ],\n    \"payload\": {\n        \"name\": \"kafka\",\n        \"description\": \"Kafka is a distributed streaming platform\",\n        \"url\": \"https://kafka.apache.org/\"\n    }\n}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003cb\u003eNamed multiple vectors\u003c/b\u003e\u003c/summary\u003e\n\nReference: [Creating a collection with multiple vectors](https://qdrant.tech/documentation/concepts/collections/#collection-with-multiple-vectors).\n\n```json\n{\n    \"collection_name\": \"{collection_name}\",\n    \"id\": 1,\n    \"vector\": {\n        \"some-dense\": [\n            0.1,\n            0.2,\n            0.3,\n            0.4,\n            0.5,\n            0.6,\n            0.7,\n            0.8\n        ],\n        \"some-other-dense\": [\n            0.1,\n            0.2,\n            0.3,\n            0.4,\n            0.5,\n            0.6,\n            0.7,\n            0.8\n        ]\n    },\n    \"payload\": {\n        \"name\": \"kafka\",\n        \"description\": \"Kafka is a distributed streaming platform\",\n        \"url\": \"https://kafka.apache.org/\"\n    }\n}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003cb\u003eSparse vectors\u003c/b\u003e\u003c/summary\u003e\n\nReference: [Creating a collection with sparse vectors](https://qdrant.tech/documentation/concepts/collections/#collection-with-sparse-vectors).\n\n```json\n{\n    \"collection_name\": \"{collection_name}\",\n    \"id\": 1,\n    \"vector\": {\n        \"some-sparse\": {\n            \"indices\": [\n                0,\n                1,\n                2,\n                3,\n                4,\n                5,\n                6,\n                7,\n                8,\n                9\n            ],\n            \"values\": [\n                0.1,\n                0.2,\n                0.3,\n                0.4,\n                0.5,\n                0.6,\n                0.7,\n                0.8,\n                0.9,\n                1.0\n            ]\n        }\n    },\n    \"payload\": {\n        \"name\": \"kafka\",\n        \"description\": \"Kafka is a distributed streaming platform\",\n        \"url\": \"https://kafka.apache.org/\"\n    }\n}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003cb\u003eMulti-vector\u003c/b\u003e\u003c/summary\u003e\n\n```json\n{\n    \"collection_name\": \"{collection_name}\",\n    \"id\": 1,\n    \"vector\": {\n        \"some-multi\": [\n            [\n                0.1,\n                0.2,\n                0.3,\n                0.4,\n                0.5,\n                0.6,\n                0.7,\n                0.8,\n                0.9,\n                1.0\n            ],\n            [\n                1.0,\n                0.9,\n                0.8,\n                0.5,\n                0.4,\n                0.8,\n                0.6,\n                0.4,\n                0.2,\n                0.1\n            ]\n        ]\n    },\n    \"payload\": {\n        \"name\": \"kafka\",\n        \"description\": \"Kafka is a distributed streaming platform\",\n        \"url\": \"https://kafka.apache.org/\"\n    }\n}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003cb\u003eCombination of named dense and sparse vectors\u003c/b\u003e\u003c/summary\u003e\n\nReference:\n\n- [Creating a collection with multiple vectors](https://qdrant.tech/documentation/concepts/collections/#collection-with-multiple-vectors).\n\n- [Creating a collection with sparse vectors](https://qdrant.tech/documentation/concepts/collections/#collection-with-sparse-vectors).\n\n```json\n{\n    \"collection_name\": \"{collection_name}\",\n    \"id\": \"a10435b5-2a58-427a-a3a0-a5d845b147b7\",\n    \"vector\": {\n        \"some-other-dense\": [\n            0.1,\n            0.2,\n            0.3,\n            0.4,\n            0.5,\n            0.6,\n            0.7,\n            0.8\n        ],\n        \"some-sparse\": {\n            \"indices\": [\n                0,\n                1,\n                2,\n                3,\n                4,\n                5,\n                6,\n                7,\n                8,\n                9\n            ],\n            \"values\": [\n                0.1,\n                0.2,\n                0.3,\n                0.4,\n                0.5,\n                0.6,\n                0.7,\n                0.8,\n                0.9,\n                1.0\n            ]\n        }\n    },\n    \"payload\": {\n        \"name\": \"kafka\",\n        \"description\": \"Kafka is a distributed streaming platform\",\n        \"url\": \"https://kafka.apache.org/\"\n    }\n}\n```\n\n\u003c/details\u003e\n\n## LICENSE\n\nApache 2.0 © [2024](https://github.com/qdrant/qdrant-kafka/blob/main/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqdrant%2Fqdrant-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqdrant%2Fqdrant-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqdrant%2Fqdrant-kafka/lists"}