{"id":25176714,"url":"https://github.com/conduitio/conduit-connector-protocol","last_synced_at":"2025-06-10T08:33:28.668Z","repository":{"id":40544213,"uuid":"433101168","full_name":"ConduitIO/conduit-connector-protocol","owner":"ConduitIO","description":"A low-level Go binding for the Conduit plugin protocol.","archived":false,"fork":false,"pushed_at":"2025-06-03T06:13:35.000Z","size":791,"stargazers_count":3,"open_issues_count":0,"forks_count":3,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-06-03T18:13:03.441Z","etag":null,"topics":["conduit","go","golang","grpc","protobuf","protocol-buffers"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ConduitIO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-11-29T15:42:53.000Z","updated_at":"2025-06-03T06:13:37.000Z","dependencies_parsed_at":"2023-07-12T20:00:25.858Z","dependency_job_id":"6bcb8b0a-f92b-44f2-b002-a9b344497c37","html_url":"https://github.com/ConduitIO/conduit-connector-protocol","commit_stats":{"total_commits":53,"total_committers":5,"mean_commits":10.6,"dds":"0.18867924528301883","last_synced_commit":"3e8b861b4c2457d653d1c9ce9a6d5ed01649f1ec"},"previous_names":["conduitio/conduit-plugin","conduitio/connector-plugin","conduitio/conduit-plugin-protocol"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ConduitIO%2Fconduit-connector-protocol","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ConduitIO%2Fconduit-connector-protocol/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ConduitIO%2Fconduit-connector-protocol/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ConduitIO%2Fconduit-connector-protocol/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ConduitIO","download_url":"https://codeload.github.com/ConduitIO/conduit-connector-protocol/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ConduitIO%2Fconduit-connector-protocol/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259038144,"owners_count":22796565,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conduit","go","golang","grpc","protobuf","protocol-buffers"],"created_at":"2025-02-09T13:17:57.401Z","updated_at":"2025-06-10T08:33:28.641Z","avatar_url":"https://github.com/ConduitIO.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Conduit Connector Protocol\n\n[![protobuf-docs](https://img.shields.io/badge/protobuf-docs-blue)](https://buf.build/conduitio/conduit-connector-protocol/docs/main:connector.v1)\n\n\u003e **Warning**  \n\u003e **If you want to implement a Conduit connector in Go, you should use the\n[Connector SDK](https://github.com/ConduitIO/conduit-connector-sdk).**\n\nThis repository contains the definition of\nthe [Conduit](https://github.com/conduitio/conduit) connector protocol in gRPC.\nIt also contains a thin Go layer that hides the gRPC implementation details\nwithout adding any functionality on top.\n\nThis repository is the only connection point between Conduit and a connector.\n\n## Implementing a connector in Go\n\nWe provide a [Connector SDK](https://github.com/ConduitIO/conduit-connector-sdk)\nfor writing connectors in Go. In this case you won't directly use the contents\nof this repository, instead the SDK hides implementation details and provides\nutilities to make developing a connector as simple as possible.\n\nIf you want to implement a connector in any other language you will need to\ngenerate the protocol code yourself, this is explained in the next chapter.\n\n## Implementing a connector in other languages\n\nYou can use [buf](https://buf.build/) to generate code for building a Conduit\nconnector in virtually any major language. To do that you need to create\na [`buf.gen.yaml`](https://docs.buf.build/generate/usage#31-create-a-bufgenyaml)\nfile and configure the connectors for the language you want to use.\n\nFor example here is a `buf.gen.yaml` file that is configured to generate C++ and\nJava code:\n\n```yaml\nversion: v1\nplugins:\n  # C++\n  - plugin: buf.build/grpc/cpp\n    out: gen/proto/cpp\n  - plugin: buf.build/protocolbuffers/cpp\n    out: gen/proto/cpp\n  # Java\n  - plugin: buf.build/grpc/java\n    out: gen/proto/java\n  - plugin: buf.build/protocolbuffers/java\n    out: gen/proto/java\n```\n\nThen you can run this command to generate the code:\n\n```shell\nbuf generate buf.build/conduitio/conduit-connector-protocol --template buf.gen.yaml\n```\n\nAt this point you should have everything you need to start developing a\nconnector. Make sure to implement all gRPC services according to the\ndocumentation in the\n[proto definition](https://buf.build/conduitio/conduit-connector-protocol/file/main/connector/v1/connector.proto)\nand to follow\nthe [go-plugin instructions](https://github.com/hashicorp/go-plugin/blob/master/docs/guide-plugin-write-non-go.md)\nabout writing a plugin in a language other than Go.\n\nOnce the connector is ready you need to create an entrypoint file which Conduit\ncan run to start the connector. In case of compiled languages that is the\ncompiled binary, in case of scripted languages you can create a simple shell\nscript that starts the connector. Here is an example for python:\n\n```\n#!/usr/bin/env python my-connector.py\n```\n\nThe connector executable can be put in a `connectors` directory, which should\nbe in the same directory as the Conduit binary. Here's an example:\n```\nconduit\nconnectors/\n    my-connector.py\n```\n\nTo run your connector as part of a Conduit pipeline you can create it using the\nconnectors API and use the `standalone:connector-name` in the field `plugin`, where\n`connector-name` is the name of your connector as mentioned in the connector specification.\n\nHere is an example request to `POST /v1/connectors` (find more about\nthe [Conduit API](https://github.com/conduitio/conduit#api)):\n\n```json\n{\n  \"type\": \"TYPE_SOURCE\",\n  \"plugin\": \"standalone:connector-name\",\n  \"pipelineId\": \"...\",\n  \"config\": {\n    \"name\": \"my-connector\",\n    \"settings\": {\n      \"my-key\": \"my-value\"\n    }\n  }\n}\n```\n\nClick [here](https://github.com/ConduitIO/conduit/blob/main/docs/connector_discovery.md) to learn more \nabout how Conduit discovers connectors.\n\n## Development\n\nTo generate protobuf code run `make proto-generate`.\n\n## Acknowledgment\n\nWe took inspiration for our connector implementation from\n[hashicorp/terraform-plugin-go](https://github.com/hashicorp/terraform-plugin-go).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconduitio%2Fconduit-connector-protocol","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconduitio%2Fconduit-connector-protocol","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconduitio%2Fconduit-connector-protocol/lists"}