{"id":25176700,"url":"https://github.com/conduitio/conduit-operator","last_synced_at":"2025-10-28T12:10:59.073Z","repository":{"id":257658986,"uuid":"580208989","full_name":"ConduitIO/conduit-operator","owner":"ConduitIO","description":"Automatically provision and manage Conduit instances in Kubernetes","archived":false,"fork":false,"pushed_at":"2025-03-31T19:55:57.000Z","size":497,"stargazers_count":1,"open_issues_count":8,"forks_count":0,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-03-31T20:23:13.435Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ConduitIO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-20T01:34:52.000Z","updated_at":"2025-03-11T17:49:22.000Z","dependencies_parsed_at":"2025-02-21T22:22:28.953Z","dependency_job_id":"a3a84138-6a86-4df8-873b-33eee737c526","html_url":"https://github.com/ConduitIO/conduit-operator","commit_stats":null,"previous_names":["conduitio/conduit-operator","conduitio-labs/conduit-operator"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ConduitIO%2Fconduit-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ConduitIO%2Fconduit-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ConduitIO%2Fconduit-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ConduitIO%2Fconduit-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ConduitIO","download_url":"https://codeload.github.com/ConduitIO/conduit-operator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247107830,"owners_count":20884797,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-09T13:17:55.642Z","updated_at":"2025-10-28T12:10:54.025Z","avatar_url":"https://github.com/ConduitIO.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Conduit Operator\n\nThis reposistory contains a kubernetes operator for deploying Conduit instances.\nThe operator extends the Kubernetes API with a custom `Conduit` resource.\n\nThe operator is based on [Kubebuilder](https://github.com/kubernetes-sigs/kubebuilder).\n\n## Overview\n\nConduit pipelines are represented as Conduit custom resources, where each pipeline will be provisioned \nas a distinct Conduit instance with its own lifecycle. \n\nThe Conduit custom resource definition format is very similar to that of [pipeline configurations](https://conduit.io/docs/pipeline-configuration-files/getting-started).\n\n```yaml\napiVersion: operator.conduit.io/v1alpha\nkind: Conduit\nmetadata:\n  name: conduit-generator\nspec:\n  running: true\n  name: generator.log\n  description: generator pipeline\n  connectors:\n    - name: source-connector\n      type: source\n      plugin: builtin:generator\n      settings:\n        - name: format.type\n          value: structured\n        - name: format.options.id\n          value: \"int\"\n        - name: format.options.name\n          value: \"string\"\n        - name: format.options.company\n          value: \"string\"\n        - name: format.options.trial\n          value: \"bool\"\n        - name: recordCount\n          value: \"3\"\n    - name: destination-connector\n      type: destination\n      plugin: builtin:log\n```\n\nThe operator can install standalone connectors referred by their github org / repo names.\nVersion can optionally be specified or the latest will be used. Format is `github-org/repo-name`, e.g. `conduitio/conduit-connector-generator`\n\n```yaml\napiVersion: operator.conduit.io/v1alpha\nkind: Conduit\nmetadata:\n  name: conduit-generator\nspec:\n  running: true\n  name: generator.log\n  description: generator pipeline\n  connectors:\n    - name: source-connector\n      type: source\n      plugin: conduitio/conduit-connector-generator\n      settings:\n        - name: format.type\n          value: structured\n        - name: format.options.id\n          value: \"int\"\n        - name: format.options.name\n          value: \"string\"\n        - name: format.options.company\n          value: \"string\"\n        - name: format.options.trial\n          value: \"bool\"\n        - name: recordCount\n          value: \"3\"\n    - name: destination-connector\n      type: destination\n      plugin: conduitio/conduit-connector-log\n```\n### Schema registry\n\nAs of [v0.11.0](https://conduit.io/changelog/2024-08-19-conduit-0-11-0-release) Conduit supports the use of a schema registry. \nThis allows connectors to automatically extract or use the schema referred to by the OpenCDC record to encode/decode data. \n\nBy default conduit uses a builtin schema registry, however in certain use cases a schema registry needs to be shared between\nmultiple instances. The conduit resource allows for schema registry to be defined as of [v0.2.0](https://github.com/ConduitIO/conduit-operator/releases/tag/v0.0.2).\n\n```yaml\napiVersion: operator.conduit.io/v1alpha\nkind: Conduit\nmetadata:\n  name: conduit-generator-schema-registry\nspec:\n  running: true\n  name: generator.standalone.log\n  description: generator pipeline\n  schemaRegistry:\n    url: http://apicurio:8080/apis/ccompat/v7\n    # basicAuthUser:\n    #   - value: \u003cschemaUser\u003e\n    # basicAuthPassword:\n    #   - secretRef:\n    #     key: schema-registry-password\n    #     name: schema-registry-secret\n  connectors:\n    - name: source-connector\n      type: source\n      plugin: conduitio/conduit-connector-generator\n      settings:\n        - name: format.type\n          value: structured\n        - name: format.options.id\n          value: \"int\"\n        - name: format.options.name\n          value: \"string\"\n        - name: format.options.company\n          value: \"string\"\n        - name: format.options.trial\n          value: \"bool\"\n        - name: recordCount\n          value: \"3\"\n    - name: destination-connector\n      type: destination\n      plugin: conduitio/conduit-connector-log\n      pluginVersion: v0.4.0\n```\n\n\n## Quickstart\n\n1. Download this repository locally\n   \n2. Run `make dev` to initialize a [kind cluster](https://kind.sigs.k8s.io) and install the operator using helm.\n   \n3. Deploy a sample pipeline configuration\n   ```shell\n   kubectl apply -f config/samples/conduit-generator.yaml\n   ```\n\n4. Wait for instance to become ready\n   ```\n   kubectl wait --for=condition=Ready -l app.kubernetes.io/name=conduit-server-conduit-generator pod\n   ```\n\n5. Follow logs of provisioned instalce\n   ```\n   kubectl logs -f -l app.kubernetes.io/name=conduit-server-conduit-generator\n   ```\n## Deployment\n\nThe operator provides a [helm chart](charts/conduit-operator) which can be used for deployment on any cluster.\n\nAdditional metadata can be injected in each provisioned conduit instance via `controller.conduitMetadata` configuration. \n\nFor example, to instruct a prometheus instance to scrape each Conduit instance metrics:\n\n```yaml\ncontroller:\n  conduitMetadata:\n    podAnnotations:\n      prometheus.io/scrape: true\n      prometheus.io/path: /metrics\n      prometheus.io/port: 8080\n```\n\nFor more configuration options see [charts/conduit-operator/values.yaml](charts/conduit-operator/values.yaml)\n\n### Using the helm chart repository\n\nAlternatively the operator can be deployed via the helm repository\nTo add the repository to your helm repos:\n\n```shell\nhelm repo add conduit https://conduitio.github.io/conduit-operator\n```\n\nInstall the operator in the `conduit-operator` namespace of your cluster:\n\n```shell\nhelm install conduit-operator conduit/conduit-operator --create-namespace -n conduit-operator\n```\n\n## Development\n\nChanges to the operator can be deployed to the kind cluster via `make dev` \n\n## Scaffolding \n\nThe project scaffolding was generated using [Kubebuilder](https://book.kubebuilder.io), using the following commands:\n\n```\nkubebuilder init --domain conduit.io --plugins=go.kubebuilder.io/v4 --project-version 3 --project-name conduit-operator\nkubebuilder create api --group operator --version v1alpha --kind Conduit\nkubebuilder create webhook --group operator --version v1alpha --kind Conduit --conversion --spoke v1alpha --defaulting --programmatic-validation\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconduitio%2Fconduit-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconduitio%2Fconduit-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconduitio%2Fconduit-operator/lists"}