{"id":17997444,"url":"https://github.com/scylladb/scylla-cdc-source-connector","last_synced_at":"2026-01-30T20:05:20.099Z","repository":{"id":44642535,"uuid":"355503446","full_name":"scylladb/scylla-cdc-source-connector","owner":"scylladb","description":"A Kafka source connector capturing Scylla CDC changes","archived":false,"fork":false,"pushed_at":"2025-06-10T17:46:12.000Z","size":1120,"stargazers_count":53,"open_issues_count":35,"forks_count":20,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-06-10T18:57:13.425Z","etag":null,"topics":["apache-kafka","cdc","change-data-capture","debezium","event-streaming","java","kafka","kafka-connect","kafka-producer","nosql","scylla"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scylladb.png","metadata":{"files":{"readme":"README-QUICKSTART.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-04-07T10:29:15.000Z","updated_at":"2025-06-10T17:46:16.000Z","dependencies_parsed_at":"2024-01-16T12:53:24.189Z","dependency_job_id":"df4a5287-21d9-48c9-af64-d957fcfc2c4a","html_url":"https://github.com/scylladb/scylla-cdc-source-connector","commit_stats":{"total_commits":60,"total_committers":5,"mean_commits":12.0,"dds":"0.30000000000000004","last_synced_commit":"d8e93732961ae65347246edca504a1f7f9db87d9"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/scylladb/scylla-cdc-source-connector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scylladb%2Fscylla-cdc-source-connector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scylladb%2Fscylla-cdc-source-connector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scylladb%2Fscylla-cdc-source-connector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scylladb%2Fscylla-cdc-source-connector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scylladb","download_url":"https://codeload.github.com/scylladb/scylla-cdc-source-connector/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scylladb%2Fscylla-cdc-source-connector/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260967882,"owners_count":23090107,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-kafka","cdc","change-data-capture","debezium","event-streaming","java","kafka","kafka-connect","kafka-producer","nosql","scylla"],"created_at":"2024-10-29T21:18:28.159Z","updated_at":"2026-01-30T20:05:20.054Z","avatar_url":"https://github.com/scylladb.png","language":"Java","funding_links":[],"categories":["Drivers and Libraries"],"sub_categories":["Java"],"readme":"# Scylla CDC Source Connector Quickstart\n\n## Synopsis\n\nThis quickstart will show you how to setup the Scylla CDC Source Connector to replicate changes made in \na Scylla table using [Scylla CDC](https://docs.scylladb.com/using-scylla/cdc/cdc-intro/).\n\n## Scylla setup\n\nFirst, let's setup a Scylla cluster and create a CDC-enabled table.\n\n### Scylla installation\n\nFor the purpose of this quickstart, we will configure a Scylla instance using Docker. You can skip this \nsection if you have already installed Scylla. To learn more about installing Scylla in production\nenvironments, please refer to the [Install Scylla page](https://docs.scylladb.com/getting-started/install_scylla/).\n\n1. Using [Docker](https://hub.docker.com/r/scylladb/scylla/), follow the instructions to launch Scylla.\n2. Start the Docker container, replacing the `--name` and `--host name` parameters with your own information. For example:\n   ```\n   docker run --name scylla-cdc-quickstart --hostname scylla-cdc-quickstart -d scylladb/scylla\n   ```\n3. Run `docker ps` to show the exposed ports. The output should be similar to this example:\n    ```\n    docker ps \n    \n    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                            NAMES\n    4fca02217055        scylladb/scylla     \"/docker-entrypoint.…\"   8 seconds ago       Up 7 seconds        22/tcp, 7000-7001/tcp, 9042/tcp, 9160/tcp, 9180/tcp, 10000/tcp   scylla-cdc-quickstart\n    ```\n   \n### Creating a CDC-enabled table\n\nLet's connect to your Scylla cluster and create a new CDC-enabled table. We will create an example table by \nissuing the following CQL query and insert some example data:\n\n```\nCREATE KEYSPACE quickstart_keyspace WITH REPLICATION = {'class': 'SimpleStrategy', 'replication_factor': 1};\n\nCREATE TABLE quickstart_keyspace.orders(\n  customer_id int, \n  order_id int, \n  product text, \n  PRIMARY KEY(customer_id, order_id)) WITH cdc = {'enabled': true};\n\nINSERT INTO quickstart_keyspace.orders(customer_id, order_id, product) VALUES (1, 1, 'pizza'); \nINSERT INTO quickstart_keyspace.orders(customer_id, order_id, product) VALUES (1, 2, 'cookies');\nINSERT INTO quickstart_keyspace.orders(customer_id, order_id, product) VALUES (1, 3, 'tea');\n```\n\nIf you already have a table you wish to use, but it does not have CDC enabled, you can turn it on by using the following CQL query:\n```\nALTER TABLE keyspace.table_name with cdc = {'enabled': true};\n```\n\nTo learn more about Scylla CDC, visit [Change Data Capture (CDC) page](https://docs.scylladb.com/using-scylla/cdc/).\n\n## Kafka setup\n\nScylla CDC Source Connector works well with both [open-source Kafka](https://kafka.apache.org/) \nand [Confluent Platform](https://www.confluent.io/). Kafka version 2.6.0 or greater is\nrequired. In this quickstart we will show how to install the Confluent Platform and deploy \nthe connector (applicable to both open-source Kafka and Confluent Platform).\n\n### Installing Confluent Platform\n\nIf you are new to Confluent, [download Confluent Platform](https://www.confluent.io/download/).\n\n1. In the *Download Confluent Platform* section fill in your email address\n2. Open the *Select Deployment Type* drop-down and select *ZIP*\n3. Accept the Terms \u0026 Conditions and click *DOWNLOAD FREE*\n4. You will receive an email with instructions. Download / move the file to the desired location\n5. Continue with the setup following [this document](https://docs.confluent.io/current/quickstart/ce-quickstart.html#ce-quickstart)\n\n### Installing Scylla CDC Source Connector\n\n1. Download or build Scylla CDC Source Connector using [the project build instructions](https://github.com/scylladb/scylla-cdc-source-connector#building)\n2. Deploy the connector:\n   1. If you use Confluent Platform, move connector JAR files to the `share/java` folder\n   2. If you use open-source Kafka, make sure that `plugin.path` of Kafka Connect configuration contains the folder with connector JAR files\n\n## Connector configuration\n\nAfter you have successfully configured Scylla and Kafka, the next step is to configure the connector\nand start it up.\n\n### Configuration using Confluent Control Center\n\nIf you use Confluent Platform, the easiest way to configure and start up the Scylla CDC Source Connector\nis to use Confluent Control Center web interface.\n\n1. Open the Confluent Control Center. By default, it is started at port `9021`:\n    ![Confluent Control Center main page](images/scylla-cdc-source-connector-control-center1.png)\n    \n2. Click on the cluster you want to start the connector in and open the \"Connect\" tab:\n    ![Confluent Control Center \"Connect\" tab](images/scylla-cdc-source-connector-control-center2.png)\n\n3. Click on the Kafka Connect cluster:\n    ![Confluent Control Center \"connect-default\" cluster](images/scylla-cdc-source-connector-control-center3.png)\n\n4. Click \"Add connector\":\n    ![Confluent Control Center \"Add connector\"](images/scylla-cdc-source-connector-control-center4.png)\n\n5. Click \"ScyllaConnector (Source Connector)\":\n    ![Confluent Control Center \"ScyllaConnector (Source Connector)\"](images/scylla-cdc-source-connector-control-center5.png)\n\n6. Configure the connector. You need to fill in these required configuration parameters:\n\n   1. Name: the name of this configuration\n   2. Key converter class, value converter class: converters that determine the format \n      of produced messages. You can read more about them at [Kafka Connect Deep Dive – Converters and Serialization Explained](https://www.confluent.io/blog/kafka-connect-deep-dive-converters-serialization-explained/)\n   3. Hosts: contact points of Scylla\n   4. Namespace: a unique name that identifies the Scylla cluster and that is used as a prefix for all schemas, topics.\n   5. Table names: the names of CDC-enabled tables you want to replicate\n\n   For the quickstart example here are the values we will use:\n\n   1. Name: `QuickstartConnector`\n   2. Key converter class, value converter class: `org.apache.kafka.connect.json.JsonConverter`\n   3. Hosts: `172.17.0.2:9042` (Scylla started in Docker)\n   4. Namespace: `QuickstartConnectorNamespace`\n   5. Table names: `quickstart_keyspace.orders`\n   \n   ![Confluent Control Center connector configuration](images/scylla-cdc-source-connector-control-center6.png)\n\n7. Click \"Continue\" and \"Launch\"\n\n8. After a short while, a new `QuickstartConnectorNamespace.quickstart_keyspace.orders` topic will be automatically created\n   and inserted rows will be replicated. You can browse them by going to the \"Topics\" tab, selecting \n   `QuickstartConnectorNamespace.quickstart_keyspace.orders` topic, going to \"Message\" tab and inputting `0` to \"Jump to offset\"\n   field:\n   \n    ![Confluent Control Center connector messages](images/scylla-cdc-source-connector-control-center7.png)\n\n### Configuration using open-source Kafka\n\n1. Start Kafka Connect standalone using [this guide](https://kafka.apache.org/documentation/#connect_running). You\n   will have to create a `connector.properties` file with the following contents:\n   \n    ```\n    name = QuickstartConnector\n    connector.class = com.scylladb.cdc.debezium.connector.ScyllaConnector\n    key.converter = org.apache.kafka.connect.json.JsonConverter\n    value.converter = org.apache.kafka.connect.json.JsonConverter\n    scylla.cluster.ip.addresses = 172.17.0.2:9042\n    scylla.name = QuickstartConnectorNamespace\n    scylla.table.names = quickstart_keyspace.orders\n    ```\n\n2. After starting the connector, you can see the generated messages by using `kafka-console-consumer` tool:\n    ```\n    bin/kafka-console-consumer --bootstrap-server localhost:9092 --topic QuickstartConnectorNamespace.quickstart_keyspace.orders --from-beginning\n    ```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscylladb%2Fscylla-cdc-source-connector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscylladb%2Fscylla-cdc-source-connector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscylladb%2Fscylla-cdc-source-connector/lists"}