{"id":29746693,"url":"https://github.com/m0nirul/kafka-connect-demo","last_synced_at":"2026-02-07T12:01:40.762Z","repository":{"id":306268123,"uuid":"197855777","full_name":"m0nirul/kafka-connect-demo","owner":"m0nirul","description":"How to use kafka-connect with other source/sink","archived":false,"fork":false,"pushed_at":"2020-04-10T05:28:08.000Z","size":86489,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-07-24T18:43:24.933Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Dockerfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/m0nirul.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-07-19T23:42:09.000Z","updated_at":"2020-04-10T05:28:10.000Z","dependencies_parsed_at":"2025-07-24T18:43:34.292Z","dependency_job_id":"396e3c49-525f-49af-a933-7de5c1d692aa","html_url":"https://github.com/m0nirul/kafka-connect-demo","commit_stats":null,"previous_names":["m0nirul/kafka-connect-demo"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/m0nirul/kafka-connect-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m0nirul%2Fkafka-connect-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m0nirul%2Fkafka-connect-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m0nirul%2Fkafka-connect-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m0nirul%2Fkafka-connect-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/m0nirul","download_url":"https://codeload.github.com/m0nirul/kafka-connect-demo/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m0nirul%2Fkafka-connect-demo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29194000,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T07:37:03.739Z","status":"ssl_error","status_checked_at":"2026-02-07T07:37:03.029Z","response_time":63,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-26T07:43:31.022Z","updated_at":"2026-02-07T12:01:40.757Z","avatar_url":"https://github.com/m0nirul.png","language":"Dockerfile","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Overview\nThis is a quick demo of how to work on the confluent platform\n\u003e Note: This is not an official repo. Just tried to get the things working from confluent opensource repo\n\n\nThe codes used are all Apache 2.0 licence\n\n| Description | Source | Licence |\n| ------ | ------ | ------------ |\n| cp-zookeeper | [docker hub](https://hub.docker.com/r/confluentinc/cp-zookeeper) | Apache 2 |\n| cp-kafka | [docker hub](https://hub.docker.com/r/confluentinc/cp-kafka) | Apache 2 |\n| cp-kafka-rest | [docker hub](https://hub.docker.com/r/confluentinc/cp-kafka-rest) | Apache 2 |\n| cp-schema-registry | [docker hub](https://hub.docker.com/r/confluentinc/cp-schema-registry) | Apache 2 |\n| cp-kafka-connect | [docker hub](https://hub.docker.com/r/confluentinc/cp-kafka-connect) | Apache 2 |\n| Connector Jars | [Github/Landoop](https://github.com/Landoop/stream-reactor/releases) | Apache 2 |\n| Confluent connect datagen | [Github/confluentinc](https://github.com/confluentinc/kafka-connect-datagen) | Apache 2 |\n\nFor an example of how to use this Docker setup, refer to the Confluent Platform quickstart: https://docs.confluent.io/current/quickstart/index.html\n\n## Steps\n### 1. Install necessary packages\n```\ndocker pull confluentinc/cp-zookeeper\ndocker pull confluentinc/cp-kafka\ndocker pull confluentinc/cp-schema-registry\ndocker pull confluentinc/cp-kafka-rest\n```\n\n### 2. Download extra jars and put it in ./extraJars/jars/*\n\n\n### 3. Build the connect Image\n```\ndocker build --rm -t cp-kafka-connect-new .\n```\n### 4. Deploy the stack on the swarm\n```\ndocker stack deploy -c docker-composev2.yml kafka\n```\n### 5. Access the services\nThe services wil l be deployed as per the details in ``docker-composev2.yml``. As per default configuration this are some servcice urls\n\n* Kafka - host:9092\n* zookeeper - host:2181\n* kafka-connect - host:8083\n* kafka-rest - host:8082\n\n### 6. Create a sink/source\nTo use the created service and create connectors make a post request to ``http://host:8083/connectors`` and the body will be name and the config details. Please check the respective websites for the config details. For cassandra it will be \n```json\n{\n  \"name\" : \"cassandraSinkConnector1\",\n  \"config\" : {\n    \"connector.class\" : \"com.datamountaineer.streamreactor.connect.cassandra.sink.CassandraSinkConnector\",\n    \"tasks.max\" : \"1\",\n    \"topics\" : \"swipesRecord\",\n    \"connect.cassandra.kcql\" : \"INSERT INTO swipes SELECT * FROM swipesRecord;\",\n    \"connect.cassandra.contact.points\":\"cassandrahost\",\n    \"connect.cassandra.port\":9042,\n    \"connect.cassandra.key.space\":\"swipes\"\n  }\n}\n```\n\n\n## List of available connector lists\n\n| class|name|version|encodedVersion|type|typeName|location|\n| ------ | ------ | ------- | ------ | ------ | ------ | ------ |\n| com.datamountaineer.streamreactor.connect.cassandra.sink.CassandraSinkConnector|com.datamountaineer.streamreactor.connect.cassandra.sink.CassandraSinkConnector|1.2.2|1.2.2|sink|sink|file:/var/jars/kafka-connect-cassandra-1.2.2/|\n|com.datamountaineer.streamreactor.connect.cassandra.source.CassandraSourceConnector|com.datamountaineer.streamreactor.connect.cassandra.source.CassandraSourceConnector|1.2.2|1.2.2|source|source|file:/var/jars/kafka-connect-cassandra-1.2.2/|\n|io.confluent.connect.activemq.ActiveMQSourceConnector|io.confluent.connect.activemq.ActiveMQSourceConnector|5.1.2|5.1.2|source|source|file:/usr/share/java/kafka-connect-activemq/|\n|io.confluent.connect.elasticsearch.ElasticsearchSinkConnector|io.confluent.connect.elasticsearch.ElasticsearchSinkConnector|5.1.2|5.1.2|sink|sink|file:/usr/share/java/kafka-connect-elasticsearch/|\n|io.confluent.connect.gcs.GcsSinkConnector|io.confluent.connect.gcs.GcsSinkConnector|5.0.1|5.0.1|sink|sink|file:/usr/share/confluent-hub-components/confluentinc-kafka-connect-gcs/|\n|io.confluent.connect.hdfs.HdfsSinkConnector|io.confluent.connect.hdfs.HdfsSinkConnector|5.1.2|5.1.2|sink|sink|file:/usr/share/java/kafka-connect-hdfs/|\n|io.confluent.connect.hdfs.tools.SchemaSourceConnector|io.confluent.connect.hdfs.tools.SchemaSourceConnector|2.1.1-cp1|2.1.1-cp1|source|source|file:/usr/share/java/kafka-connect-hdfs/|\n|io.confluent.connect.ibm.mq.IbmMQSourceConnector|io.confluent.connect.ibm.mq.IbmMQSourceConnector|5.1.2|5.1.2|source|source|file:/usr/share/java/kafka-connect-ibmmq/|\n|io.confluent.connect.jdbc.JdbcSinkConnector|io.confluent.connect.jdbc.JdbcSinkConnector|5.1.2|5.1.2|sink|sink|file:/usr/share/java/kafka-connect-jdbc/|\n|io.confluent.connect.jdbc.JdbcSourceConnector|io.confluent.connect.jdbc.JdbcSourceConnector|5.1.2|5.1.2|source|source|file:/usr/share/java/kafka-connect-jdbc/|\n|io.confluent.connect.jms.JmsSourceConnector|io.confluent.connect.jms.JmsSourceConnector|5.1.2|5.1.2|source|source|file:/usr/share/java/kafka-connect-jms/|\n|io.confluent.connect.s3.S3SinkConnector|io.confluent.connect.s3.S3SinkConnector|5.1.2|5.1.2|sink|sink|file:/usr/share/java/kafka-connect-s3/|\n|io.confluent.connect.storage.tools.SchemaSourceConnector|io.confluent.connect.storage.tools.SchemaSourceConnector|2.1.1-cp1|2.1.1-cp1|source|source|file:/usr/share/java/kafka-connect-hdfs/|\n|io.confluent.kafka.connect.datagen.DatagenConnector|io.confluent.kafka.connect.datagen.DatagenConnector|null|null|source|source|file:/usr/share/confluent-hub-components/confluentinc-kafka-connect-datagen/|\n|org.apache.kafka.connect.file.FileStreamSinkConnector|org.apache.kafka.connect.file.FileStreamSinkConnector|2.1.1-cp1|2.1.1-cp1|sink|sink|file:/usr/share/java/kafka/|\n|org.apache.kafka.connect.file.FileStreamSourceConnector|org.apache.kafka.connect.file.FileStreamSourceConnector|2.1.1-cp1|2.1.1-cp1|source|source|file:/usr/share/java/kafka/|\n|org.apache.kafka.connect.tools.MockConnector|org.apache.kafka.connect.tools.MockConnector|2.1.1-cp1|2.1.1-cp1|connector|connector|file:/usr/share/java/confluent-control-center/|\n|org.apache.kafka.connect.tools.MockSinkConnector|org.apache.kafka.connect.tools.MockSinkConnector|2.1.1-cp1|2.1.1-cp1|sink|sink|file:/usr/share/java/confluent-control-center/|\n|org.apache.kafka.connect.tools.MockSourceConnector|org.apache.kafka.connect.tools.MockSourceConnector|2.1.1-cp1|2.1.1-cp1|source|source|file:/usr/share/java/confluent-control-center/|\n|org.apache.kafka.connect.tools.SchemaSourceConnector|org.apache.kafka.connect.tools.SchemaSourceConnector|2.1.1-cp1|2.1.1-cp1|source|source|file:/usr/share/java/confluent-control-center/|\n|org.apache.kafka.connect.tools.VerifiableSinkConnector|org.apache.kafka.connect.tools.VerifiableSinkConnector|2.1.1-cp1|2.1.1-cp1|source|source|file:/usr/share/java/confluent-control-center/|\n|org.apache.kafka.connect.tools.VerifiableSourceConnector|org.apache.kafka.connect.tools.VerifiableSourceConnector|2.1.1-cp1|2.1.1-cp1|source|source|file:/usr/share/java/confluent-control-center/|\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm0nirul%2Fkafka-connect-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fm0nirul%2Fkafka-connect-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm0nirul%2Fkafka-connect-demo/lists"}