{"id":19704636,"url":"https://github.com/datastax/pulsar-3rdparty-connector","last_synced_at":"2025-04-29T14:31:37.075Z","repository":{"id":42467731,"uuid":"438393152","full_name":"datastax/pulsar-3rdparty-connector","owner":"datastax","description":"This project provides simple templates and instructions to build Apache Pulsar connectors on base of the existing Apache Kafka connectors.","archived":false,"fork":false,"pushed_at":"2024-11-04T17:30:16.000Z","size":865,"stargazers_count":8,"open_issues_count":0,"forks_count":10,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-05T17:51:12.027Z","etag":null,"topics":["apache","apache-pulsar","connectors","kafka","kafka-connect","kafka-connector","pulsar","pulsar-io"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datastax.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-14T20:38:36.000Z","updated_at":"2024-11-04T17:30:20.000Z","dependencies_parsed_at":"2024-11-04T18:35:54.516Z","dependency_job_id":null,"html_url":"https://github.com/datastax/pulsar-3rdparty-connector","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastax%2Fpulsar-3rdparty-connector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastax%2Fpulsar-3rdparty-connector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastax%2Fpulsar-3rdparty-connector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastax%2Fpulsar-3rdparty-connector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datastax","download_url":"https://codeload.github.com/datastax/pulsar-3rdparty-connector/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251519146,"owners_count":21602281,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache","apache-pulsar","connectors","kafka","kafka-connect","kafka-connector","pulsar","pulsar-io"],"created_at":"2024-11-11T21:23:43.410Z","updated_at":"2025-04-29T14:31:32.067Z","avatar_url":"https://github.com/datastax.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Third Party connectors for Apache Pulsar\n\nThis project provides simple templates and instructions to build Apache Pulsar connectors on the base of \nexisting Apache Kafka connectors. \n\nApache Pulsar's current acceptance criteria for connectors requires a developer brave and experienced enough \nwith both Pulsar and third party systems to contribute the connector and required integration tests.\n\nThis project relaxes the criteria to allow developers to quickly move connectors\nthey used with their Apache Kafka infrastructure into Apache Pulsar's, and incrementally work on improvements.\n\nThe project uses Apache Pulsar's Kafka Connect Adaptor (KCA). More information about KCA is available in \n[this blog post](https://www.datastax.com/blog/simplify-migrating-kafka-to-pulsar-kafka-connect-support).\nKCA is used for such popular Pulsar connectors as \n[Pulsar Debezium Source Connectors](https://github.com/apache/pulsar/tree/master/pulsar-io/debezium) and\n[Pulsar Snowflake Sink Connector](https://github.com/datastax/snowflake-connector).\n\nThe connectors built with this project require Datastax Pulsar Luna 2.8+ or Apache Pulsar 2.9+. \n\nFor the details of the status of the specific connector and available connectors, navigate to \n`pulsar-connector/\u003cconnector name\u003e` and check the readme provided by the contributor.\n\nAdded connectors, so far:\n\n1. [Azure Data Explorer (Kusto)](pulsar-connectors/azure-kusto)\n2. [Azure DocumentDB](pulsar-connectors/azure-documentdb)\n3. [Apache Geode](pulsar-connectors/geode)\n4. [Apache Kudu](pulsar-connectors/kudu)\n5. [Apache Phoenix](pulsar-connectors/phoenix)\n6. [Apache PLC4X](pulsar-connectors/plc4x)\n7. [CoAP](pulsar-connectors/coap)\n8. [Couchbase](pulsar-connectors/couchbase)\n9. [DataDog Logs](pulsar-connectors/datadog)\n10. [Diffusion](pulsar-connectors/diffusion)\n11. [Google BigQuery](pulsar-connectors/bigquery)\n12. [Hazelcast Jet](pulsar-connectors/hazelcast)\n13. [Humio HEC](pulsar-connectors/humio)\n14. [JMS](pulsar-connectors/jms)\n15. [Kinetica](pulsar-connectors/kinetica)\n16. [MarkLogic](pulsar-connectors/marklogic)\n17. [MQTT](pulsar-connectors/mqtt)\n18. [Neo4J](pulsar-connectors/neo4j)\n19. [New Relic](pulsar-connectors/newrelic)\n20. [OrientDB](pulsar-connectors/orientdb)\n21. [Redis](pulsar-connectors/redis)\n22. [SAP HANA](pulsar-connectors/sap-hana)\n23. [SingleStore](pulsar-connectors/singlestore)\n24. [Splunk](pulsar-connectors/splunk)\n25. [XTDB](pulsar-connectors/xtdb)\n26. [Zeebe](pulsar-connectors/zeebe)\n1. [camel-aws-cloudwatch-sink](pulsar-connectors/camel-aws-cloudwatch-sink)\n1. [camel-aws-ddb-sink](pulsar-connectors/camel-aws-ddb-sink)\n1. [camel-aws-ddb-streams-source](pulsar-connectors/camel-aws-ddb-streams-source)\n1. [camel-aws-ec2-sink](pulsar-connectors/camel-aws-ec2-sink)\n1. [camel-aws-eventbridge-sink](pulsar-connectors/camel-aws-eventbridge-sink)\n1. [camel-aws-kinesis-firehose-sink](pulsar-connectors/camel-aws-kinesis-firehose-sink)\n1. [camel-aws-kinesis-sink](pulsar-connectors/camel-aws-kinesis-sink)\n1. [camel-aws-kinesis-source](pulsar-connectors/camel-aws-kinesis-source)\n1. [camel-aws-lambda-sink](pulsar-connectors/camel-aws-lambda-sink)\n1. [camel-aws-redshift-sink](pulsar-connectors/camel-aws-redshift-sink)\n1. [camel-aws-redshift-source](pulsar-connectors/camel-aws-redshift-source)\n1. [camel-aws-s3-sink](pulsar-connectors/camel-aws-s3-sink)\n1. [camel-aws-s3-source](pulsar-connectors/camel-aws-s3-source)\n1. [camel-aws-s3-streaming-upload-sink](pulsar-connectors/camel-aws-s3-streaming-upload-sink)\n1. [camel-aws-secrets-manager-sink](pulsar-connectors/camel-aws-secrets-manager-sink)\n1. [camel-aws-ses-sink](pulsar-connectors/camel-aws-ses-sink)\n1. [camel-aws-sns-fifo-sink](pulsar-connectors/camel-aws-sns-fifo-sink)\n1. [camel-aws-sns-sink](pulsar-connectors/camel-aws-sns-sink)\n1. [camel-aws-sqs-batch-sink](pulsar-connectors/camel-aws-sqs-batch-sink)\n1. [camel-aws-sqs-fifo-sink](pulsar-connectors/camel-aws-sqs-fifo-sink)\n1. [camel-aws-sqs-sink](pulsar-connectors/camel-aws-sqs-sink)\n1. [camel-aws-sqs-source](pulsar-connectors/camel-aws-sqs-source)\n1. [camel-aws2-iam](pulsar-connectors/camel-aws2-iam)\n1. [camel-aws2-kms](pulsar-connectors/camel-aws2-kms)\n1. [camel-azure-cosmosdb-source](pulsar-connectors/camel-azure-cosmosdb-source)\n1. [camel-azure-eventhubs-sink](pulsar-connectors/camel-azure-eventhubs-sink)\n1. [camel-azure-eventhubs-source](pulsar-connectors/camel-azure-eventhubs-source)\n1. [camel-azure-functions-sink](pulsar-connectors/camel-azure-functions-sink)\n1. [camel-azure-servicebus-sink](pulsar-connectors/camel-azure-servicebus-sink)\n1. [camel-azure-servicebus-source](pulsar-connectors/camel-azure-servicebus-source)\n1. [camel-azure-storage-blob-changefeed-source](pulsar-connectors/camel-azure-storage-blob-changefeed-source)\n1. [camel-azure-storage-blob-sink](pulsar-connectors/camel-azure-storage-blob-sink)\n1. [camel-azure-storage-blob-source](pulsar-connectors/camel-azure-storage-blob-source)\n1. [camel-azure-storage-queue-sink](pulsar-connectors/camel-azure-storage-queue-sink)\n1. [camel-azure-storage-queue-source](pulsar-connectors/camel-azure-storage-queue-source)\n1. [camel-beer-source](pulsar-connectors/camel-beer-source)\n1. [camel-bitcoin-source](pulsar-connectors/camel-bitcoin-source)\n1. [camel-cassandra-sink](pulsar-connectors/camel-cassandra-sink)\n1. [camel-cassandra-source](pulsar-connectors/camel-cassandra-source)\n1. [camel-ceph-sink](pulsar-connectors/camel-ceph-sink)\n1. [camel-ceph-source](pulsar-connectors/camel-ceph-source)\n1. [camel-chuck-norris-source](pulsar-connectors/camel-chuck-norris-source)\n1. [camel-couchbase-sink](pulsar-connectors/camel-couchbase-sink)\n1. [camel-cron-source](pulsar-connectors/camel-cron-source)\n1. [camel-cxf](pulsar-connectors/camel-cxf)\n1. [camel-cxfrs](pulsar-connectors/camel-cxfrs)\n1. [camel-dropbox-sink](pulsar-connectors/camel-dropbox-sink)\n1. [camel-dropbox-source](pulsar-connectors/camel-dropbox-source)\n1. [camel-earthquake-source](pulsar-connectors/camel-earthquake-source)\n1. [camel-elasticsearch-index-sink](pulsar-connectors/camel-elasticsearch-index-sink)\n1. [camel-elasticsearch-search-source](pulsar-connectors/camel-elasticsearch-search-source)\n1. [camel-exec-sink](pulsar-connectors/camel-exec-sink)\n1. [camel-fhir-source](pulsar-connectors/camel-fhir-source)\n1. [camel-file](pulsar-connectors/camel-file)\n1. [camel-file-watch-source](pulsar-connectors/camel-file-watch-source)\n1. [camel-ftp-sink](pulsar-connectors/camel-ftp-sink)\n1. [camel-ftp-source](pulsar-connectors/camel-ftp-source)\n1. [camel-ftps-sink](pulsar-connectors/camel-ftps-sink)\n1. [camel-ftps-source](pulsar-connectors/camel-ftps-source)\n1. [camel-github-commit-source](pulsar-connectors/camel-github-commit-source)\n1. [camel-github-event-source](pulsar-connectors/camel-github-event-source)\n1. [camel-github-pullrequest-comment-source](pulsar-connectors/camel-github-pullrequest-comment-source)\n1. [camel-github-pullrequest-source](pulsar-connectors/camel-github-pullrequest-source)\n1. [camel-github-tag-source](pulsar-connectors/camel-github-tag-source)\n1. [camel-google-bigquery-sink](pulsar-connectors/camel-google-bigquery-sink)\n1. [camel-google-calendar-source](pulsar-connectors/camel-google-calendar-source)\n1. [camel-google-functions-sink](pulsar-connectors/camel-google-functions-sink)\n1. [camel-google-mail-source](pulsar-connectors/camel-google-mail-source)\n1. [camel-google-pubsub-sink](pulsar-connectors/camel-google-pubsub-sink)\n1. [camel-google-pubsub-source](pulsar-connectors/camel-google-pubsub-source)\n1. [camel-google-sheets-source](pulsar-connectors/camel-google-sheets-source)\n1. [camel-google-storage-sink](pulsar-connectors/camel-google-storage-sink)\n1. [camel-google-storage-source](pulsar-connectors/camel-google-storage-source)\n1. [camel-hdfs](pulsar-connectors/camel-hdfs)\n1. [camel-http-secured-sink](pulsar-connectors/camel-http-secured-sink)\n1. [camel-http-secured-source](pulsar-connectors/camel-http-secured-source)\n1. [camel-http-sink](pulsar-connectors/camel-http-sink)\n1. [camel-http-source](pulsar-connectors/camel-http-source)\n1. [camel-https](pulsar-connectors/camel-https)\n1. [camel-infinispan-sink](pulsar-connectors/camel-infinispan-sink)\n1. [camel-infinispan-source](pulsar-connectors/camel-infinispan-source)\n1. [camel-jdbc](pulsar-connectors/camel-jdbc)\n1. [camel-jira-add-comment-sink](pulsar-connectors/camel-jira-add-comment-sink)\n1. [camel-jira-add-issue-sink](pulsar-connectors/camel-jira-add-issue-sink)\n1. [camel-jira-oauth-source](pulsar-connectors/camel-jira-oauth-source)\n1. [camel-jira-source](pulsar-connectors/camel-jira-source)\n1. [camel-jira-transition-issue-sink](pulsar-connectors/camel-jira-transition-issue-sink)\n1. [camel-jira-update-issue-sink](pulsar-connectors/camel-jira-update-issue-sink)\n1. [camel-jms-amqp-10-sink](pulsar-connectors/camel-jms-amqp-10-sink)\n1. [camel-jms-amqp-10-source](pulsar-connectors/camel-jms-amqp-10-source)\n1. [camel-jms-apache-activemq-sink](pulsar-connectors/camel-jms-apache-activemq-sink)\n1. [camel-jms-apache-activemq-source](pulsar-connectors/camel-jms-apache-activemq-source)\n1. [camel-jms-apache-artemis-sink](pulsar-connectors/camel-jms-apache-artemis-sink)\n1. [camel-jms-apache-artemis-source](pulsar-connectors/camel-jms-apache-artemis-source)\n1. [camel-jms-ibm-mq-sink](pulsar-connectors/camel-jms-ibm-mq-sink)\n1. [camel-jms-ibm-mq-source](pulsar-connectors/camel-jms-ibm-mq-source)\n1. [camel-kafka-not-secured-sink](pulsar-connectors/camel-kafka-not-secured-sink)\n1. [camel-kafka-not-secured-source](pulsar-connectors/camel-kafka-not-secured-source)\n1. [camel-kafka-sink](pulsar-connectors/camel-kafka-sink)\n1. [camel-kafka-source](pulsar-connectors/camel-kafka-source)\n1. [camel-kafka-ssl-sink](pulsar-connectors/camel-kafka-ssl-sink)\n1. [camel-kafka-ssl-source](pulsar-connectors/camel-kafka-ssl-source)\n1. [camel-kubernetes-namespaces-source](pulsar-connectors/camel-kubernetes-namespaces-source)\n1. [camel-kubernetes-nodes-source](pulsar-connectors/camel-kubernetes-nodes-source)\n1. [camel-kubernetes-pods-source](pulsar-connectors/camel-kubernetes-pods-source)\n1. [camel-log-sink](pulsar-connectors/camel-log-sink)\n1. [camel-mail-imap-source](pulsar-connectors/camel-mail-imap-source)\n1. [camel-mail-sink](pulsar-connectors/camel-mail-sink)\n1. [camel-mariadb-sink](pulsar-connectors/camel-mariadb-sink)\n1. [camel-mariadb-source](pulsar-connectors/camel-mariadb-source)\n1. [camel-minio-sink](pulsar-connectors/camel-minio-sink)\n1. [camel-minio-source](pulsar-connectors/camel-minio-source)\n1. [camel-mongodb-changes-stream-source](pulsar-connectors/camel-mongodb-changes-stream-source)\n1. [camel-mongodb-sink](pulsar-connectors/camel-mongodb-sink)\n1. [camel-mongodb-source](pulsar-connectors/camel-mongodb-source)\n1. [camel-mqtt-sink](pulsar-connectors/camel-mqtt-sink)\n1. [camel-mqtt-source](pulsar-connectors/camel-mqtt-source)\n1. [camel-mqtt5-sink](pulsar-connectors/camel-mqtt5-sink)\n1. [camel-mqtt5-source](pulsar-connectors/camel-mqtt5-source)\n1. [camel-mysql-sink](pulsar-connectors/camel-mysql-sink)\n1. [camel-mysql-source](pulsar-connectors/camel-mysql-source)\n1. [camel-nats-sink](pulsar-connectors/camel-nats-sink)\n1. [camel-nats-source](pulsar-connectors/camel-nats-source)\n1. [camel-netty-http](pulsar-connectors/camel-netty-http)\n1. [camel-netty](pulsar-connectors/camel-netty)\n1. [camel-oracle-database-sink](pulsar-connectors/camel-oracle-database-sink)\n1. [camel-oracle-database-source](pulsar-connectors/camel-oracle-database-source)\n1. [camel-postgresql-sink](pulsar-connectors/camel-postgresql-sink)\n1. [camel-postgresql-source](pulsar-connectors/camel-postgresql-source)\n1. [camel-pulsar-sink](pulsar-connectors/camel-pulsar-sink)\n1. [camel-pulsar-source](pulsar-connectors/camel-pulsar-source)\n1. [camel-rabbitmq-source](pulsar-connectors/camel-rabbitmq-source)\n1. [camel-redis-sink](pulsar-connectors/camel-redis-sink)\n1. [camel-redis-source](pulsar-connectors/camel-redis-source)\n1. [camel-rest-openapi-sink](pulsar-connectors/camel-rest-openapi-sink)\n1. [camel-salesforce-create-sink](pulsar-connectors/camel-salesforce-create-sink)\n1. [camel-salesforce-delete-sink](pulsar-connectors/camel-salesforce-delete-sink)\n1. [camel-salesforce-source](pulsar-connectors/camel-salesforce-source)\n1. [camel-salesforce-update-sink](pulsar-connectors/camel-salesforce-update-sink)\n1. [camel-scp-sink](pulsar-connectors/camel-scp-sink)\n1. [camel-sftp-sink](pulsar-connectors/camel-sftp-sink)\n1. [camel-sftp-source](pulsar-connectors/camel-sftp-source)\n1. [camel-sjms2](pulsar-connectors/camel-sjms2)\n1. [camel-slack-sink](pulsar-connectors/camel-slack-sink)\n1. [camel-slack-source](pulsar-connectors/camel-slack-source)\n1. [camel-solr-sink](pulsar-connectors/camel-solr-sink)\n1. [camel-solr-source](pulsar-connectors/camel-solr-source)\n1. [camel-splunk-hec-sink](pulsar-connectors/camel-splunk-hec-sink)\n1. [camel-splunk-sink](pulsar-connectors/camel-splunk-sink)\n1. [camel-splunk-source](pulsar-connectors/camel-splunk-source)\n1. [camel-sqlserver-sink](pulsar-connectors/camel-sqlserver-sink)\n1. [camel-sqlserver-source](pulsar-connectors/camel-sqlserver-source)\n1. [camel-ssh-sink](pulsar-connectors/camel-ssh-sink)\n1. [camel-ssh-source](pulsar-connectors/camel-ssh-source)\n1. [camel-syslog](pulsar-connectors/camel-syslog)\n1. [camel-telegram-sink](pulsar-connectors/camel-telegram-sink)\n1. [camel-telegram-source](pulsar-connectors/camel-telegram-source)\n1. [camel-timer-source](pulsar-connectors/camel-timer-source)\n1. [camel-twitter-directmessage-source](pulsar-connectors/camel-twitter-directmessage-source)\n1. [camel-twitter-search-source](pulsar-connectors/camel-twitter-search-source)\n1. [camel-twitter-timeline-source](pulsar-connectors/camel-twitter-timeline-source)\n1. [camel-webhook-source](pulsar-connectors/camel-webhook-source)\n1. [camel-websocket-source](pulsar-connectors/camel-websocket-source)\n1. [camel-wttrin-source](pulsar-connectors/camel-wttrin-source)\n\u003c!--- add new connector modules here ---\u003e\n\nThe rest of this documentation will dive into details of:\n\n* How to build connectors\n* How to use connectors\n* How to add a connector\n\n## Building the connectors\n\nEnsure you have JDK 11+ and Maven 3.8 installed.\n\nClone the connector's repo and run `mvn clean install` from the root. \n\nThe connector's `.nar` files can be found at `pulsar-connectors/\u003cconnector name\u003e\u003e/target/pulsar-3rdparty-pulsar-connectors-\u003cconnector name\u003e-0.1.0-SNAPSHOT.nar`\n\n## Using the connectors\n\nFollow [Pulsar's documentation](https://pulsar.apache.org/docs/en/io-use/) to use the packaged connector.\n\n### Providing configuration for connectors\n\n#### Sink Connectors\n\nFollow the example below to create a config yaml file: \n\n```yaml\n# Pulsar KCA Sink expects \"processingGuarantees\" to be \"EFFECTIVELY_ONCE\"`\nprocessingGuarantees: \"EFFECTIVELY_ONCE\"\nconfigs:\n  # Size of messages in bytes the sink will attempt to batch messages together before flush.\n  # batchSize: 16384\n  # Time interval in milliseconds the sink will attempt to batch messages together before flush.\n  # lingerTimeMs: 2147483647\n  # In case of Record\u003cKeyValue\u003c\u003e\u003e data use key from KeyValue\u003c\u003e instead of one from Record.\n  # unwrapKeyValueIfAvailable: \"true\"\n  # The Kafka topic name that passed to Kafka sink.\n  topic: \"my-topic\"\n  # Pulsar topic to store offsets at.\n  offsetStorageTopic: \"kafka-connect-sink-offsets\"\n  # A Kafka connector sink class to use.\n  kafkaConnectorSinkClass: \"com.third.party.CoolSinkConnector\"\n  # Config properties to pass to the Kafka connector.\n  kafkaConnectorConfigProperties:\n    # The following properties passed directly to Kafka Connect Sink and defined by it or by\n    # https://github.com/apache/kafka/blob/2.7/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/ConnectorConfig.java\n    name: \"test-sink\"\n    connector.class: \"com.third.party.CoolSinkConnector\"\n    tasks.max: \"1\"\n    topics: \"my-topic\"\n    ...\n```\n\n#### Source Connectors\n\nFollow the example below to create a config yaml file:\n\n```yaml\ntenant: \"public\"\nnamespace: \"default\"\nname: \"test-source\"\ntopicName: \"test-topic\"\nparallelism: 1\n# A Kafka connector source class to use.\nclassName: \"com.third.party.CoolSourceConnector\"\nconfigs:\n  # Present the message only consist of payload.\n  # json-with-envelope: \"false\"\n\n  # Pulsar topic to store Kafka connector offsets at\n  offset.storage.topic: \"kafka-connect-source-offsets\"\n  # Pulsar namespace to store the output topics\n  topic.namespace: \"public/default\"\n  \n  # Config properties to pass to the Kafka connector.\n  # The following properties passed directly to Kafka Connect Sink and defined by it or by\n  # https://github.com/apache/kafka/blob/2.7/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/ConnectorConfig.java\n\n  # A Kafka connector source class to use.\n  task.class: \"com.third.party.CoolSourceConnector\"\n  # The converter provided by Kafka Connect to convert record value.\n  value.converter: \"org.apache.kafka.connect.json.JsonConverter\"\n  # The converter provided by Kafka Connect to convert record key.\n  key.converter: \"org.apache.kafka.connect.json.JsonConverter\"\n  ...\n```\n\n## Adding a new connector\n\nThese steps help avoid some common problems encountered while using KCA to create a new connector:\n\n* Ensure the connector's license allows its use and redistribution. A helpful starting point is [here](https://www.apache.org/legal/resolved.html).  \n* Maven dependency conflict of transitive dependencies in build time.\n* Dependency conflict in runtime caused by third-party dependencies packaged with the connector.\n\n### 1. Decide if you need to shade the original connector\n\nCheck the content of the Kafka connector's jar file. If it includes third-party dependencies, \nyou may need to \"shade\" it (rename some classes).\n\nTo do so, copy `shaded-dependencies/template-shaded/` to `shaded-dependencies/\u003cconnector name\u003e` \nand add the new module into `shaded-dependencies/pom.xml`.\n\nEnsure that third-party dependencies are renamed as specified \nin `shaded-dependencies/\u003cconnector name\u003e/pom.xml`and build (`mvn clean install`).\n\n### 2. Add new subproject\n\n1. Copy `pulsar-connectors/template/` to `pulsar-connectors/\u003cconnector name\u003e/`\n2. Add the new module into `pulsar-connectors/pom.xml`\n3. Update connector's name and description in `pulsar-connectors/\u003cconnector name\u003e/src/main/resources/META-INF/services/pulsar-io.yaml`\n4. Update the `pulsar-connectors/\u003cconnector name\u003e/README.md`\n5. Update the root `README.md`\n6. Update `LICENSE` and `NOTICE` files\n7. Build (`mvn clean install`). \n8. Run `mvn dependency:tree -Dverbose` to review how Maven auto-resolved potential dependency conflicts and fix as needed\n\nTo check the connector for CVEs:\n```shell\nmvn clean install verify -Powasp-dependency-check -DskipTests -f pulsar-connectors/\u003cconnector dir\u003e/pom.xml\n```\nDetailed report will be at `pulsar-connectors/\u003cconnector dir\u003e/target/dependency-check-report.html`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatastax%2Fpulsar-3rdparty-connector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatastax%2Fpulsar-3rdparty-connector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatastax%2Fpulsar-3rdparty-connector/lists"}