{"id":22096869,"url":"https://github.com/streamnative/pulsar-io-template","last_synced_at":"2025-07-24T22:32:34.890Z","repository":{"id":42977805,"uuid":"168632336","full_name":"streamnative/pulsar-io-template","owner":"streamnative","description":"It is a project template for developing an Apache Pulsar connector","archived":false,"fork":false,"pushed_at":"2024-04-24T09:59:40.000Z","size":54,"stargazers_count":7,"open_issues_count":3,"forks_count":6,"subscribers_count":30,"default_branch":"master","last_synced_at":"2024-04-24T15:09:49.630Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/streamnative.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-02-01T02:46:54.000Z","updated_at":"2024-04-24T09:59:44.000Z","dependencies_parsed_at":"2024-04-24T11:01:11.613Z","dependency_job_id":"3654d406-954a-45ad-adb3-b2f3af84fb50","html_url":"https://github.com/streamnative/pulsar-io-template","commit_stats":null,"previous_names":[],"tags_count":0,"template":true,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamnative%2Fpulsar-io-template","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamnative%2Fpulsar-io-template/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamnative%2Fpulsar-io-template/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/streamnative%2Fpulsar-io-template/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/streamnative","download_url":"https://codeload.github.com/streamnative/pulsar-io-template/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227482464,"owners_count":17779968,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-01T04:12:58.189Z","updated_at":"2024-12-01T04:12:58.677Z","avatar_url":"https://github.com/streamnative.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Connector Development Guide\n\nWelcome to contribute to [StreamNative Hub](https://streamnative.io/en/blog/tech/2020-05-26-intro-to-hub)! You can submit connectors around [Apache Pulsar](https://pulsar.apache.org/en/) and [StreamNatvie](https://streamnative.io/) ecosystems and host them on [StreamNative Hub](https://hub.streamnative.io/).\n\nThis instruction guides you through every step of submitting a connector to [StreamNative Hub](https://hub.streamnative.io/), including the workflow for both code and doc.\n\n## Code\n\n[streamnative / pulsar-io-template](https://github.com/streamnative/pulsar-io-template) is a project template for developing an enterprise-grade Pulsar connector. It sets up a project structure and contains necessary dependencies and plugins. \n\nTo develop a connector quicker and easier, you can clone this project template.\n\nThis example develops a Pulsar connector named `pulsar-io-foo`.\n\n1. **Create your connector project**. \n\n    (1) On your local machine anywhere, clone the [pulsar-io-template](https://github.com/streamnative/pulsar-io-template) to create the `pulsar-io-foo` project.\n\n    ```bash\n    git clone --bare https://github.com/streamnative/pulsar-io-template.git pulsar-io-foo\n    ```\n\n    You will get the following directories to host different files.\n\n    ```bash\n    ├── conf // stores configuration examples.\n    ├── docs // stores user guides.\n    ├── src // stores source codes.\n    │   ├── checkstyle // stores checkstyle configuration files.\n    │   ├── license // stores license headers. You can use `mvn license:format` to format the project with the stored license header.\n    │   │   └── ALv2\n    │   ├── main // stores all main source files.\n    │   │   └── java\n    │   ├── spotbugs // stores spotbugs configuration files.\n    │   └── test // stores all related tests.\n    │ \n    ```\n\n    (2) Push the `pulsar-io-foo` project to your GitHub account.\n\n    ```\n    cd pulsar-io-foo\n    git push https://github.com/\u003cyour-github-account\u003e/pulsar-io-foo\n    ```\n\n2. **Develop your connector**.\n\n    (1) To customize your connector, update the following configurations in the [`pom`](https://github.com/streamnative/pulsar-io-template/blob/c415593b04d40a868bd3d51b5d399569a50a4b67/pom.xml) file.\n\n    Configuration|Description\n    |---|---\n    `artifactId`|Update `artifactId` to your connector name.\n    `version` |Update `version` to the desired connector version.\u003cbr\u003e\u003cbr\u003e**Tip**: it is recommended to define the connector’s version NO. same as Pulsar’s version NO. In this way, it is easy to figure out the version relationships between the connector and Pulsar.\n    `name`|Update `name` to `Pulsar Ecosystem :: IO Connector :: \u003cyour-connector-name\u003e`.\n    `description`|Update `description` to the descriptions of your connector.\n\n    (2) Create a package `org.apache.pulsar.ecosystem.io.foo` in `src/main/java/org/apache/pulsar/ecosystem/io` to develop your connector logic. \n\n    \u003e **Tip**\n    \u003e\n    \u003e - [Here](https://github.com/streamnative/pulsar-io-template/tree/master/src/main/java/org/apache/pulsar/ecosystem/io/random) are some code examples for developing a connector.\n    \u003e\n    \u003e - To ensure a consistent codebase, it is recommended to run the checkstyle and spotbugs. For more information, see [check your code](#check-your-code). \n\n3. **Test your connector**.\n\n    Create a package `org.apache.pulsar.ecosystem.io.foo` in the `src/test` directory to develop your connector tests. For more information, see [how to write a unit test in Java](https://www.webucator.com/how-to/how-write-unit-test-java.cfm).\n\n    \u003e **Tip**\n    \u003e \n    \u003e - It is strongly recommended to write tests for your connector. For more information, see [test examples](https://github.com/streamnative/pulsar-io-template/tree/master/src/test/java/org/apache/pulsar/ecosystem/io/random).\n    \u003e\n    \u003e - To ensure a consistent codebase, it is recommended to run the checkstyle and spotbugs. For more information, see [check your code](#check-your-code). \n\n4. **Check your code**.\n\n    To ensure a consistent codebase, it is recommended to run the checkstyle and spotbugs, which are already set by the project template.\n    \n    (1) Run the checkstyle.\n\n    ```bash\n    mvn checkstyle:check\n    ```\n\n    (2) Run the spotbugs.\n\n    ```bash\n    mvn spotbugs:check\n    ```\n\n5. **Choose a license**.\n\n    You can choose [Apache License 2.0](https://github.com/streamnative/pulsar-io-template/blob/master/LICENSE). For more information, see [choose an open source license](https://choosealicense.com/).\n\n    After choosing the license, you need to finish the following tasks.\n\n    (1) Replace the [`LICENSE`](https://github.com/streamnative/pulsar-io-template/blob/master/LICENSE) file in the project template with your desired license.\n\n    (2) Add your license header to the `src/license/\u003cyour-license-header\u003e.txt` file.\n\n    (3) Update the license-maven-plugin configuration in the `pom.xml` file to point to your license header.\n\n    (4) Run the `license:format` command to format the project with your license.\n\n## Doc\n\nDocumentation is one of the maturity indicators for measuring a project. High-quality user documentation helps users to use the connector efficiently and improves users’ experiences.\n\nYou can maintain documentation using the same tools and processes that used to maintain your connector code.\n\n1. **Create a `docs` repo**.\n\n    In your root directory of connector repo, create a doc repo named `docs`.\n\n    \u003e **Note**\n    \u003e\n    \u003e Name the repo with `docs` instead of other names, or else your docs are not synced to StreamNative Hub.\n\n    \u003cimg width=\"589\" alt=\"1\" src=\"https://user-images.githubusercontent.com/50226895/119435808-85cf0300-bd4d-11eb-97b2-ab63da049d5e.png\"\u003e\n\n2. **Write the docs** for each type of connector.\n\n    **Example**\n\n    - [`sqs-source.md`](https://github.com/streamnative/pulsar-io-sqs/blob/master/docs/sqs-source.md)\n\n    - [`sqs-sink.md`](https://github.com/streamnative/pulsar-io-sqs/blob/master/docs/sqs-sink.md)\n    \n    \u003cimg width=\"705\" alt=\"2\" src=\"https://user-images.githubusercontent.com/50226895/119435868-a303d180-bd4d-11eb-953b-7de91608b4e0.png\"\u003e\n\n    \u003e **Note**\n    \u003e\n    \u003e -  [Here](https://github.com/streamnative/pulsar-io-sqs/tree/master/docs) are examples\n of the SQS connector docs. It is strongly recommended to follow the doc architecture in this example, or else your docs might not be synced to StreamNative Hub.\n    \u003e\n    \u003e - The docs on these branches (master, branch-x.x.x. For example, branch-2.7.0) and tags (vx.x.x. For example, v2.7.1, v2.7.2) can be shown on the StreamNative Hub website. Or else the docs on other branches and tags are not shown on the StreamNative Hub website. Pay attention to your branch and tag names.\n    \u003e \u003cimg width=\"616\" alt=\"3\" src=\"https://user-images.githubusercontent.com/50226895/119435914-ba42bf00-bd4d-11eb-9f83-409eb78f219d.png\"\u003e\n    \u003e \u003cimg width=\"614\" alt=\"4\" src=\"https://user-images.githubusercontent.com/50226895/119435917-bc0c8280-bd4d-11eb-820c-cebb74cc6c55.png\"\u003e\n\n\n    \u003e **Tip**\n    \u003e\n    \u003e - To reduce maintenance costs, use a variable (for example, `{{connector:version}}`) instead of writing the specific connector version. In this way, the corresponding connector version is generated and shown automatically along with each connector release. \n    \u003e\n    \u003e - To ensure consistency throughout all connector documentation, it is recommended to follow the [Google Developer Documentation Style Guide](https://developers.google.com/style).\n\n\n3. **Sync the docs** to [StreamNative Hub](https://hub.streamnative.io/).\n\n    Create YAML files in your connector doc repository. The backend script syncs docs from your connector repository to [StreamNative Hub](https://hub.streamnative.io/) automatically using the YAML files.\n\n    ```\n    connectors/\u003cyour-connector-repository-name\u003e/\u003cyour-connector-repository-name\u003e.yaml\n    ```\n\n    **Example**\n\n    - [`sqs-source.yaml`](https://github.com/streamnative/pulsar-hub/blob/master/connectors/sqs-source/sqs-source.yaml)\n    \u003cimg width=\"744\" alt=\"6\" src=\"https://user-images.githubusercontent.com/50226895/119436182-48b74080-bd4e-11eb-9db6-e0e7ea68de50.png\"\u003e\n    \n    - [`sqs-sink.yaml`](https://github.com/streamnative/pulsar-hub/blob/master/connectors/sqs-sink/sqs-sink.yaml)\n    \u003cimg width=\"747\" alt=\"7\" src=\"https://user-images.githubusercontent.com/50226895/119436191-49e86d80-bd4e-11eb-942a-cd078dc745d7.png\"\u003e\n\n\n4. Send a PR and **request docs review**.\n\n    Your doc will be reviewed by connector maintainers. You will work together to finalize the doc.\n\n    [Here](https://github.com/streamnative/pulsar-hub/pull/139/files) is a PR example.\n    \n## Blog\n\nAfter finishing the code and documentation, you can create a post to announce the connector. You can contact us to publish it on the [StreamNative website](https://streamnative.io/blog) and we can work together to promote it to multiple channels.\n\n**Example**\n\n- [Announcing AWS SQS Connector for Apache Pulsar](https://streamnative.io/en/blog/tech/2021-03-17-announcing-aws-sqs-connector-for-apache-pulsar)\n\n## More info\n\nIf you have any questions about contributing your connector to [StreamNative Hub](https://hub.streamnative.io/), feel free to [open an issue](https://github.com/streamnative/pulsar-io-template/issues/new/choose) to discuss it with us or [contact us](https://streamnative.io/en/contact). We look forward to your contribution!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamnative%2Fpulsar-io-template","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstreamnative%2Fpulsar-io-template","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamnative%2Fpulsar-io-template/lists"}