{"id":13742577,"url":"https://github.com/sky-big/pulsar-operator","last_synced_at":"2025-06-30T02:36:59.709Z","repository":{"id":55978767,"uuid":"209729377","full_name":"sky-big/pulsar-operator","owner":"sky-big","description":"Pulsar Operator creates/configures/manages pulsar clusters atop Kubernetes","archived":false,"fork":false,"pushed_at":"2020-12-03T08:23:15.000Z","size":32007,"stargazers_count":74,"open_issues_count":6,"forks_count":21,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-06-13T17:08:40.052Z","etag":null,"topics":["kubernetes","kubernetes-operator","operator-sdk","pulsar","pulsar-operator"],"latest_commit_sha":null,"homepage":"https://github.com/sky-big/pulsar-operator","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sky-big.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-20T07:15:48.000Z","updated_at":"2025-04-25T17:22:48.000Z","dependencies_parsed_at":"2022-08-15T10:40:45.323Z","dependency_job_id":null,"html_url":"https://github.com/sky-big/pulsar-operator","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sky-big/pulsar-operator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sky-big%2Fpulsar-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sky-big%2Fpulsar-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sky-big%2Fpulsar-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sky-big%2Fpulsar-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sky-big","download_url":"https://codeload.github.com/sky-big/pulsar-operator/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sky-big%2Fpulsar-operator/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262699851,"owners_count":23350386,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kubernetes","kubernetes-operator","operator-sdk","pulsar","pulsar-operator"],"created_at":"2024-08-03T05:00:33.763Z","updated_at":"2025-06-30T02:36:59.671Z","avatar_url":"https://github.com/sky-big.png","language":"Go","funding_links":[],"categories":["Repository is obsolete"],"sub_categories":["Awesome Operators in the Wild"],"readme":"# Pulsar Operator\n\n## Overview\n\nPulsar Operator is to manage Pulsar service instances deployed on the Kubernetes cluster.\nIt is built using the [Operator SDK](https://github.com/operator-framework/operator-sdk), which is part of the [Operator Framework](https://github.com/operator-framework/).\n\n## Quick Start\n\n### Deploy Pulsar Operator\n\n1. Clone the project on your Kubernetes cluster master node:\n```\n$ git clone https://github.com/sky-big/pulsar-operator.git\n$ cd pulsar-operator\n```\n\n2. To deploy the Pulsar Operator on your Kubernetes cluster, please run the following script:\n```\n$ make install\n```\n\n3. Use command ```kubectl get pods``` to check Pulsar Operator deploy status like:\n```\n$ kubectl get pods\nNAME                                      READY   STATUS    RESTARTS   AGE\npulsar-operator-564b5d75d-jllzk           1/1     Running   0          108s\n```\n\nNow you can use the CRDs provide by Pulsar Operator to deploy your Pulsar Cluster.\n\n### Define Your Pulsar Cluster\n\n1. Check the file ```pulsar_v1alpha1_pulsarcluster_cr.yaml```in the deploy/crd directory, for example:\n```\napiVersion: pulsar.apache.org/v1alpha1\nkind: PulsarCluster\nmetadata:\n  name: example-pulsarcluster\nspec:\n  zookeeper:\n    size: 3\n  autoRecovery:\n    size: 3\n  bookie:\n    size: 3\n  broker:\n    size: 3\n  proxy:\n    size: 3\n```\n\nwhich defines pulsar cluster zookeeper, bookkeeper, broker, proxy components configuration\n\n2. If you need pulsar prometheus, grafana need configuration, for example:\n```\napiVersion: pulsar.apache.org/v1alpha1\nkind: PulsarCluster\nmetadata:\n  name: example-pulsarcluster\nspec:\n  zookeeper:\n    size: 3\n  autoRecovery:\n    size: 3\n  bookie:\n    size: 3\n  broker:\n    size: 3\n  proxy:\n    size: 3\n  monitor:\n    enable: true                // true/false: active monitor\n    prometheusPort: 30002       // prometheus expose port on kubernetes\n    grafanaPort: 30003          // grafana expose port on kubernetes\n    ingress:                    // ingress configuration\n      enable: true\n      annotations:\n        kubernetes.io/ingress.class: \"nginx\"\n```\n\n3. If you need pulsar manager expose by ingress, for example:\n```\napiVersion: pulsar.apache.org/v1alpha1\nkind: PulsarCluster\nmetadata:\n  name: example-pulsarcluster\nspec:\n  zookeeper:\n    size: 3\n  autoRecovery:\n    size: 3\n  bookie:\n    size: 3\n  broker:\n    size: 3\n  proxy:\n    size: 3\n  manager:\n    enable: true                            // true/false: active manager\n    host: manager.pulsar.com\n    annotations:\n      kubernetes.io/ingress.class: \"nginx\"\n```\n\n### Create Your Pulsar Cluster\n\n1. Deploy the pulsar cluster by running:\n```\n$ kubectl create -f deploy/crds/pulsar_v1alpha1_pulsarcluster_cr.yaml\n```\n\n2. Use command ```kubectl get pods```to check example Pulsar Cluster status like:\n```\n$ kubectl get pods\nNAME                                                              READY   STATUS      RESTARTS   AGE\nexample-pulsarcluster-bookie-autorecovery-deployment-6775f2sbxj   1/1     Running     0          67s\nexample-pulsarcluster-bookie-autorecovery-deployment-6775fdqhmb   1/1     Running     0          67s\nexample-pulsarcluster-bookie-autorecovery-deployment-6775ftnftd   1/1     Running     0          67s\nexample-pulsarcluster-bookie-statefulset-0                        1/1     Running     0          68s\nexample-pulsarcluster-bookie-statefulset-1                        1/1     Running     0          55s\nexample-pulsarcluster-bookie-statefulset-2                        1/1     Running     0          42s\nexample-pulsarcluster-broker-deployment-5bb58577b4-4tr4l          1/1     Running     0          67s\nexample-pulsarcluster-broker-deployment-5bb58577b4-6vzhm          1/1     Running     0          67s\nexample-pulsarcluster-broker-deployment-5bb58577b4-mzphh          1/1     Running     0          67s\nexample-pulsarcluster-init-cluster-metadata-job-98rsd             0/1     Completed   0          80s\nexample-pulsarcluster-proxy-deployment-6555968487-7df5l           1/1     Running     0          67s\nexample-pulsarcluster-proxy-deployment-6555968487-cfxl7           1/1     Running     0          67s\nexample-pulsarcluster-proxy-deployment-6555968487-cxhc6           1/1     Running     0          67s\nexample-pulsarcluster-zookeeper-statefulset-0                     1/1     Running     0          2m2s\nexample-pulsarcluster-zookeeper-statefulset-1                     1/1     Running     0          109s\nexample-pulsarcluster-zookeeper-statefulset-2                     1/1     Running     0          95s\n```\n\n### Storage of Pulsar Cluster\n\n1. Use EmptyDir Volume For Test(Default)\n\n2. Use PVC For Storage\n```\napiVersion: pulsar.apache.org/v1alpha1\nkind: PulsarCluster\nmetadata:\n  name: example-pulsarcluster\nspec:\n  zookeeper:\n    size: 3\n  bookie:\n    size: 3\n    storageClassName: \"fast\"                // storage class name\n    journalStorageCapacity: 10              // bookie journal volume capacity(Gi)\n    ledgersStorageCapacity: 10              // bookie ledgers volume capacity(Gi)\n  broker:\n    size: 3\n  proxy:\n    size: 3\n    \n$ kubectl get pvc\nNAME                                                                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE\njournal-disk-volume-pvc-example-pulsarcluster-bookie-statefulset-0   Bound    pvc-8c7247e0-9a1f-4eba-9a1f-8ad10847d54d   10Gi       RWO            fast           2m\njournal-disk-volume-pvc-example-pulsarcluster-bookie-statefulset-1   Bound    pvc-27e5b83b-6ea2-47e2-a64e-783772506959   10Gi       RWO            fast           84s\njournal-disk-volume-pvc-example-pulsarcluster-bookie-statefulset-2   Bound    pvc-88ccca2e-0f6b-4ae9-b5c6-1b7bb57bdc69   10Gi       RWO            fast           62s\nledgers-disk-volume-pvc-example-pulsarcluster-bookie-statefulset-0   Bound    pvc-3afc1103-7992-41e1-9aa5-762681ac5e5d   10Gi       RWO            fast           2m\nledgers-disk-volume-pvc-example-pulsarcluster-bookie-statefulset-1   Bound    pvc-dd848627-6838-4cda-be77-83fe5b0b7c53   10Gi       RWO            fast           84s\nledgers-disk-volume-pvc-example-pulsarcluster-bookie-statefulset-2   Bound    pvc-98aea962-1a7a-4c4c-a058-cef9bccacb6c   10Gi       RWO            fast           62s\n```\n\n## Horizontal Scale Pulsar Cluster\n\n### Scale Pulsar Proxy\n\n1. If you want to enlarge your proxy component. Modify your CR file pulsar_v1alpha1_pulsarcluster_cr.yaml, increase the field size to the number you want, for example, from ```size: 3``` to ```size: 5```\n```\napiVersion: pulsar.apache.org/v1alpha1\nkind: PulsarCluster\nmetadata:\n  name: example-pulsarcluster\nspec:\n  proxy:\n    size: 5\n```\n\n2. After configuring the size fields, simply run:\n```\n$ kubectl apply -f deploy/crds/pulsar_v1alpha1_pulsarcluster_cr.yaml\n```\n\n### Scale Pulsar Broker\n\n1. If you want to enlarge your broker component. Modify your CR file pulsar_v1alpha1_pulsarcluster_cr.yaml, increase the field size to the number you want, for example, from ```size: 3``` to ```size: 5```\n```\napiVersion: pulsar.apache.org/v1alpha1\nkind: PulsarCluster\nmetadata:\n  name: example-pulsarcluster\nspec:\n  broker:\n    size: 5\n```\n\n2. After configuring the size fields, simply run:\n```\n$ kubectl apply -f deploy/crds/pulsar_v1alpha1_pulsarcluster_cr.yaml\n```\n\n### Scale Pulsar Bookie\n\n1. If you want to enlarge your bookie component. Modify your CR file pulsar_v1alpha1_pulsarcluster_cr.yaml, increase the field size to the number you want, for example, from ```size: 3``` to ```size: 5```\n```\napiVersion: pulsar.apache.org/v1alpha1\nkind: PulsarCluster\nmetadata:\n  name: example-pulsarcluster\nspec:\n  bookie:\n    size: 5\n```\n\n2. After configuring the size fields, simply run:\n```\n$ kubectl apply -f deploy/crds/pulsar_v1alpha1_pulsarcluster_cr.yaml\n```\n\n## Local Test On Kubernetes\n\n## Start Local Test\n\n1. Install kubernetes cluster\n\n2. Install golang on kubernetes master node\n\n3. Install operator sdk[Install Operator SDK](https://github.com/operator-framework/operator-sdk/blob/master/doc/user/install-operator-sdk.md) on kubernetes master node\n\n4. Clone project to the kubernetes master node(project must in GOPATH):\n```\n$ git clone https://github.com/sky-big/pulsar-operator.git\n$ cd pulsar-operator\n```\n\n5. Execute script on kubernetes master node:\n```\n$ make start-local\n```\n\n## Stop Local Test\n1. Execute script on kubernetes master node:\n```\n$ make stop-local\n```\n\n## Compile Pulsar Operator\n```\n$ make build\n```\n\n## Build Pulsar Operator Image\n```\n$ make image\n```\n\n## Generate CRD Code And Project Vendor Code\n```\n$ make generate\n```\n\n## Discussions Happen Here\n\n### Slack\n\n(1). [join](https://join.slack.com/t/pulsaroperator/shared_invite/enQtOTE0MzIwOTEwOTY0LTMyNGEzYTU2OTZlMGE4MWJlOTI3NmM1ZWVlNjQ1NTc5YzMyMzRmNGRhN2EyNzY2ODAxMDNkNDEyOWUwYjQyZjY)\n\n(2). [pulsar-operator-dev](https://app.slack.com/client/TSK90ANQK/CSYV5L9FG)\n\n(3). [pulsar-operator-user](https://app.slack.com/client/TSK90ANQK/CSLHRDVCH)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsky-big%2Fpulsar-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsky-big%2Fpulsar-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsky-big%2Fpulsar-operator/lists"}