{"id":21001675,"url":"https://github.com/zncdatadev/spark-k8s-operator","last_synced_at":"2025-06-10T18:05:42.722Z","repository":{"id":187399772,"uuid":"676398572","full_name":"zncdatadev/spark-k8s-operator","owner":"zncdatadev","description":"Operator for Apache Spark-on-Kubernetes of the Kubedoop","archived":false,"fork":false,"pushed_at":"2025-05-11T13:26:08.000Z","size":865,"stargazers_count":1,"open_issues_count":4,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-11T14:29:18.072Z","etag":null,"topics":["k8s","kubernetes","spark"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zncdatadev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-08-09T05:41:44.000Z","updated_at":"2025-05-11T13:24:21.000Z","dependencies_parsed_at":"2023-09-14T11:07:56.900Z","dependency_job_id":"8166bf83-cf38-4b0f-b351-798bf2d6b460","html_url":"https://github.com/zncdatadev/spark-k8s-operator","commit_stats":null,"previous_names":["zncdata-labs/spark-history-operator","zncdata-labs/spark-k8s-operator","zncdatadev/spark-k8s-operator"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zncdatadev%2Fspark-k8s-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zncdatadev%2Fspark-k8s-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zncdatadev%2Fspark-k8s-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zncdatadev%2Fspark-k8s-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zncdatadev","download_url":"https://codeload.github.com/zncdatadev/spark-k8s-operator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254248453,"owners_count":22039008,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["k8s","kubernetes","spark"],"created_at":"2024-11-19T08:16:09.214Z","updated_at":"2025-06-10T18:05:42.710Z","avatar_url":"https://github.com/zncdatadev.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kubedoop Operator for Apache Spark\n\n[![Build](https://github.com/zncdatadev/spark-k8s-operator/actions/workflows/publish.yml/badge.svg)](https://github.com/zncdatadev/spark-k8s-operator/actions/workflows/publish.yml)\n[![LICENSE](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Go Report Card](https://goreportcard.com/badge/github.com/zncdatadev/spark-k8s-operator)](https://goreportcard.com/report/github.com/zncdatadev/spark-k8s-operator)\n[![Artifact HUB](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/spark-k8s-operator)](https://artifacthub.io/packages/helm/kubedoop/spark-k8s-operator)\n\nThis is a Kubernetes operator to manage Apache Spark jobs.\n\nIt's part of the kubedoop Data Platform, a modular open source data platform built on Kubernetes that provides Kubernetes native deployment\nand management of popular open source data apps like Apache Kafka, Apache Doris, Apache Kyuubi, Trino or Apache Spark, all working\ntogether seamlessly. Based on Kubernetes, it runs everywhere – on prem or in the cloud.\n\n## Quick Start\n\n### Add helm repository\n\n\u003e Please make sure helm version is v3.0.0+\n\n```bash\nhelm repo add kubedoop https://zncdatadev.github.io/kubedoop-helm-charts/\n```\n\n### Add required dependencies\n\n```bash\nhelm install commons-operator kubedoop/commons-operator\nhelm install listener-operator kubedoop/listener-operator\nhelm install secret-operator kubedoop/secret-operator\n```\n\n### Add spark-k8s-operator\n\n```bash\nhelm install spark-k8s-operator kubedoop/spark-k8s-operator\n```\n\n### Deploy spark-k8s cluster\n\n```bash\nkubectl apply -f config/samples\n```\n\n## Kubedoop Data Platform Operators\n\nThese are the operators that are currently part of the Kubedoop Data Platform:\n\n- [Kubedoop Operator for Apache Airflow](https://github.com/zncdatadev/airflow-operator)\n- [Kubedoop Operator for Apache DolphinScheduler](https://github.com/zncdatadev/dolphinscheduler-operator)\n- [Kubedoop Operator for Apache Doris](https://github.com/zncdatadev/doris-operator)\n- [Kubedoop Operator for Apache Hadoop HDFS](https://github.com/zncdatadev/hdfs-operator)\n- [Kubedoop Operator for Apache HBase](https://github.com/zncdatadev/hbase-operator)\n- [Kubedoop Operator for Apache Hive](https://github.com/zncdatadev/hive-operator)\n- [Kubedoop Operator for Apache Kafka](https://github.com/zncdatadev/kafka-operator)\n- [Kubedoop Operator for Apache Kyuubi](https://github.com/zncdatadev/kyuubi-operator)\n- [Kubedoop Operator for Apache Nifi](https://github.com/zncdatadev/nifi-operator)\n- [Kubedoop Operator for Apache Spark](https://github.com/zncdatadev/spark-k8s-operator)\n- [Kubedoop Operator for Apache Superset](https://github.com/zncdatadev/superset-operator)\n- [Kubedoop Operator for Trino](https://github.com/zncdatadev/trino-operator)\n- [Kubedoop Operator for Apache Zookeeper](https://github.com/zncdatadev/zookeeper-operator)\n\nAnd our internal operators: :\n\n- [Commons Operator](https://github.com/zncdatadev/commons-operator)\n- [Listener Operator](https://github.com/zncdatadev/listener-operator)\n- [Secret Operator](https://github.com/zncdatadev/secret-operator)\n\n## Contributing\n\nIf you'd like to contribute to Kubedoop, please refer to our [Contributing Guide](https://kubedoop.dev/docs/developer-manual/collaboration) for more information.\nWe welcome contributions of all kinds, including but not limited to code, documentation, and use cases.\n\n## License\n\nCopyright 2024.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzncdatadev%2Fspark-k8s-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzncdatadev%2Fspark-k8s-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzncdatadev%2Fspark-k8s-operator/lists"}