https://github.com/zncdatadev/spark-k8s-operator
Operator for Apache Spark-on-Kubernetes of the Kubedoop
https://github.com/zncdatadev/spark-k8s-operator
k8s kubernetes spark
Last synced: about 1 month ago
JSON representation
Operator for Apache Spark-on-Kubernetes of the Kubedoop
- Host: GitHub
- URL: https://github.com/zncdatadev/spark-k8s-operator
- Owner: zncdatadev
- License: apache-2.0
- Created: 2023-08-09T05:41:44.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-05-11T13:26:08.000Z (2 months ago)
- Last Synced: 2025-05-11T14:29:18.072Z (2 months ago)
- Topics: k8s, kubernetes, spark
- Language: Go
- Homepage:
- Size: 845 KB
- Stars: 1
- Watchers: 1
- Forks: 5
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Kubedoop Operator for Apache Spark
[](https://github.com/zncdatadev/spark-k8s-operator/actions/workflows/publish.yml)
[](https://opensource.org/licenses/Apache-2.0)
[](https://goreportcard.com/report/github.com/zncdatadev/spark-k8s-operator)
[](https://artifacthub.io/packages/helm/kubedoop/spark-k8s-operator)
This is a Kubernetes operator to manage Apache Spark jobs.
It's part of the kubedoop Data Platform, a modular open source data platform built on Kubernetes that provides Kubernetes native deployment
and management of popular open source data apps like Apache Kafka, Apache Doris, Apache Kyuubi, Trino or Apache Spark, all working
together seamlessly. Based on Kubernetes, it runs everywhere – on prem or in the cloud.
## Quick Start
### Add helm repository
> Please make sure helm version is v3.0.0+
```bash
helm repo add kubedoop https://zncdatadev.github.io/kubedoop-helm-charts/
```
### Add required dependencies
```bash
helm install commons-operator kubedoop/commons-operator
helm install listener-operator kubedoop/listener-operator
helm install secret-operator kubedoop/secret-operator
```
### Add spark-k8s-operator
```bash
helm install spark-k8s-operator kubedoop/spark-k8s-operator
```
### Deploy spark-k8s cluster
```bash
kubectl apply -f config/samples
```
## Kubedoop Data Platform Operators
These are the operators that are currently part of the Kubedoop Data Platform:
- [Kubedoop Operator for Apache Airflow](https://github.com/zncdatadev/airflow-operator)
- [Kubedoop Operator for Apache DolphinScheduler](https://github.com/zncdatadev/dolphinscheduler-operator)
- [Kubedoop Operator for Apache Doris](https://github.com/zncdatadev/doris-operator)
- [Kubedoop Operator for Apache Hadoop HDFS](https://github.com/zncdatadev/hdfs-operator)
- [Kubedoop Operator for Apache HBase](https://github.com/zncdatadev/hbase-operator)
- [Kubedoop Operator for Apache Hive](https://github.com/zncdatadev/hive-operator)
- [Kubedoop Operator for Apache Kafka](https://github.com/zncdatadev/kafka-operator)
- [Kubedoop Operator for Apache Kyuubi](https://github.com/zncdatadev/kyuubi-operator)
- [Kubedoop Operator for Apache Nifi](https://github.com/zncdatadev/nifi-operator)
- [Kubedoop Operator for Apache Spark](https://github.com/zncdatadev/spark-k8s-operator)
- [Kubedoop Operator for Apache Superset](https://github.com/zncdatadev/superset-operator)
- [Kubedoop Operator for Trino](https://github.com/zncdatadev/trino-operator)
- [Kubedoop Operator for Apache Zookeeper](https://github.com/zncdatadev/zookeeper-operator)
And our internal operators: :
- [Commons Operator](https://github.com/zncdatadev/commons-operator)
- [Listener Operator](https://github.com/zncdatadev/listener-operator)
- [Secret Operator](https://github.com/zncdatadev/secret-operator)
## Contributing
If you'd like to contribute to Kubedoop, please refer to our [Contributing Guide](https://kubedoop.dev/docs/developer-manual/collaboration) for more information.
We welcome contributions of all kinds, including but not limited to code, documentation, and use cases.
## License
Copyright 2024.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.