https://github.com/hatamiarash7/kubernetes-spark

Deploy Apache Spark cluster in Kubernetes
https://github.com/hatamiarash7/kubernetes-spark

apache apache-spark kubernetes spark

Last synced: 12 days ago
JSON representation

Deploy Apache Spark cluster in Kubernetes

• Host: GitHub
• URL: https://github.com/hatamiarash7/kubernetes-spark
• Owner: hatamiarash7
• License: gpl-3.0
• Created: 2020-03-14T10:58:47.000Z (almost 5 years ago)
• Default Branch: master
• Last Pushed: 2024-04-02T20:58:50.000Z (11 months ago)
• Last Synced: 2024-12-09T06:22:11.021Z (2 months ago)
• Topics: apache, apache-spark, kubernetes, spark
• Homepage:
• Size: 15.6 KB
• Stars: 0
• Watchers: 3
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Kubernetes Apache Spark

 Deploy Apache Spark in Kubernetes

## Install

Change configuration in `helm-values.yml` and :

```

helm install spark -f helm-values.yaml bitnami/spark

```

Read more about helm values [here](https://github.com/bitnami/charts/tree/master/bitnami/spark)

## WebUI

* You should enable `ingress` to access Spark from a web browser

* Set hostname in `ingres` section. Default is https://spark.local

## Scaling

In `worker` section you can set scaling policy and limits. That will enable replica autoscaling depending on CPU

```yaml

resources:

    limits: 

       cpu: 100m

       memory: 150Mi

autoscaling:

    enabled: true

    CpuTargetPercentage: 50

    replicasMax: 10

```