https://github.com/ntheanh201/spark-gitops

https://github.com/ntheanh201/spark-gitops

Last synced: 24 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/ntheanh201/spark-gitops
• Owner: ntheanh201
• Created: 2023-08-16T06:37:38.000Z (about 1 year ago)
• Default Branch: master
• Last Pushed: 2023-08-17T10:40:10.000Z (about 1 year ago)
• Last Synced: 2024-04-15T07:44:35.091Z (7 months ago)
• Language: Scala
• Size: 48.8 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# spark-gitops

## Run manually

```shell

./bin/spark-submit \

  --master k8s://https://spark-server:spark-api-server-port \

  --deploy-mode cluster \

  --name spark-pi \

  --conf spark.kubernetes.driverEnv.SPARK_MASTER_URL=spark://spark-master-svc:spark-master-port \

  --conf spark.kubernetes.container.image=bitnami/spark:3 \

  --conf spark.kubernetes.file.upload.path='local:///spark-gitops' \

  --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \

  --conf spark.kubernetes.namespace=spark \

  --driver-memory 1g \

  --executor-memory 1g \

  --executor-cores 1 \

  --class com.ntheanh201.uw.Main \

  ./target/spark-poc-0.0.1.jar

```

## GitOps