https://github.com/navicore/mysparksql

https://github.com/navicore/mysparksql

apache-cassandra apache-spark cassandra kubernetes spark

Last synced: 20 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/navicore/mysparksql
• Owner: navicore
• License: apache-2.0
• Created: 2017-02-11T15:53:12.000Z (almost 8 years ago)
• Default Branch: master
• Last Pushed: 2022-11-25T23:52:18.000Z (about 2 years ago)
• Last Synced: 2024-11-19T22:49:53.508Z (3 months ago)
• Topics: apache-cassandra, apache-spark, cassandra, kubernetes, spark
• Language: Scala
• Size: 15.6 KB
• Stars: 0
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# mysparksql

a minimal code example of reading from cassandra via spark 2.x

client mode:

```console

/opt/spark/bin/spark-submit --master spark://:7077 --class onextent.demo.mysparksql.Main --driver-java-options="-DCASSANDRA_HOST= -DKEYSPACE= -DTABLE=" --deploy-mode client ./mysparksql.jar

```

via docker:

```console

docker run -e SPARK_MASTER=spark:// -e CASSANDRA_HOST= -e KEYSPACE= -e TABLE= -it navicore/mysparksql

```

via kubernetes job:

```yaml

apiVersion: batch/v1

kind: Job

metadata:

  name: mysparksql

spec:

  template:

    metadata:

      name: mysparksql

    spec:

      containers:

      - name: mysparksql

        image: navicore/mysparksql

        env: 

        - name: CASSANDRA_HOST

          value: ""

        - name: SPARK_MASTER

          value: "spark://"

        - name: KEYSPACE

          value: ""

        - name: TABLE

          value: ""

      restartPolicy: Never

```