https://github.com/googlecloudplatform/spark-on-k8s-gcp-examples

Example Spark applications that run on Kubernetes and access GCP products, e.g., GCS, BigQuery, and Cloud PubSub
https://github.com/googlecloudplatform/spark-on-k8s-gcp-examples

bigquery cloud-pubsub gcs gcs-connector kubernetes spark

Last synced: about 4 hours ago
JSON representation

Example Spark applications that run on Kubernetes and access GCP products, e.g., GCS, BigQuery, and Cloud PubSub

• Host: GitHub
• URL: https://github.com/googlecloudplatform/spark-on-k8s-gcp-examples
• Owner: GoogleCloudPlatform
• License: apache-2.0
• Archived: true
• Created: 2017-12-13T22:53:38.000Z (about 7 years ago)
• Default Branch: master
• Last Pushed: 2018-02-13T21:50:42.000Z (almost 7 years ago)
• Last Synced: 2024-12-18T08:41:25.213Z (about 1 month ago)
• Topics: bigquery, cloud-pubsub, gcs, gcs-connector, kubernetes, spark
• Language: Java
• Homepage:
• Size: 26.4 KB
• Stars: 36
• Watchers: 6
• Forks: 27
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

README

        
This is not an official Google product.

# Accessing GCP Products in Spark

This repository is for examples showing how to access Google Cloud Platform products such as [Google Cloud Storage](https://cloud.google.com/storage/) (GCS), [BigQuery](https://cloud.google.com/bigquery/), and [Cloud PubSub](https://cloud.google.com/pubsub/) as data sources or sinks in [Spark on Kubernetes](https://github.com/apache-spark-on-k8s/spark) applications. Please refer to the `README` in each example directory for details on how to build and run the example.