https://github.com/hienduyph/docker-spark

Spark Docker Images with Iceberg included.
https://github.com/hienduyph/docker-spark

Last synced: 4 days ago
JSON representation

Spark Docker Images with Iceberg included.

• Host: GitHub
• URL: https://github.com/hienduyph/docker-spark
• Owner: hienduyph
• Created: 2022-11-27T13:52:56.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-11-19T02:40:52.000Z (about 2 months ago)
• Last Synced: 2024-11-19T03:26:36.041Z (about 2 months ago)
• Language: Shell
• Size: 25.4 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Docker Spark

```bash

docker pull ghcr.io/hienduyph/spark:latest

```

## Useful Jars

- https://iceberg.apache.org/

**Google CLoud**

BigQuery:

- gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-CONNECTOR_VERSION.jar

- https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases

GCS:

- https://storage.googleapis.com/hadoop-lib/gcs/gcs-connector-hadoop3-latest.jar

- gs://hadoop-lib/gcs/gcs-connector-hadoop3-latest.jar