https://github.com/giabar/gb-spark

Apache Spark cluster in Docker - https://hub.docker.com/r/giabar/gb-spark/
https://github.com/giabar/gb-spark

apache apache-spark docker docker-compose docker-image dockerfiles spark

Last synced: 2 months ago
JSON representation

Apache Spark cluster in Docker - https://hub.docker.com/r/giabar/gb-spark/

• Host: GitHub
• URL: https://github.com/giabar/gb-spark
• Owner: giabar
• License: apache-2.0
• Created: 2018-10-31T15:16:43.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2018-11-01T19:56:21.000Z (over 7 years ago)
• Last Synced: 2025-01-20T12:32:49.708Z (over 1 year ago)
• Topics: apache, apache-spark, docker, docker-compose, docker-image, dockerfiles, spark
• Language: Shell
• Homepage:
• Size: 1.67 MB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Spark Docker images

_Docker base image: openjdk:8u181-jdk-stretch (Debian 9.5 with OpenJDK 1.8.181)_

_Apache Spark version: 2.3.2_

[![DockerPulls](https://img.shields.io/docker/pulls/giabar/gb-spark.svg)](https://registry.hub.docker.com/u/giabar/gb-spark/)

[![DockerStars](https://img.shields.io/docker/stars/giabar/gb-spark.svg)](https://registry.hub.docker.com/u/giabar/gb-spark/)

## Quick start online

You can try your Spark cluster online! Click the below button:

[![Try in PWD](https://raw.githubusercontent.com/play-with-docker/stacks/master/assets/images/button.png)](https://labs.play-with-docker.com/?stack=https://raw.githubusercontent.com/giabar/gb-spark/master/docker-compose-pwd.yml)

## Requirements

* Docker CE >= 17.06.* ( [Ubuntu installation script](https://gist.github.com/giabar/9c04cea19746c036ba5d9357eb47751a) / [CentOS installation script](https://gist.github.com/giabar/ac77abc295c0fb8ddcd646533207fe80) )

* Docker Compose >= 1.22.* ( [installation script](https://gist.github.com/giabar/f966aaecd84cbbce363214065c90ae0b) )

## Quick start locally

```

git clone https://github.com/giabar/gb-spark

cd gb-spark

docker network create spark-net

docker-compose up -d

```

### Launch a Spark application:

```

docker-compose -f docker-compose-app.yml up -d

```

### How to scale your workers:

```

docker-compose scale spark-worker=2

```

## How to build

If you want to build all the images on your machine:

```

git clone https://github.com/giabar/gb-spark

cd gb-spark

./build.sh

```