Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/plandes/docker-spark-service
Extends the docker-spark, which creates a Spark cluster in a docker image by making it a service.
https://github.com/plandes/docker-spark-service
docker docker-spark dockerfile spark
Last synced: about 1 month ago
JSON representation
Extends the docker-spark, which creates a Spark cluster in a docker image by making it a service.
- Host: GitHub
- URL: https://github.com/plandes/docker-spark-service
- Owner: plandes
- License: apache-2.0
- Created: 2016-12-22T20:21:44.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2017-03-05T01:05:43.000Z (almost 8 years ago)
- Last Synced: 2024-11-08T21:27:27.735Z (3 months ago)
- Topics: docker, docker-spark, dockerfile, spark
- Language: Makefile
- Size: 5.86 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Spark Cluster as a Service in Docker
This extends the [docker-spark](https://github.com/P7h/docker-spark) repo,
which creates a [Spark](http://spark.apache.org) cluster in a
[docker](http://www.docker.com) image by making it a service (avoid having to
run the start commands) and/or use it in a docker compose configuration.
Note that the version of the image is defined in
[makefile](https://github.com/plandes/docker-spark-service/blob/master/makefile#L1)
and coresponds to the `docker-spark` docker image and Spark version.
## Obtaining the Image
The image is available on
[docker hub](https://hub.docker.com/r/plandes/docker-spark-service/) or by
pulling it with:
```bash
$ docker pull plandes/docker-spark-service
```
## Docker Compose
The following is an example docker compose file:
```yaml
spark-service:
container_name: spark
image: plandes/docker-spark-service:2.1.0
hostname: spark
ports:
- "8080:8080"
- "8081:8081"
- "4040:4040"
volumes:
- /Volumns/spark-mount:/root
links:
- nlpserv
```
To use this, create the mount point and start docker:
```bash
$ mkdir -p /Volumns/spark-mount
$ docker-compose up -d
```
## License
Copyright © 2016, 2017 Paul Landes
Apache License version 2.0
Licensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License. You may obtain a copy of the
License at
[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)
Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.