Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/aartintelligent/docker-spark
Docker Spark Base
https://github.com/aartintelligent/docker-spark
docker spark
Last synced: 25 days ago
JSON representation
Docker Spark Base
- Host: GitHub
- URL: https://github.com/aartintelligent/docker-spark
- Owner: aartintelligent
- Created: 2024-12-01T17:44:02.000Z (26 days ago)
- Default Branch: master
- Last Pushed: 2024-12-01T18:18:41.000Z (26 days ago)
- Last Synced: 2024-12-01T18:33:35.846Z (26 days ago)
- Topics: docker, spark
- Language: Dockerfile
- Homepage:
- Size: 0 Bytes
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Docker Spark
=============
### Usage Docker
```shell
docker build . -t aartintelligent/spark:latest
```
```shell
docker push aartintelligent/spark:latest
```
### Usage Docker Compose
```shell
docker compose build
```
```shell
docker compose up -d
```
```shell
docker compose run -it spark bin/spark-submit app/example.py
```
```shell
docker compose down -v
```
### Compose
```yaml
services:
database:
image: postgres:17
user: root
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=spark
- POSTGRES_PASSWORD=password
- PGDATA=/var/lib/postgresql/data/pgdata
volumes:
- database-volume:/var/lib/postgresql/data
# - /opt/postgres/data:/var/lib/postgresql/data
ports:
- '5432:5432'
spark:
build:
context: .
command: /opt/spark/sbin/start-master.sh
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# capabilities: [ gpu ]
volumes:
- ./app:/opt/spark/app
ports:
- '7077:7077'
- '8080:8080'
volumes:
database-volume:
```