https://github.com/madhums/ds101
A small boilerplate/project with docker for data science.
https://github.com/madhums/ds101
data-science docker jupyter-notebook machine-learning
Last synced: 2 months ago
JSON representation
A small boilerplate/project with docker for data science.
- Host: GitHub
- URL: https://github.com/madhums/ds101
- Owner: madhums
- Created: 2024-02-16T11:36:43.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-15T20:21:32.000Z (over 2 years ago)
- Last Synced: 2025-02-17T06:31:54.953Z (over 1 year ago)
- Topics: data-science, docker, jupyter-notebook, machine-learning
- Language: Jupyter Notebook
- Homepage:
- Size: 155 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## ds101
A small boilerplate/project with docker for data science.
### Getting started
```sh
cp config/.env.jupyter.example .env.jupyter
cp config/.env.minio.example .env.minio
cp config/.env.postgres.example .env.postgres
cp config/.env.airflow.example .env.airflow
cp config/.env.database.example .env.database
```
Provide appropriate values in the .env files and then run
```sh
docker-compose up -d
```
if you want to persist jupyter settings, make sure to commit the container before taking it down. So
```sh
docker commit jupyter
# and then
docker-compose down
```
Otherwise you can simply stick to `docker-compose start` and `stop`.
**Services:**
- `jupyterlab`: Jupyter notebooks and jupyter lab where you can do fancy stuff ([localhost:8888](http://localhost:8888))
- `minio`: A key value file store like aws ([localhost:9001](http://localhost:9001))
- `postgres`: Database store
- `metabase`: Cool data science stuff
- `superset`: Another cool visualisations service
- `airflow`: Scheduler and task runner ([localhost:8080](http://localhost:8080))
**Structure:**
- `config`: Contains environment variables, keys, secrets etc
- `jupyter`: Contains notebooks
- `dags`: Task runners
Data folders:
- `data`: Contains data which is mounted on to minio
- `db_data`: Postgres database persistant volumne
- `metabase_data`: Metabase data persistant volume
---
Inspired by [data-science-stack](https://github.com/jgoerner/data-science-stack-cookiecutter)