https://github.com/maresb/simple-airflow
A simple Docker Compose based Airflow using LocalExecutor and Docker provider
https://github.com/maresb/simple-airflow
Last synced: about 1 month ago
JSON representation
A simple Docker Compose based Airflow using LocalExecutor and Docker provider
- Host: GitHub
- URL: https://github.com/maresb/simple-airflow
- Owner: maresb
- Created: 2024-03-21T23:38:17.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2025-02-14T18:20:35.000Z (over 1 year ago)
- Last Synced: 2025-03-16T11:38:05.702Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 21.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Simple minimal Airflow setup
## Introduction
Here is a slim Airflow setup with the following conditions:
* Provision with Docker Compose.
* Replace `CeleryExecutor` by `LocalExecutor` and remove Redis/Flower.
* Enable the Docker provider.
* Provide a clean and concise Git history showing the individual alterations to the [standard configuration](https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html).
* Use socat to access `/var/run/docker.sock` to avoid permissions issues with the Airflow user.
Also, I want to be able to use Airflow in a project where it isn't installed in that project's environment, only running via Docker Compose. Towards this end, I created the [airflow-stubs](https://github.com/maresb/airflow-stubs) package.
## Running Airflow
To start Airflow,
```bash
git clone https://github.com/maresb/simple-airflow.git
cd simple-airflow
mkdir -p ./dags ./logs ./plugins ./config
echo -e "AIRFLOW_UID=$(id -u)" > .env
docker compose up airflow-init
docker compose up
```
Open with the credentials `airflow`/`airflow`.
For more details, refer to the [official instructions](https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html).