Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/feluelle/airflow-diagrams

Auto-generated Diagrams from Airflow DAGs. 🔮 🪄
https://github.com/feluelle/airflow-diagrams

airflow auto-generated cli diagrams python

Last synced: 6 days ago
JSON representation

Auto-generated Diagrams from Airflow DAGs. 🔮 🪄

Host: GitHub
URL: https://github.com/feluelle/airflow-diagrams
Owner: feluelle
License: apache-2.0
Created: 2020-06-29T11:13:10.000Z (over 4 years ago)
Default Branch: main
Last Pushed: 2025-01-08T20:36:57.000Z (15 days ago)
Last Synced: 2025-01-11T04:07:33.690Z (13 days ago)
Topics: airflow, auto-generated, cli, diagrams, python
Language: Python
Homepage:
Size: 1.23 MB
Stars: 330
Watchers: 5
Forks: 29
Open Issues: 6
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

        # airflow-diagrams

[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/feluelle/airflow-diagrams/main.svg)](https://results.pre-commit.ci/latest/github/feluelle/airflow-diagrams/main)

![test workflow](https://github.com/feluelle/airflow-diagrams/actions/workflows/test.yml/badge.svg)

![codeql-analysis workflow](https://github.com/feluelle/airflow-diagrams/actions/workflows/codeql-analysis.yml/badge.svg)

[![codecov](https://codecov.io/gh/feluelle/airflow-diagrams/branch/main/graph/badge.svg?token=J8UEP8IVY4)](https://codecov.io/gh/feluelle/airflow-diagrams)

[![PyPI version](https://img.shields.io/pypi/v/airflow-diagrams)](https://pypi.org/project/airflow-diagrams/)

[![License](https://img.shields.io/pypi/l/airflow-diagrams)](https://github.com/feluelle/airflow-diagrams/blob/main/LICENSE)

[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/airflow-diagrams)](https://pypi.org/project/airflow-diagrams/)

[![PyPI version](https://img.shields.io/pypi/dm/airflow-diagrams)](https://pypi.org/project/airflow-diagrams/)

> Auto-generated Diagrams from Airflow DAGs. 🔮 🪄

This project aims to easily visualise your [Airflow](https://github.com/apache/airflow) DAGs on service level

from providers like AWS, GCP, Azure, etc. via [diagrams](https://github.com/mingrammer/diagrams).

![demo](assets/images/demo.svg)

Before | After

--- | ---

![dag](assets/images/dbt_dag.png) | ![diagram](assets/images/dbt_diagram.png)

## 🚀 Get started

To install it from [PyPI](https://pypi.org/) run:

```console

pip install airflow-diagrams

```

> **_NOTE:_** Make sure you have [Graphviz](https://www.graphviz.org/) installed.

Then just call it like this:

![usage](assets/images/usage.png)

_Examples of generated diagrams can be found in the [examples](examples) directory._

## 🤔 How it Works

1. ℹ️ It connects, by using the official [Apache Airflow Python Client](https://github.com/apache/airflow-client-python), to your Airflow installation to retrieve all DAGs (in case you don't specify any `dag_id`) and all Tasks for the DAG(s).

1. 🪄 It processes every DAG and its Tasks and 🔮 tries to find a diagram node for every DAGs task, by using [Fuzzy String Matching](https://github.com/seatgeek/thefuzz), that matches the most. If you are unhappy about the match you can also provide a `mapping.yml` file to statically map from Airflow task to diagram node.

1. 🎨 It renders the results into a python file which can then be executed to retrieve the rendered diagram. 🎉

## ❤️ Contributing

Contributions are very welcome. Please go ahead and raise an issue if you have one or open a PR. Thank you.