https://github.com/angadsingh/airflow-ditto

An airflow DAG transformation framework
https://github.com/angadsingh/airflow-ditto

airflow airflow-dag aws azure dataflow emr extensible framework graph-algorithms graph-manipulation hdinsight isomorphism livy networkx spark yarn

Last synced: about 1 month ago
JSON representation

An airflow DAG transformation framework

• Host: GitHub
• URL: https://github.com/angadsingh/airflow-ditto
• Owner: angadsingh
• License: apache-2.0
• Created: 2020-06-30T06:30:35.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2020-07-10T11:45:01.000Z (over 4 years ago)
• Last Synced: 2024-10-06T13:16:06.509Z (2 months ago)
• Topics: airflow, airflow-dag, aws, azure, dataflow, emr, extensible, framework, graph-algorithms, graph-manipulation, hdinsight, isomorphism, livy, networkx, spark, yarn
• Language: Python
• Homepage: https://angadsingh.github.io/airflow-ditto/
• Size: 6.51 MB
• Stars: 5
• Watchers: 4
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.assets/check_cluster_emr_dag.png
  • License: LICENSE

Awesome Lists containing this project

• awesome-apache-airflow - Airflow Ditto - An extensible framework to do transformations to an Airflow DAG and convert it into another DAG which is flow-isomorphic with the original DAG, to be able to run it on different environments (e.g. on different clouds, or even different container frameworks - Apache Spark on YARN vs Kubernetes). Comes with out-of-the-box support for EMR-to-HDInsight-DAG transforms. (Libraries, Hooks, Utilities)