{"id":17343753,"url":"https://github.com/zbrookle/avionix_airflow","last_synced_at":"2025-10-31T09:38:14.697Z","repository":{"id":54571323,"uuid":"282072851","full_name":"zbrookle/avionix_airflow","owner":"zbrookle","description":"Apache Airflow Hosted on a Kubernetes cluster, ready out of the box, with monitoring stack included (Grafana, ElasticSearch, Filebeat)","archived":false,"fork":false,"pushed_at":"2021-03-23T21:07:04.000Z","size":400,"stargazers_count":7,"open_issues_count":4,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-14T20:13:08.685Z","etag":null,"topics":["airflow","apache-airflow","avionix","aws","aws-eks","chart-builder","cluster","elasticsearch","filebeat","grafana","helm","helm-chart","kubernetes","kubernetes-executor"],"latest_commit_sha":null,"homepage":"https://avionix-airflow.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zbrookle.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-23T22:51:19.000Z","updated_at":"2023-11-13T16:02:39.000Z","dependencies_parsed_at":"2022-08-13T20:10:14.565Z","dependency_job_id":null,"html_url":"https://github.com/zbrookle/avionix_airflow","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zbrookle%2Favionix_airflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zbrookle%2Favionix_airflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zbrookle%2Favionix_airflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zbrookle%2Favionix_airflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zbrookle","download_url":"https://codeload.github.com/zbrookle/avionix_airflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248952352,"owners_count":21188426,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airflow","apache-airflow","avionix","aws","aws-eks","chart-builder","cluster","elasticsearch","filebeat","grafana","helm","helm-chart","kubernetes","kubernetes-executor"],"created_at":"2024-10-15T16:10:16.078Z","updated_at":"2025-10-31T09:38:14.594Z","avatar_url":"https://github.com/zbrookle.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Avionix Airflow\n===============\n\n.. image:: https://github.com/zbrookle/avionix_airflow/workflows/CI/badge.svg?branch=master\n    :target: https://github.com/zbrookle/avionix_airflow/actions?query=workflow\n\nWhy are there so many guides on how to set up `Apache Airflow`_ when the code could\njust be given to you?!\n\n.. _Apache Airflow: https://airflow.apache.org\n \nWelcome to ``Avionix Airflow``, an out of the box solution to installing airflow on\nKubernetes_.\n\n.. _Kubernetes: https://kubernetes.io\n \nCurrently supported platforms are Minikube_ and `AWS EKS`_ (managed nodegroups only),\npull requests adding support for GKE and AKS are welcome.\n\n.. _Minikube: https://minikube.sigs.k8s.io/docs/\n.. _AWS EKS: https://aws.amazon.com/eks/\n \nOverview of Avionix Airflow Capabilities\n----------------------------------------\n\n``Avionix airflow`` provides the following out of the box solutions for airflow\n\n1. Airflow Webserver and Scheduler with configured RBAC for the KubernetesExecutor\n2. Airflow metric collection (statsd -\u003e telegraf -\u003e elasticsearch)\n3. Airflow log collection (stdout -\u003e filebeat -\u003e elasticsearch)\n4. Metric and log visualization (elasticsearch \u0026 postgres -\u003e grafana)\n\nInstallation\n------------\n\n.. code-block::\n\n    pip install avionix_airflow\n\n\nRequirements\n------------\n\n- kubectl_\n- helm_\n\n.. _helm: https://helm.sh\n.. _kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/\n\nConfiguration\n-------------\n\n``Avionix airflow`` is configured by using Options objects, there are currently 4\ndifferent objects:\n \n1. AirflowOptions\n2. PostgresOptions\n3. MonitoringOptions\n4. CloudOptions (Specified using LocalOptions or AwsOptions)\n5. RedisOptions (Only for use with CeleryExecutor, ***not recommended***)\n\nThese options are then passed into the function *get_chart_builder*, which can be\nused to retrieve an avionix_ chart builder object. For instruction on how to use the\nbuilder object, see avionix_\n\n.. _avionix: https://github.com/zbrookle/avionix\n\nFAQ\n---\n\n- How do I change the grafana dashboard?\n    - You can change the dashboard by setting the MonitoringOptions role to \"Admin\"\n    \nDocumentation\n-------------\n\nAdditional documentation can be found `here \u003chttps://avionix-airflow.readthedocs.io/en/latest/\u003e`__","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzbrookle%2Favionix_airflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzbrookle%2Favionix_airflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzbrookle%2Favionix_airflow/lists"}