{"id":19976688,"url":"https://github.com/nccloud/airflow-chart","last_synced_at":"2026-02-11T06:03:45.143Z","repository":{"id":48198097,"uuid":"393003118","full_name":"NCCloud/airflow-chart","owner":"NCCloud","description":"Airflow modified chart","archived":false,"fork":false,"pushed_at":"2021-08-05T12:05:27.000Z","size":44,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-25T07:25:39.583Z","etag":null,"topics":["devops","etl"],"latest_commit_sha":null,"homepage":"","language":"Mustache","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NCCloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-05T10:39:43.000Z","updated_at":"2021-08-05T12:05:29.000Z","dependencies_parsed_at":"2022-09-13T19:01:36.885Z","dependency_job_id":null,"html_url":"https://github.com/NCCloud/airflow-chart","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NCCloud/airflow-chart","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCCloud%2Fairflow-chart","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCCloud%2Fairflow-chart/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCCloud%2Fairflow-chart/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCCloud%2Fairflow-chart/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NCCloud","download_url":"https://codeload.github.com/NCCloud/airflow-chart/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCCloud%2Fairflow-chart/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29327881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-11T03:52:29.695Z","status":"ssl_error","status_checked_at":"2026-02-11T03:52:23.094Z","response_time":97,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["devops","etl"],"created_at":"2024-11-13T03:25:09.827Z","updated_at":"2026-02-11T06:03:45.131Z","avatar_url":"https://github.com/NCCloud.png","language":"Mustache","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Airflow Helm Chart\n\n[Airflow](https://airflow.apache.org/) is a platform to programmatically author, schedule and monitor workflows.\n\n## Installation\n\nTo install the Airflow Helm Chart:\n```bash\nhelm install stable/airflow \\\n  --version \"X.X.X\" \\\n  --name \"airflow\" \\\n  --namespace \"airflow\" \\\n  --values ./custom-values.yaml\n```\n\nTo get the status of the Airflow Helm Chart:\n```bash\nhelm status \"airflow\"\n```\n\nTo uninstall the Airflow Helm Chart:\n```bash\nhelm delete \"airflow\"\n```\n\nTo run bash commands in the Airflow Scheduler Pod:\n```bash\n# use this to run commands like: `airflow create_user`\nkubectl exec \\\n  -it \\\n  --namespace airflow \\\n  --container airflow-scheduler \\\n  Deployment/airflow-scheduler \\\n  /bin/bash\n```\n\n### Upgrade Steps:\n\n\u003e NOTE: for chart version numbers, see [Chart.yaml](Chart.yaml) or [helm hub](https://hub.helm.sh/charts/stable/airflow).\n\nFor steps you must take when upgrading this chart, please review:\n* [v7.2.X → v7.3.0](UPGRADE.md#v72x--v730)\n* [v7.1.X → v7.2.0](UPGRADE.md#v71x--v720)\n* [v7.0.X → v7.1.0](UPGRADE.md#v70x--v710)\n* [v6.X.X → v7.0.0](UPGRADE.md#v6xx--v700)\n* [v5.X.X → v6.0.0](UPGRADE.md#v5xx--v600)\n* [v4.X.X → v5.0.0](UPGRADE.md#v4xx--v500)\n* [v3.X.X → v4.0.0](UPGRADE.md#v3xx--v400)\n\n### Examples:\n\nThere are many ways to deploy this chart, but here are some starting points for your `custom-values.yaml`:\n\n| Name | File | Description |\n| --- | --- | --- |\n| (CeleryExecutor) Minimal | [examples/minikube/custom-values.yaml](examples/minikube/custom-values.yaml) | a __non-production__ starting point |\n| (CeleryExecutor) Google Cloud | [examples/google-gke/custom-values.yaml](examples/google-gke/custom-values.yaml) | a __production__ starting point for GKE on Google Cloud |\n\n## Airflow-Configs\n\n### Airflow-Configs/General\n\nWhile we don't expose the `airflow.cfg` directly, you can use [environment variables](https://airflow.apache.org/docs/stable/howto/set-config.html) to set Airflow configs.\n\nWe expose the `airflow.config` value to make this easier:\n```yaml\nairflow:\n  config:\n    ## Security\n    AIRFLOW__CORE__SECURE_MODE: \"True\"\n    AIRFLOW__API__AUTH_BACKEND: \"airflow.api.auth.backend.deny_all\"\n    AIRFLOW__WEBSERVER__EXPOSE_CONFIG: \"False\"\n    AIRFLOW__WEBSERVER__RBAC: \"False\"\n\n    ## DAGS\n    AIRFLOW__SCHEDULER__DAG_DIR_LIST_INTERVAL: \"30\"\n    AIRFLOW__CORE__LOAD_EXAMPLES: \"False\"\n\n    ## Email (SMTP)\n    AIRFLOW__EMAIL__EMAIL_BACKEND: \"airflow.utils.email.send_email_smtp\"\n    AIRFLOW__SMTP__SMTP_HOST: \"smtpmail.example.com\"\n    AIRFLOW__SMTP__SMTP_STARTTLS: \"False\"\n    AIRFLOW__SMTP__SMTP_SSL: \"False\"\n    AIRFLOW__SMTP__SMTP_PORT: \"25\"\n    AIRFLOW__SMTP__SMTP_MAIL_FROM: \"admin@example.com\"\n\n    ## Disable noisy \"Handling signal: ttou\" Gunicorn log messages\n    GUNICORN_CMD_ARGS: \"--log-level WARNING\"\n```\n\n### Airflow-Configs/Connections\n\nWe expose the `scheduler.connections` value to allow specifying [Airflow Connections](https://airflow.apache.org/docs/stable/concepts.html#connections) at deployment time, these connections will be automatically imported by the Airflow scheduler when it starts up.\n\nFor example, to add a connection called `my_aws`:\n```yaml\nscheduler:\n  connections:\n    - id: my_aws\n      type: aws\n      extra: |\n        {\n          \"aws_access_key_id\": \"XXXXXXXXXXXXXXXXXXX\",\n          \"aws_secret_access_key\": \"XXXXXXXXXXXXXXX\",\n          \"region_name\":\"eu-central-1\"\n        }\n```\n\n__NOTE:__ As connections may include sensitive data, we store the bash script which generates the connections in a Kubernetes Secret, and mount this to the pods.\n\n__WARNING:__ Because some values are sensitive, you should take care to store your custom `values.yaml` securely before passing it to helm with: `helm -f \u003cmy-secret-values.yaml\u003e`\n\n### Airflow-Configs/Variables\n\nWe expose the `scheduler.variables` value to allow specifying [Airflow Variables](https://airflow.apache.org/docs/stable/concepts.html#variables) at deployment time, variables will be automatically imported by the Airflow scheduler when it starts up.\n\nFor example, to specify a variable called `environment`:\n```yaml\nscheduler:\n  variables: |\n    { \"environment\": \"dev\" }\n```\n\n### Airflow-Configs/Pools\n\nWe expose the `scheduler.pools` value to allow specifying [Airflow Pools](https://airflow.apache.org/docs/stable/concepts.html#pools) at deployment time, these pools will be automatically imported by the Airflow scheduler when it starts up.\n\nFor example, to create a pool called `example`:\n```yaml\nscheduler:\n  pools: |\n    {\n      \"example\": {\n        \"description\": \"This is an example pool with 2 slots.\",\n        \"slots\": 2\n      }\n    }\n```\n\n### Airflow-Configs/Secret-Environment-Variables\n\nIt is possible to specify additional environment variables using the same format as in a pod's `.spec.containers.env` definition.\nThese environment variables will be mounted in the web, scheduler, and worker pods.\nYou can use this feature to pass additional secret environment variables to Airflow.\n\nHere is a simple example showing how to pass in a Fernet key and LDAP password:\n```yaml\nextraEnv:\n  - name: AIRFLOW__CORE__FERNET_KEY\n    valueFrom:\n      secretKeyRef:\n        name: airflow\n        key: fernet-key\n  - name: AIRFLOW__LDAP__BIND_PASSWORD\n    valueFrom:\n      secretKeyRef:\n        name: ldap\n        key: password\n```\n\n__NOTE:__ For this example to work, both the `airflow` and `ldap` Kubernetes secrets must already exist in the proper namespace.\n\n## Kubernetes-Configs\n\nIn this chart we expose many Kubernetes-specific configs not usually found in Airflow.\n\n### Kubernetes-Configs/Ingress\n\n#### Overview:\n\nThis chart provides an optional Ingress resource, which can be enabled and configured by passing a custom `values.yaml` to helm.\n\nThis chart exposes 2 endpoints on the Ingress:\n* Airflow WebUI\n* Flower (A debug UI for Celery)\n\n#### Custom Paths:\n\nThis chart enables you to add various paths to the ingress. \nIt includes two values for you to customize these paths, `precedingPaths` and `succeedingPaths`, which are before and after the default path to `Service/airflow-web`.\nA common use case is enabling https with the `aws-alb-ingress-controller` [ssl-redirect](https://kubernetes-sigs.github.io/aws-alb-ingress-controller/guide/tasks/ssl_redirect/), which needs a redirect path to be hit before the airflow-web one. \n\nYou would set the values of `precedingPaths` as the following:\n```yaml\nprecedingPaths:\n  - path: \"/*\"\n    serviceName: \"ssl-redirect\"\n    servicePort: \"use-annotation\"\n```\n\n#### URL Prefix:\n\nIf you already have something hosted at the root of your domain, you might want to place airflow/flower under a url-prefix.\nFor example, with a url-prefix of `/airflow/`:\n* http://example.com/airflow/\n* http://example.com/airflow/flower\n\n\nWhen customizing this, please note:\n- Airflow WebUI behaves transparently, to configure it one just needs to specify the `web.baseUrl` value. \n- Flower requires a URL rewrite mechanism in front of it.\n  For specifics on this, see the comments of `flower.urlPrefix` inside `values.yaml`.\n\n### Kubernetes-Configs/Worker-Autoscaling\n\nWe use a Kubernetes StatefulSet for the Celery workers, this allows the webserver to requests logs from each workers individually, with a fixed DNS name.\n\nCelery workers can be scaled using the [Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/).\nTo enable autoscaling, you must set `workers.autoscaling.enabled=true`, then provide `workers.autoscaling.maxReplicas`, and `workers.replicas` for the minimum amount.\nMake sure to set a resource request in `workers.resources`, and `dags.git.gitSync.resources`, otherwise worker pods will not scale.\n(For git-sync, `64Mi` should be enough.)\n\nAssume every task a worker executes consumes approximately `200Mi` memory, that means memory is a good metric for utilisation monitoring.\nFor a worker pod you can calculate it: `WORKER_CONCURRENCY * 200Mi`, so for `10 tasks` a worker will consume `~2Gi` of memory.\n\nHere is the `values.yaml` config for that example:\n```yaml\nworkers:\n  replicas: 1\n\n  resources:\n    requests:\n      memory: \"2Gi\"\n\n  autoscaling:\n    enabled: true\n    maxReplicas: 16\n    metrics:\n    - type: Resource\n      resource:\n        name: memory\n        target:\n          type: Utilization\n          averageUtilization: 80\n\n  celery:\n    instances: 10\n\n    # wait until all tasks are finished before SIGTERM of Pod\n    gracefullTermination: true\n\n  # wait AT MOST 10min for tasks on a worker to finish before SIGKILL\n  terminationPeriod: 600\n\ndags:\n  git:\n    gitSync:\n      resources:\n        requests:\n          memory: \"64Mi\"\n```\n\n__NOTE:__ With this config if a worker consumes `80%` of `2Gi` (which will happen if it runs 9-10 tasks at the same time), an autoscale event will be triggered, and a new worker will be added.\nIf you have many tasks in a queue, Kubernetes will keep adding workers until maxReplicas reached, in this case `16`.\n\n### Kubernetes-Configs/Worker-Secrets\n\nYou can add Kubernetes Secrets which will be mounted as volumes on the worker nodes at `{workers.secretsDir}/\u003csecret-name\u003e`:\n```yaml\nworkers:\n  secretsDir: /var/airflow/secrets\n  secrets:\n    - redshift-user\n    - redshift-password\n    - elasticsearch-user\n    - elasticsearch-password\n```\n\nWith the above configuration, you could read the `redshift-user` password from within a dag or other function using:\n```python\nimport os\nfrom pathlib import Path\n\ndef get_secret(secret_name):\n    secrets_dir = Path('/var/airflow/secrets')\n    secret_path = secrets_dir / secret_name\n    assert secret_path.exists(), f'could not find {secret_name} at {secret_path}'\n    secret_data = secret_path.read_text().strip()\n    return secret_data\n\nredshift_user = get_secret('redshift-user')\n```\n\nTo create a secret, you can use:\n```bash\n# where `~/secrets/redshift-user.txt` contains the user secret as a single text string\nkubectl create secret generic redshift-user --from-file=redshift-user=~/secrets/redshift-user.txt\n```\n\n### Kubernetes-Configs/Extra-Python-Packages\n\nSometimes you may need to install extra pip packages for things to work, we provide `airflow.extraPipPackages` and `web.extraPipPackages` for this purpose.\n\nFor example, enabling the airflow `airflow-exporter` package:\n```yaml\nairflow:\n  extraPipPackages:\n    - \"airflow-exporter==1.3.1\"\n```\n\nFor example, you may be using `flask_oauthlib` to integrate with Okta/Google/etc for authorizing WebUI users:\n```yaml\nweb:\n  extraPipPackages:\n    - \"apache-airflow[google_auth]==1.10.10\"\n```\n\n__NOTE:__ these work with any pip package that you can install with the `pip install XXXX` cli\n\n### Kubernetes-Configs/Additional-Manifests\n\nIt is possible to add additional manifests into a deployment, to extend the chart. \nOne of the reason is to deploy a manifest specific to a cloud provider.\n\nFor example, adding a `BackendConfig` on GKE:\n```yaml\nextraManifests:\n- apiVersion: cloud.google.com/v1beta1\n  kind: BackendConfig\n  metadata:\n    name: \"{{ .Release.Name }}-test\"\n  spec:\n    securityPolicy:\n      name: \"gcp-cloud-armor-policy-test\"\n```\n\n## Database-Configs\n\n### Database-Configs/Initialization\n\nIf the value `scheduler.initdb` is set to `true`, the airflow-scheduler container will run `airflow initdb` before starting the scheduler as part of its startup script.\n\nIf the value `scheduler.preinitdb` is set to `true`, the airflow-scheduler pod will run `airflow initdb` as an initContainer, before the git-clone initContainer (if that is enabled).  \nThis is rarely necessary but can be so under certain conditions if your synced DAGs include custom database hooks that prevent `initdb` from running successfully.\nFor example, if they have dependencies on variables that won't be present yet.\nThe initdb initcontainer will retry up to 5 times before giving up.\n\n\n### Database-Configs/Passwords\n\nPostgres is used as the default database backing Airflow in this chart, we use insecure username/password combinations by default.\n\nFor a real production deployment, it's a good idea to create secure credentials before installing the Helm chart.\nFor example, from the command line, run:\n```bash\nkubectl create secret generic airflow-postgresql --from-literal=postgresql-password=$(openssl rand -base64 13)\nkubectl create secret generic airflow-redis --from-literal=redis-password=$(openssl rand -base64 13)\n```\n\nNext, you can use those secrets with your `values.yaml`:\n```yaml\npostgresql:\n  existingSecret: airflow-postgresql\n\nredis:\n  existingSecret: airflow-redis\n```\n\n### Database-Configs/External-Database\n\nWhile this chart comes with an embedded [stable/postgresql](https://github.com/helm/charts/tree/master/stable/postgresql), this is NOT SUITABLE for production.\nYou should make use of an external `mysql` or `postgres` database, for example, one that is managed by your cloud provider.\n\n__Postgres:__\n\nValues for an external Postgres database, with an existing `airflow_cluster1` database:\n```yaml\nexternalDatabase:\n  type: postgres\n  host: postgres.example.org\n  port: 5432\n  database: airflow_cluster1\n  user: airflow_cluster1\n  passwordSecret: \"airflow-cluster1-postgres-password\"\n  passwordSecretKey: \"postgresql-password\"\n```\n\n__MySQL:__\n\nValues for an external MySQL database, with an existing `airflow_cluster1` database:\n```yaml\nexternalDatabase:\n  type: mysql\n  host: mysql.example.org\n  port: 3306\n  database: airflow_cluster1\n  user: airflow_cluster1\n  passwordSecret: \"airflow-cluster1-mysql-password\"\n  passwordSecretKey: \"mysql-password\"\n```\n\n__WARNING:__ Airflow requires that `explicit_defaults_for_timestamp=1` in your MySQL instance, [see here](https://airflow.apache.org/docs/stable/howto/initialize-database.html)\n\n## Other-Configs\n\n### Other-Configs/Local-Binaries\n\nPlease note a folder `~/.local/bin` will be automatically created and added to the PATH so that bash operators can use command line tools installed by `pip install --user`.\n\n### Other-Configs/Logging\n\nBy default logs from the web server, scheduler, and Celery workers are written within the Docker container's filesystem, therefore any restart of the pod will wipe the logs.\nFor production purposes, you will likely want to persist the logs externally (e.g. S3), which you have to set up yourself through configuration.\n\nFor example, to use a GCS Bucket:\n```yaml\nairflow:\n  config:\n    AIRFLOW__CORE__REMOTE_LOGGING: \"True\"\n    AIRFLOW__CORE__REMOTE_BASE_LOG_FOLDER: \"gs://\u003c\u003cMY-BUCKET-NAME\u003e\u003e/airflow/logs\"\n    AIRFLOW__CORE__REMOTE_LOG_CONN_ID: \"google_cloud_airflow\"\n```\n\n__NOTE:__ a connection called `google_cloud_airflow` must exist in airflow, which could be created using the `scheduler.connections` value.\n\nFor example, to use a persistent volume:\n```yaml\nlogs:\n  persistence:\n    enabled: true\n```\n\n__NOTE:__ it is also possible to persist logs by mounting a `PersistentVolume` to the log directory (`/opt/airflow/logs` by default) using `airflow.extraVolumes` and `airflow.extraVolumeMounts`.\n\n### Other/Service-Monitor\n\nThe service monitor is something introduced by the [CoresOS Prometheus Operator](https://github.com/coreos/prometheus-operator).\nTo be able to expose metrics to prometheus you need install a plugin, this can be added to the docker image. \nA good one is [epoch8/airflow-exporter](https://github.com/epoch8/airflow-exporter), which exposes dag and task based metrics from Airflow.\nFor more information: see the `serviceMonitor` section of `values.yaml`.\n\n## DAGs\n\n### DAGs/Storage\n\nThere are several options for options synchronizing your Airflow DAGs between pods\n\n#### Option 1 - Git-Sync Sidecar (Recommended)\n\nThis method places a git sidecar in each worker/scheduler/web Kubernetes Pod, that perpetually syncs your git repo into the dag folder every `dags.git.gitSync.refreshTime` seconds.\n\nFor example:\n```yaml\ndags:\n  git:\n    url: ssh://git@repo.example.com/example.git\n    repoHost: repo.example.com\n    secret: airflow-git-keys\n    privateKeyName: id_rsa\n\n    gitSync:\n      enabled: true\n      refreshTime: 60\n```\n\nYou can create the `dags.git.secret` from your local `~/.ssh` folder using:\n```bash\nkubectl create secret generic airflow-git-keys \\\n  --from-file=id_rsa=~/.ssh/id_rsa \\\n  --from-file=id_rsa.pub=~/.ssh/id_rsa.pub \\\n  --from-file=known_hosts=~/.ssh/known_hosts\n```\n\n__NOTE:__  In the `dags.git.secret` the `known_hosts` file is present to reduce the possibility of a man-in-the-middle attack.\nHowever, if you want to implicitly trust all repo host signatures set `dags.git.sshKeyscan` to `true`.\n\n\n#### Option 2 - Mount a Shared Persistent Volume\n\nIn this method you store your DAGs in a Kubernetes Persistent Volume Claim (PVC), and use some external system to ensure this volume has your latest DAGs.\nFor example, you could use your CI/CD pipeline system to preform a sync as changes are pushed to a git repo.\n\nSince ALL Pods MUST HAVE the same collection of DAG files, it is recommended to create just one PVC that is shared. \nTo share a PVC with multiple Pods, the PVC needs to have `accessMode` set to `ReadOnlyMany` or `ReadWriteMany` (Note: different StorageClass support different [access modes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes)).\nIf you are using Kubernetes on a public cloud, a persistent volume controller is likely built in:\n[Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html),\n[Azure AKS](https://docs.microsoft.com/en-us/azure/aks/azure-files-dynamic-pv),\n[Google GKE](https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes).\n\nFor example, to use the storage class called `default`:\n```yaml\ndags:\n  persistence:\n    enabled: true\n    storageClass: default\n    accessMode: ReadOnlyMany\n    size: 1Gi\n```\n\n#### Option 2a -- Single PVC for DAGs \u0026 Logs\n\nYou may want to store DAGs and logs on the same volume and configure Airflow to use subdirectories for them.\nOne reason is that mounting the same volume multiple times with different subPaths can cause problems in Kubernetes, e.g. one of the mounts gets stuck during container initialisation.\n\nHere's an approach that achieves this:\n* Configure `airflow.extraVolume` and `airflow.extraVolumeMount` to put a volume at `/opt/airflow/efs`\n* Configure `dags.persistence.enabled` and `logs.persistence.enabled` to be `false`\n* Configure `dags.path` to be `/opt/airflow/efs/dags`\n* Configure `logs.path` to be `/opt/airflow/efs/logs`\n\n__WARNING:__ you must use a PVC which supports `accessMode: ReadWriteMany`\n\n### DAGs/Python requirements.txt\n\nIf you need to install Python packages to run your DAGs, you can place a `requirements.txt` file at the root of your `dags.path` folder and set:\n```yaml\ndags:\n  installRequirements: true\n```\n\n## Helm Chart Values\n\nFull documentation can be found in the comments of the `values.yaml` file, but a high level overview is provided here.\n\n__Global Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `airflow.image.*` | configs for the docker image of the web/scheduler/worker | `\u003csee values.yaml\u003e` |\n| `airflow.executor` | the airflow executor type to use | `CeleryExecutor` |\n| `airflow.fernetKey` | the fernet key used to encrypt the connections/variables in the database | `7T512UXSSmBOkpWimFHIVb8jK6lfmSAvx4mO6Arehnc=` |\n| `airflow.config` | environment variables for the web/scheduler/worker pods (for airflow configs) | `{}` |\n| `airflow.podAnnotations` | extra annotations for the web/scheduler/worker/flower Pods | `{}` |\n| `airflow.extraEnv` | extra environment variables for the web/scheduler/worker/flower Pods | `[]` |\n| `airflow.extraConfigmapMounts` | extra configMap volumeMounts for the web/scheduler/worker/flower Pods | `[]` |\n| `airflow.extraContainers` | extra containers for the web/scheduler/worker Pods | `[]` |\n| `airflow.extraPipPackages` | extra pip packages to install in the web/scheduler/worker Pods | `[]` |\n| `airflow.extraVolumeMounts` | extra volumeMounts for the web/scheduler/worker Pods | `[]` |\n| `airflow.extraVolumes` | extra volumes for the web/scheduler/worker Pods | `[]` |\n\n__Airflow Scheduler values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `scheduler.resources` | resource requests/limits for the scheduler pod | `{}` |\n| `scheduler.nodeSelector` | the nodeSelector configs for the scheduler pods | `{}` |\n| `scheduler.affinity` | the affinity configs for the scheduler pods | `{}` |\n| `scheduler.tolerations` | the toleration configs for the scheduler pods | `[]` |\n| `scheduler.labels` | labels for the scheduler Deployment | `{}` |\n| `scheduler.podLabels` | Pod labels for the scheduler Deployment | `{}` |\n| `scheduler.annotations` | annotations for the scheduler Deployment | `{}` |\n| `scheduler.podAnnotations` | Pod Annotations for the scheduler Deployment | `{}` |\n| `scheduler.podDisruptionBudget.*` | configs for the PodDisruptionBudget of the scheduler | `\u003csee values.yaml\u003e` |\n| `scheduler.connections` | custom airflow connections for the airflow scheduler | `[]` |\n| `scheduler.variables` | custom airflow variables for the airflow scheduler | `\"{}\"` |\n| `scheduler.pools` | custom airflow pools for the airflow scheduler | `\"{}\"` |\n| `scheduler.numRuns` | the value of the `airflow --num_runs` parameter used to run the airflow scheduler | `-1` |\n| `scheduler.initdb` | if we run `airflow initdb` when the scheduler starts | `true` |\n| `scheduler.preinitdb` | if we run `airflow initdb` inside a special initContainer | `false` |\n| `scheduler.initialStartupDelay` | the number of seconds to wait (in bash) before starting the scheduler container | `0` |\n| `scheduler.extraInitContainers` | extra init containers to run before the scheduler pod | `[]` |\n\n__Airflow WebUI Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `web.resources` | resource requests/limits for the airflow web pods | `{}` |\n| `web.replicas` | the number of web Pods to run | `1` |\n| `web.nodeSelector` | the number of web Pods to run | `{}` |\n| `web.affinity` | the affinity configs for the web Pods | `{}` |\n| `web.tolerations` | the toleration configs for the web Pods | `[]` |\n| `web.labels` | labels for the web Deployment | `{}` |\n| `web.podLabels` | Pod labels for the web Deployment | `{}` |\n| `web.annotations` | annotations for the web Deployment | `{}` |\n| `web.podAnnotations` | Pod annotations for the web Deployment | `{}` |\n| `web.service.*` | configs for the Service of the web pods | `\u003csee values.yaml\u003e` |\n| `web.baseUrl` | sets `AIRFLOW__WEBSERVER__BASE_URL` | `http://localhost:8080` |\n| `web.serializeDAGs` | sets `AIRFLOW__CORE__STORE_SERIALIZED_DAGS` | `false` |\n| `web.extraPipPackages` | extra pip packages to install in the web container | `[]` |\n| `web.initialStartupDelay` | the number of seconds to wait (in bash) before starting the web container | `0` |\n| `web.minReadySeconds` | the number of seconds to wait before declaring a new Pod available | `5` |\n| `web.readinessProbe.*` | configs for the web Service readiness probe | `\u003csee values.yaml\u003e` |\n| `web.livenessProbe.*` | configs for the web Service liveness probe | `\u003csee values.yaml\u003e` |\n| `web.secretsDir` | the directory in which to mount secrets on web containers | `/var/airflow/secrets` |\n| `web.secrets` | secret names which will be mounted as a file at `{web.secretsDir}/\u003csecret_name\u003e` | `[]` |\n| `web.secretsMap` | you can use secretsMap to specify a map and all the secrets will be stored within it secrets will be mounted as files at `{web.secretsDir}/\u003csecrets_in_map\u003e`. If you use web.secretsMap, then it overrides `web.secrets`.| `\"\"` |\n\n__Airflow Worker Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `workers.enabled` | if the airflow workers StatefulSet should be deployed | `true` |\n| `workers.resources` | resource requests/limits for the airflow worker Pods | `{}` |\n| `workers.replicas` | the number of workers Pods to run | `1` |\n| `workers.nodeSelector` | the nodeSelector configs for the worker Pods | `{}` |\n| `workers.affinity` | the affinity configs for the worker Pods | `{}` |\n| `workers.tolerations` | the toleration configs for the worker Pods | `[]` |\n| `workers.labels` | labels for the worker StatefulSet | `{}` |\n| `workers.podLabels` | Pod labels for the worker StatefulSet | `{}` |\n| `workers.annotations` | annotations for the worker StatefulSet | `{}` |\n| `workers.podAnnotations` | Pod annotations for the worker StatefulSet | `{}` |\n| `workers.autoscaling.*` | configs for the HorizontalPodAutoscaler of the worker Pods | `\u003csee values.yaml\u003e` |\n| `workers.initialStartupDelay` | the number of seconds to wait (in bash) before starting each worker container | `0` |\n| `workers.celery.*` | configs for the celery worker Pods | `\u003csee values.yaml\u003e` |\n| `workers.terminationPeriod` | how many seconds to wait for tasks on a worker to finish before SIGKILL | `60` |\n| `workers.secretsDir` | directory in which to mount secrets on worker containers | `/var/airflow/secrets` |\n| `workers.secrets` | secret names which will be mounted as a file at `{workers.secretsDir}/\u003csecret_name\u003e` | `[]` |\n| `workers.secretsMap` | you can use secretsMap to specify a map and all the secrets will be stored within it secrets will be mounted as files at `{workers.secretsDir}/\u003csecrets_in_map\u003e`. If you use workers.secretsMap, then it overrides `workers.secrets`.| `\"\"` |\n\n__Airflow Flower Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `flower.enabled` | if the Flower UI should be deployed | `true` |\n| `flower.resources` | resource requests/limits for the flower Pods | `{}` |\n| `flower.affinity` | the affinity configs for the flower Pods | `{}` |\n| `flower.tolerations` | the toleration configs for the flower Pods | `[]` |\n| `flower.labels` | labels for the flower Deployment | `{}` |\n| `flower.podLabels` | Pod labels for the flower Deployment | `{}` |\n| `flower.annotations` | annotations for the flower Deployment | `{}` |\n| `flower.podAnnotations` | Pod annotations for the flower Deployment | `{}` |\n| `flower.basicAuthSecret` | the name of a pre-created secret containing the basic authentication value for flower | `\"\"` |\n| `flower.basicAuthSecretKey` | the key within `flower.basicAuthSecret` containing the basic authentication string | `\"\"` |\n| `flower.urlPrefix` | sets `AIRFLOW__CELERY__FLOWER_URL_PREFIX` | `\"\"` |\n| `flower.service.*` | configs for the Service of the flower Pods | `\u003csee values.yaml\u003e` |\n| `flower.initialStartupDelay` | the number of seconds to wait (in bash) before starting the flower container | `0` |\n| `flower.extraConfigmapMounts` | extra ConfigMaps to mount on the flower Pods | `[]` |\n\n__Airflow Logs Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `logs.path` | the airflow logs folder | `/opt/airflow/logs` |\n| `logs.persistence.*` | configs for the logs PVC | `\u003csee values.yaml\u003e` |\n\n__Airflow DAGs Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `dags.path` | the airflow dags folder | `/opt/airflow/dags` |\n| `dags.doNotPickle` | whether to disable pickling dags from the scheduler to workers | `false` |\n| `dags.installRequirements` | install any Python `requirements.txt` at the root of `dags.path` automatically | `false` |\n| `dags.persistence.*` | configs for the dags PVC | `\u003csee values.yaml\u003e` |\n| `dags.git.*` | configs for the DAG git repository \u0026 sync container | `\u003csee values.yaml\u003e` |\n| `dags.initContainer.*` | configs for the git-clone container | `\u003csee values.yaml\u003e` |\n\n__Airflow Ingress Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `ingress.enabled` | if we should deploy Ingress resources | `false` |\n| `ingress.web.*` | configs for the Ingress of the web Service | `\u003csee values.yaml\u003e` |\n| `ingress.flower.*` | configs for the Ingress of the flower Service | `\u003csee values.yaml\u003e` |\n\n__Airflow Kubernetes Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `rbac.create` | if Kubernetes RBAC resources are created | `true` |\n| `serviceAccount.create` | if a Kubernetes ServiceAccount is created | `true` |\n| `serviceAccount.name` | the name of the ServiceAccount | `\"\"` |\n| `serviceAccount.annotations` | annotations for the ServiceAccount | `{}` |\n| `extraManifests` | additional Kubernetes manifests to include with this chart | `[]` |\n\n__Airflow Database (Internal PostgreSQL) Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `postgresql.enabled` | if the `stable/postgresql` chart is used | `true` |\n| `postgresql.postgresqlDatabase` | the postgres database to use | `airflow` |\n| `postgresql.postgresqlUsername` | the postgres user to create | `postgres` |\n| `postgresql.postgresqlPassword` | the postgres user's password | `airflow` |\n| `postgresql.existingSecret` | the name of a pre-created secret containing the postgres password | `\"\"` |\n| `postgresql.existingSecretKey` | the key within `postgresql.passwordSecret` containing the password string | `postgresql-password` |\n| `postgresql.persistence.*` | configs for the PVC of postgresql | `\u003csee values.yaml\u003e` |\n\n__Airflow Database (External) Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `externalDatabase.type` | the type of external database: {mysql,postgres} | `postgres` |\n| `externalDatabase.host` | the host of the external database | `localhost` |\n| `externalDatabase.port` | the port of the external database | `5432` |\n| `externalDatabase.database` | the database/scheme to use within the the external database | `airflow` |\n| `externalDatabase.user` | the user of the external database | `airflow` |\n| `externalDatabase.passwordSecret` | the name of a pre-created secret containing the external database password | `\"\"` |\n| `externalDatabase.passwordSecretKey` | the key within `externalDatabase.passwordSecret` containing the password string | `postgresql-password` |\n\n__Airflow Redis (Internal) Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `redis.enabled` | if the `stable/redis` chart is used | `true` |\n| `redis.password` | the redis password | `airflow` |\n| `redis.existingSecret` | the name of a pre-created secret containing the redis password | `\"\"` |\n| `redis.existingSecretKey` | the key within `redis.existingSecret` containing the password string | `redis-password` |\n| `redis.cluster.*` | configs for redis cluster mode | `\u003csee values.yaml\u003e` |\n| `redis.master.*` | configs for the redis master | `\u003csee values.yaml\u003e` |\n| `redis.slave.*` | configs for the redis slaves | `\u003csee values.yaml\u003e` |\n\n__Airflow Redis (External) Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `externalRedis.host` | the host of the external redis | `localhost` |\n| `externalRedis.port` | the port of the external redis | `6379` |\n| `externalRedis.databaseNumber` | the database number to use within the the external redis | `1` |\n| `externalRedis.passwordSecret` | the name of a pre-created secret containing the external redis password | `\"\"` |\n| `externalRedis.passwordSecretKey` | the key within `externalRedis.passwordSecret` containing the password string | `redis-password` |\n\n__Airflow Prometheus Values:__\n\n| Parameter | Description | Default |\n| --- | --- | --- |\n| `serviceMonitor.enabled` | if the ServiceMonitor resources should be deployed | `false` |\n| `serviceMonitor.selector` | labels for ServiceMonitor, so that Prometheus can select it | `{ prometheus: \"kube-prometheus\" }` |\n| `serviceMonitor.path` | the ServiceMonitor web endpoint path | `/admin/metrics` |\n| `serviceMonitor.interval` | the ServiceMonitor web endpoint path | `30s` |\n| `prometheusRule.enabled` | if the PrometheusRule resources should be deployed | `false` |\n| `prometheusRule.additionalLabels` | labels for PrometheusRule, so that Prometheus can select it | `{}` |\n| `prometheusRule.groups` | alerting rules for Prometheus | `[]` |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnccloud%2Fairflow-chart","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnccloud%2Fairflow-chart","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnccloud%2Fairflow-chart/lists"}