{"id":19534455,"url":"https://github.com/flyteorg/airflow-provider-flyte","last_synced_at":"2025-04-26T14:35:07.131Z","repository":{"id":37775456,"uuid":"503663707","full_name":"flyteorg/airflow-provider-flyte","owner":"flyteorg","description":"Flyte Airflow Provider","archived":false,"fork":false,"pushed_at":"2022-07-15T14:03:10.000Z","size":53,"stargazers_count":9,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-11T01:56:06.382Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flyteorg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-15T07:37:07.000Z","updated_at":"2023-11-30T10:11:07.000Z","dependencies_parsed_at":"2022-07-12T16:52:30.859Z","dependency_job_id":null,"html_url":"https://github.com/flyteorg/airflow-provider-flyte","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flyteorg%2Fairflow-provider-flyte","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flyteorg%2Fairflow-provider-flyte/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flyteorg%2Fairflow-provider-flyte/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flyteorg%2Fairflow-provider-flyte/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flyteorg","download_url":"https://codeload.github.com/flyteorg/airflow-provider-flyte/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224036217,"owners_count":17245035,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T02:14:23.066Z","updated_at":"2024-11-11T02:14:24.478Z","avatar_url":"https://github.com/flyteorg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flyte Provider for Apache Airflow\n\nThis package provides an operator, a sensor, and a hook that integrates [Flyte](flyte.org/) into Apache Airflow.\n`FlyteOperator` is helpful to trigger a task/workflow in Flyte and `FlyteSensor` enables monitoring a Flyte execution status\nfor completion.\n\n## Installation\n\nPrerequisites: An environment running `apache-airflow`.\n\n```\npip install airflow-provider-flyte\n```\n\n## Configuration\n\nIn the Airflow UI, configure a _Connection_ for Flyte.\n\n- Host (required): The FlyteAdmin host.\n- Port (optional): The FlyteAdmin port.\n- Login (optional): `client_id`\n- Password (optional): `client_credentials_secret`\n- Extra (optional): Specify the `extra` parameter as JSON dictionary to provide additional parameters.\n  - `project`: The default project to connect to.\n  - `domain`: The default domain to connect to.\n  - `insecure`: Whether to use SSL or not.\n  - `command`: The command to execute to return a token using an external process.\n  - `scopes`: List of scopes to request.\n  - `auth_mode`: The OAuth mode to use. Defaults to pkce flow.\n  - `env_prefix`: Prefix that will be used to lookup for injected secrets at runtime.\n  - `default_dir`: Default directory that will be used to find secrets as individual files.\n  - `file_prefix`: Prefix for the file in the `default_dir`.\n  - `statsd_host`: The statsd host.\n  - `statsd_port`: The statsd port.\n  - `statsd_disabled`: Whether to send statsd or not.\n  - `statsd_disabled_tags`: Turn on to reduce cardinality.\n  - `local_sandbox_path`\n  - S3 Config:\n    - `s3_enable_debug`\n    - `s3_endpoint`\n    - `s3_retries`\n    - `s3_backoff`\n    - `s3_access_key_id`\n    - `s3_secret_access_key`\n  - GCS Config:\n    - `gsutil_parallelism`\n\n## Modules\n\n### [Flyte Operator](https://github.com/flyteorg/airflow-provider-flyte/blob/master/flyte_provider/operators/flyte.py)\n\nThe `FlyteOperator` requires a `flyte_conn_id` to fetch all the connection-related\nparameters that are useful to instantiate `FlyteRemote`. Also, you must give a\n`launchplan_name` to trigger a workflow, or `task_name` to trigger a task; you can give a\nhandful of other values that are optional, such as `project`, `domain`, `max_parallelism`,\n`raw_data_prefix`, `kubernetes_service_account`, `labels`, `annotations`,\n`secrets`, `notifications`, `disable_notifications`, `oauth2_client`, `version`, and `inputs`.\n\nImport into your DAG via:\n\n```\nfrom flyte_provider.operators.flyte import FlyteOperator\n```\n\n### [Flyte Sensor](https://github.com/flyteorg/airflow-provider-flyte/blob/master/flyte_provider/sensors/flyte.py)\n\nIf you need to wait for an execution to complete, use `FlyteSensor`.\nMonitoring with `FlyteSensor` allows you to trigger downstream processes only when the Flyte executions are complete.\n\nImport into your DAG via:\n\n```\nfrom flyte_provider.sensors.flyte import FlyteSensor\n```\n\n## Examples\n\nSee the [examples](https://github.com/flyteorg/airflow-provider-flyte/tree/master/flyte_provider/example_dags) directory for an example DAG.\n\n## Issues\n\nPlease file issues and open pull requests [here](https://github.com/flyteorg/airflow-provider-flyte).\nIf you hit any roadblock, hit us up on [Slack](https://slack.flyte.org/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflyteorg%2Fairflow-provider-flyte","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflyteorg%2Fairflow-provider-flyte","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflyteorg%2Fairflow-provider-flyte/lists"}