https://github.com/fangyinc/airflow-ray-executor

Airflow executor implemented using ray
https://github.com/fangyinc/airflow-ray-executor

Last synced: 1 day ago
JSON representation

Airflow executor implemented using ray

• Host: GitHub
• URL: https://github.com/fangyinc/airflow-ray-executor
• Owner: fangyinc
• License: apache-2.0
• Created: 2022-01-18T19:28:33.000Z (almost 3 years ago)
• Default Branch: master
• Last Pushed: 2022-01-18T20:29:01.000Z (almost 3 years ago)
• Last Synced: 2024-10-11T22:49:14.125Z (about 1 month ago)
• Language: Python
• Size: 16.6 KB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Airflow Ray Executor

Airflow executor implemented using [ray](https://github.com/ray-project/ray)

[**中文**](README_ZH.md)

## Usage

```shell

$ pip install airflow-ray-executor

```

Edit your ``airflow.cfg`` to set your executor to class: `airflow_ray_executor.RayExecutor` and add ray client address to this file, example:

```pycon

executor = airflow_ray_executor.RayExecutor

[ray]

# ray client address to connect to ray cluster

# Ray Executor will start Ray on a single machine if not provided

client = ray://127.0.0.1:10001

```

Please note:

Airflow not support ```sqlite``` database when executor neither ```DebugExecutor``` nor ```SequentialExecutor```