https://github.com/butla/redis_task_dags

A simple runner for task DAGs based on Redis
https://github.com/butla/redis_task_dags

Last synced: 14 days ago
JSON representation

A simple runner for task DAGs based on Redis

• Host: GitHub
• URL: https://github.com/butla/redis_task_dags
• Owner: butla
• License: mit
• Created: 2020-07-02T18:49:26.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2021-09-14T16:57:12.000Z (over 3 years ago)
• Last Synced: 2024-12-23T02:44:19.002Z (17 days ago)
• Language: Python
• Size: 4.88 KB
• Stars: 1
• Watchers: 4
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# redis_task_dags

A simple runner for task DAGs based on Redis.

Currently, a very early work in progress :) I'm just fiddling with the idea.

Maybe it'll go into `rq`, maybe it'll be a separate project.

I want to leverage as much of the built-in Redis functionality, to have as little as possible Python code.

Maybe that way, this can be a base for task DAGs implementation in many languages.

I have different issues with the current task runners for Python:

- rq: doesn't support DAGs of tasks

- celery: buggy, convoluted

- Airflow - really unflexible (why can't I just define and run DAGs from code?), needlessly heavyweight