https://github.com/imaginary-cherry/mageflow
A support package for task manager orchestration.
https://github.com/imaginary-cherry/mageflow
asyncio hatchet microservices python task task-manager
Last synced: 3 months ago
JSON representation
A support package for task manager orchestration.
- Host: GitHub
- URL: https://github.com/imaginary-cherry/mageflow
- Owner: imaginary-cherry
- License: mit
- Created: 2025-10-13T10:41:39.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2026-03-27T15:08:51.000Z (3 months ago)
- Last Synced: 2026-03-27T23:34:15.990Z (3 months ago)
- Topics: asyncio, hatchet, microservices, python, task, task-manager
- Language: Python
- Homepage: https://imaginary-cherry.github.io/mageflow/
- Size: 8.05 MB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 50
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/MIT)
[](https://codecov.io/gh/imaginary-cherry/mageflow?flags[0]=mageflow-unit)
[](https://codecov.io/gh/imaginary-cherry/mageflow?flags[0]=mageflow-integration)
[](https://codecov.io/gh/imaginary-cherry/mageflow?flags[0]=thirdmagic)
[](https://badge.fury.io/py/mageflow)
[](https://pepy.tech/project/mageflow)
[](https://imaginary-cherry.github.io/mageflow/)
[](https://coderabbit.ai)
📚 **[Full Documentation](https://imaginary-cherry.github.io/mageflow/)** | [Installation](https://imaginary-cherry.github.io/mageflow/setup/) | [API Reference](https://imaginary-cherry.github.io/mageflow/api/)
# MageFlow
**Ma**nage **G**raph **E**xecution Flow - A unified interface for task orchestration across different task managers.
## Why MageFlow?
Instead of spreading workflow logic throughout your codebase, MageFlow centralizes task orchestration with a clean, unified API. Switch between task managers (Hatchet, Taskiq, etc.) without rewriting your orchestration code.
## Key Features
🔗 **Task Chaining** - Sequential workflows where tasks depend on previous completions
🐝 **Task Swarms** - Parallel execution with intelligent coordination
📞 **Callback System** - Robust success/error handling
🎯 **Task Signatures** - Flexible task definition with validation
⏯️ **Lifecycle Control** - Pause, resume, and monitor task execution
💾 **Persistent State** - Redis-backed state management with recovery
## Installation
```bash
pip install mageflow[hatchet] # For Hatchet backend
```
## Quick Setup
```python
import asyncio
import redis
from hatchet_sdk import Hatchet, ClientConfig
import mageflow
# Configure backend and Redis
config = ClientConfig(token="your-hatchet-token")
redis_client = redis.asyncio.from_url("redis://localhost", decode_responses=True)
hatchet_client = Hatchet(config=config)
# Create MageFlow instance
mf = mageflow.Mageflow(hatchet_client, redis_client=redis_client)
```
## Example Usage
### Define Tasks
```python
from pydantic import BaseModel
class ProcessData(BaseModel):
data: str
@mf.task(name="process-data", input_validator=ProcessData)
async def process_data(msg: ProcessData):
return {"processed": msg.data}
@mf.task(name="send-notification")
async def send_notification(msg):
print(f"Notification sent: {msg}")
return {"status": "sent"}
```
### Chain Tasks
```python
# Sequential execution
workflow = await mageflow.chain([
process_data_task,
send_notification_task
], name="data-pipeline")
```
### Parallel Swarms
```python
# Parallel execution
swarm = await mageflow.swarm([
process_user_task,
update_cache_task,
send_email_task
], task_name="user-onboarding")
```
### Task Signatures with Callbacks
```python
task_signature = await mageflow.sign(
task_name="process-order",
task_identifiers={"order_id": "12345"},
success_callbacks=[send_confirmation_task],
error_callbacks=[handle_error_task]
)
```
## Use Cases
- **Data Pipelines** - ETL operations with error handling
- **Microservice Coordination** - Orchestrate distributed service calls
- **Batch Processing** - Parallel processing of large datasets
- **User Workflows** - Multi-step onboarding and registration
- **Content Processing** - Media processing with multiple stages
## MageFlow Viewer
A desktop app for visualizing your workflows as interactive task graphs. See the [full docs](docs/viewer.md) for details.
**Homebrew (macOS):**
```bash
brew install imaginary-cherry/mageflow/mage-voyance # stable
brew install imaginary-cherry/mageflow/mage-voyance-beta # beta
```
**Direct download:** [GitHub Releases](https://github.com/imaginary-cherry/mageflow/releases)
## Documentation
- [Setup Guide](docs/setup.md)
- [API Reference](docs/api/)
- [Task Lifecycle](docs/documentation/task-lifecycle.md)
- [Callbacks](docs/documentation/callbacks.md)
## License
MIT