https://github.com/yohamta/dagu
Local-first workflow engine, built for self-hosting. Alternative to Airflow, Cron, etc. It aims to solve greater problems.
https://github.com/yohamta/dagu
automation continuous-delivery cron dag-scheduling devops-pipeline directed-acyclic-graph golang scheduler task-runner task-scheduler workflow workflow-automation workflow-engine workflow-management workflow-scheduler workflow-tool
Last synced: 5 months ago
JSON representation
Local-first workflow engine, built for self-hosting. Alternative to Airflow, Cron, etc. It aims to solve greater problems.
- Host: GitHub
- URL: https://github.com/yohamta/dagu
- Owner: dagu-org
- License: gpl-3.0
- Created: 2022-04-22T13:00:42.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2025-06-26T09:50:46.000Z (5 months ago)
- Last Synced: 2025-06-26T11:03:59.268Z (5 months ago)
- Topics: automation, continuous-delivery, cron, dag-scheduling, devops-pipeline, directed-acyclic-graph, golang, scheduler, task-runner, task-scheduler, workflow, workflow-automation, workflow-engine, workflow-management, workflow-scheduler, workflow-tool
- Language: Go
- Homepage: http://dagu.cloud
- Size: 38.6 MB
- Stars: 2,328
- Watchers: 20
- Forks: 181
- Open Issues: 111
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
- awesome-trevor - Dagu - Cron alternative with a Web UI (workflows / DAGs) (Programming / Golang)
- awesome-tools - dagu - Cron alternative with a Web UI, but with much more capabilities. (Orchestration / Task Scheduling / Task scheduling)
- awesome-go-extra - dagu - 04-22T13:00:42Z|2022-08-25T14:17:10Z| (Job Scheduler / Advanced Console UIs)
- awesomeLibrary - dagu - A just another Cron alternative with a Web UI, but with much more capabilities. It aims to solve greater problems. (语言资源库 / go)
README
Local-First Workflow Engine, Built for Self-Hosting
Zero dependencies. Language agnostic. Self-contained.
Docs |
Quick Start |
Features |
Installation |
Community
## What is Dagu?
Dagu solves the problem of complex workflow orchestration without requiring a dedicated infrastructure team. Unlike traditional workflow engines that demand databases, message queues, and careful operational overhead, Dagu runs as a single binary with zero external dependencies.
After managing hundreds of cron jobs across multiple servers, I built Dagu to bring sanity to workflow automation. It handles scheduling, dependencies, error recovery, and monitoring - everything you need for production workflows, without the complexity.
[→ Learn the core concepts](https://docs.dagu.cloud/getting-started/concepts)
### Design Philosophy
1. **Local‑first.** Workflows should run offline on laptops, air‑gapped servers, or the cloud—your choice.
2. **Zero foot‑print.** One static binary; no databases, brokers, or sidecars.
3. **Bring‑your‑own language.** Bash, Python, Go, or anything just works.
## Features
- [DAG definition](https://docs.dagu.cloud/writing-workflows/examples/) - Express complex dependencies in readable YAML
- [Scheduling](https://docs.dagu.cloud/features/scheduling) - Cron expressions with timezone support
- [Queueing](https://docs.dagu.cloud/features/queues) - Control concurrency with named queues
- [Error handling](https://docs.dagu.cloud/writing-workflows/error-handling) - Retries, failure handlers, cleanup hooks
- [Conditional execution](https://docs.dagu.cloud/writing-workflows/control-flow) - Run steps based on conditions
- [Parallel execution](https://docs.dagu.cloud/features/parallel-execution) - Control concurrent step execution
- [Variables & Parameters](https://docs.dagu.cloud/writing-workflows/data-variables) - Pass data between steps, parameterize workflows
- [Docker support](https://docs.dagu.cloud/features/executors/docker) - Run steps in containers
- [SSH executor](https://docs.dagu.cloud/features/executors/ssh) - Execute commands on remote hosts
- [HTTP requests](https://docs.dagu.cloud/features/executors/http) - Integrate with APIs
- [Email notifications](https://docs.dagu.cloud/features/email-notifications) - SMTP integration
- [Hierarchical workflows](https://docs.dagu.cloud/writing-workflows/advanced#hierarchical-workflows) - Nest DAGs to any depth
- [Authentication](https://docs.dagu.cloud/features/authentication) - Basic auth, API tokens, TLS
- [Web UI](https://docs.dagu.cloud/overview/web-ui) - Real-time monitoring and control
- [REST API](https://docs.dagu.cloud/overview/api) - Full programmatic access
## Quick Start
```bash
# Install
curl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash
# Create dagu configuration directory
mkdir -p ~/.config/dagu/dags
# Create your first workflow
mkdir -p ~/.config/dagu/dags
cat > ~/.config/dagu/dags/hello.yaml << 'EOF'
steps:
- name: hello
command: echo "Hello from Dagu!"
- name: world
command: echo "Running step 2"
EOF
# Execute it
dagu start hello
# Check the status
dagu status hello
# Start the web UI
dagu start-all
# Visit http://localhost:8080
```
## Installation
### macOS / Linux
```bash
# Latest
curl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash
# Specific version
curl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash -s -- --version v1.17.0
# Install to a specific directory
curl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash -s -- --prefix /path/to/install
# Homebrew
brew install dagu-org/brew/dagu
```
### Docker
```bash
docker run -d \
--name dagu \
-p 8080:8080 \
-v ~/.dagu:/var/lib/dagu \
ghcr.io/dagu-org/dagu:latest dagu start-all
```
### Manual Download
Download from [releases](https://github.com/dagu-org/dagu/releases) and add to PATH.
## Documentation
- [Getting Started](https://docs.dagu.cloud/getting-started/quickstart) - Tutorial and first steps
- [Core Concepts](https://docs.dagu.cloud/getting-started/concepts) - Architecture and design
- [Writing Workflows](https://docs.dagu.cloud/writing-workflows/) - Complete authoring guide
- [CLI Reference](https://docs.dagu.cloud/reference/cli) - Command-line usage
- [API Reference](https://docs.dagu.cloud/reference/api) - REST API documentation
- [Configuration](https://docs.dagu.cloud/configurations/server) - Configuration options
- [Changelog](https://docs.dagu.cloud/reference/changelog) - Release notes and version history
## Examples
Find more in our [examples documentation](https://docs.dagu.cloud/writing-workflows/examples/).
### ETL Pipeline
```yaml
name: daily-etl
schedule: "0 2 * * *"
steps:
- name: extract
command: python extract.py
output: DATA_FILE
- name: validate
command: python validate.py ${DATA_FILE}
- name: transform
command: python transform.py ${DATA_FILE}
retryPolicy:
limit: 3
- name: load
command: python load.py ${DATA_FILE}
```
### Hierarchical Workflows
```yaml
steps:
- name: data-pipeline
run: etl
params: "ENV=prod REGION=us-west-2"
- name: parallel-jobs
run: batch
parallel:
items: ["job1", "job2", "job3"]
maxConcurrency: 2
params: "JOB=${ITEM}"
---
name: etl
params:
- ENV
- REGION
steps:
- name: process
command: python etl.py --env ${ENV} --region ${REGION}
---
name: batch
params:
- JOB
steps:
- name: process
command: python process.py --job ${JOB}
```
### Container-based Pipeline
```yaml
name: ml-pipeline
steps:
- name: prepare-data
executor:
type: docker
config:
image: python:3.11
autoRemove: true
volumes:
- /data:/data
command: python prepare.py
- name: train-model
executor:
type: docker
config:
image: tensorflow/tensorflow:latest-gpu
command: python train.py
- name: deploy
command: kubectl apply -f model-deployment.yaml
preconditions:
- condition: "`date +%u`"
expected: "re:[1-5]" # Weekdays only
```
## Web Interface
[Learn more about the Web UI →](https://docs.dagu.cloud/overview/web-ui)
Real-time monitoring of all workflows
Visual workflow editor with validation
Detailed execution logs with stdout/stderr separation
## Use Cases
- **Data Engineering** - ETL pipelines, data validation, warehouse loading
- **Machine Learning** - Training pipelines, model deployment, experiment tracking
- **DevOps** - CI/CD workflows, infrastructure automation, deployment orchestration
- **Media Processing** - Video transcoding, image manipulation, content pipelines
- **Business Automation** - Report generation, data synchronization, scheduled tasks
## Roadmap
- [ ] Run steps in a DAG across multiple machines (distributed execution)
- [ ] Track artifacts by dropping files in `$DAGU_ARTIFACTS`
- [ ] Pause executions for webhooks, approvals, or any event (human-in-the-loop, event-driven workflows)
- [ ] Integrate with AI agents via MCP (Model Context Protocol)
## Building from Source
Prerequisites: Go 1.24+, Node.js, pnpm
```bash
git clone https://github.com/dagu-org/dagu.git && cd dagu
make build
make run
```
## Contributing
Contributions are welcome. See our [documentation](https://docs.dagu.cloud) for development setup.
- [Discord Community](https://discord.gg/gpahPUjGRk)
- [GitHub Issues](https://github.com/dagu-org/dagu/issues)
## Contributors
## License
GNU GPLv3 - See [LICENSE](./LICENSE)
---
If you find Dagu useful, please ⭐ star this repository