{"id":29127756,"url":"https://github.com/yohamta/dagu","last_synced_at":"2025-06-30T01:01:57.783Z","repository":{"id":37010139,"uuid":"484434975","full_name":"dagu-org/dagu","owner":"dagu-org","description":"Local-first workflow engine, built for self-hosting. Alternative to Airflow, Cron, etc. It aims to solve greater problems.","archived":false,"fork":false,"pushed_at":"2025-06-26T09:50:46.000Z","size":40428,"stargazers_count":2328,"open_issues_count":111,"forks_count":181,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-06-26T11:03:59.268Z","etag":null,"topics":["automation","continuous-delivery","cron","dag-scheduling","devops-pipeline","directed-acyclic-graph","golang","scheduler","task-runner","task-scheduler","workflow","workflow-automation","workflow-engine","workflow-management","workflow-scheduler","workflow-tool"],"latest_commit_sha":null,"homepage":"http://dagu.cloud","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dagu-org.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["dagu-org"]}},"created_at":"2022-04-22T13:00:42.000Z","updated_at":"2025-06-26T09:50:50.000Z","dependencies_parsed_at":"2023-10-15T03:47:17.886Z","dependency_job_id":"a897d87c-0b3a-4597-90bb-3117bcda6eb6","html_url":"https://github.com/dagu-org/dagu","commit_stats":{"total_commits":1008,"total_committers":40,"mean_commits":25.2,"dds":0.09424603174603174,"last_synced_commit":"6d377ed7f2ea7b755df2455d5d6d6050e8813b56"},"previous_names":["yohamta/dagu","dagu-labs/dagu","yohamta/jobctl","hotaruswarm/dagu","dagu-dev/dagu","daguflow/dagu","dagu-org/dagu","dagu-go/dagu"],"tags_count":160,"template":false,"template_full_name":null,"purl":"pkg:github/dagu-org/dagu","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dagu-org%2Fdagu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dagu-org%2Fdagu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dagu-org%2Fdagu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dagu-org%2Fdagu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dagu-org","download_url":"https://codeload.github.com/dagu-org/dagu/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dagu-org%2Fdagu/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262690385,"owners_count":23349168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","continuous-delivery","cron","dag-scheduling","devops-pipeline","directed-acyclic-graph","golang","scheduler","task-runner","task-scheduler","workflow","workflow-automation","workflow-engine","workflow-management","workflow-scheduler","workflow-tool"],"created_at":"2025-06-30T01:01:48.152Z","updated_at":"2025-06-30T01:01:57.742Z","avatar_url":"https://github.com/dagu-org.png","language":"Go","funding_links":["https://github.com/sponsors/dagu-org"],"categories":["Programming","Job Scheduler","语言资源库","Orchestration / Task Scheduling","Full fledged product"],"sub_categories":["Golang","Advanced Console UIs","go","Task scheduling"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"./assets/images/dagu-logo.webp\" width=\"480\" alt=\"Dagu Logo\"\u003e\n  \n  \u003ch3\u003eLocal-First Workflow Engine, Built for Self-Hosting\u003c/h3\u003e\n  \u003cp\u003eZero dependencies. Language agnostic. Self-contained.\u003c/p\u003e\n  \n  \u003cp\u003e\n    \u003ca href=\"https://docs.dagu.cloud/reference/changelog\"\u003e\u003cimg src=\"https://img.shields.io/github/release/dagu-org/dagu.svg?style=flat-square\" alt=\"Latest Release\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/dagu-org/dagu/actions/workflows/ci.yaml\"\u003e\u003cimg src=\"https://img.shields.io/github/actions/workflow/status/dagu-org/dagu/ci.yaml?style=flat-square\" alt=\"Build Status\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://codecov.io/gh/dagu-org/dagu\"\u003e\u003cimg src=\"https://img.shields.io/codecov/c/github/dagu-org/dagu?style=flat-square\" alt=\"Code Coverage\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://discord.gg/gpahPUjGRk\"\u003e\u003cimg src=\"https://img.shields.io/discord/1095289480774172772?style=flat-square\u0026logo=discord\" alt=\"Discord\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://bsky.app/profile/dagu-org.bsky.social\"\u003e\u003cimg src=\"https://img.shields.io/badge/Bluesky-0285FF?style=flat-square\u0026logo=bluesky\u0026logoColor=white\" alt=\"Bluesky\"\u003e\u003c/a\u003e\n  \u003c/p\u003e\n  \n  \u003cp\u003e\n    \u003ca href=\"https://docs.dagu.cloud\"\u003eDocs\u003c/a\u003e |\n    \u003ca href=\"#quick-start\"\u003eQuick Start\u003c/a\u003e |\n    \u003ca href=\"#features\"\u003eFeatures\u003c/a\u003e |\n    \u003ca href=\"#installation\"\u003eInstallation\u003c/a\u003e |\n    \u003ca href=\"https://discord.gg/gpahPUjGRk\"\u003eCommunity\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\n## What is Dagu?\n\nDagu solves the problem of complex workflow orchestration without requiring a dedicated infrastructure team. Unlike traditional workflow engines that demand databases, message queues, and careful operational overhead, Dagu runs as a single binary with zero external dependencies. \n\nAfter managing hundreds of cron jobs across multiple servers, I built Dagu to bring sanity to workflow automation. It handles scheduling, dependencies, error recovery, and monitoring - everything you need for production workflows, without the complexity.\n\n[→ Learn the core concepts](https://docs.dagu.cloud/getting-started/concepts)\n\n### Design Philosophy\n\n1. **Local‑first.** Workflows should run offline on laptops, air‑gapped servers, or the cloud—your choice.  \n2. **Zero foot‑print.** One static binary; no databases, brokers, or sidecars.  \n3. **Bring‑your‑own language.** Bash, Python, Go, or anything just works.\n\n## Features\n\n- [DAG definition](https://docs.dagu.cloud/writing-workflows/examples/) - Express complex dependencies in readable YAML\n- [Scheduling](https://docs.dagu.cloud/features/scheduling) - Cron expressions with timezone support\n- [Queueing](https://docs.dagu.cloud/features/queues) - Control concurrency with named queues\n- [Error handling](https://docs.dagu.cloud/writing-workflows/error-handling) - Retries, failure handlers, cleanup hooks\n- [Conditional execution](https://docs.dagu.cloud/writing-workflows/control-flow) - Run steps based on conditions\n- [Parallel execution](https://docs.dagu.cloud/features/parallel-execution) - Control concurrent step execution\n- [Variables \u0026 Parameters](https://docs.dagu.cloud/writing-workflows/data-variables) - Pass data between steps, parameterize workflows\n- [Docker support](https://docs.dagu.cloud/features/executors/docker) - Run steps in containers\n- [SSH executor](https://docs.dagu.cloud/features/executors/ssh) - Execute commands on remote hosts\n- [HTTP requests](https://docs.dagu.cloud/features/executors/http) - Integrate with APIs\n- [Email notifications](https://docs.dagu.cloud/features/email-notifications) - SMTP integration\n- [Hierarchical workflows](https://docs.dagu.cloud/writing-workflows/advanced#hierarchical-workflows) - Nest DAGs to any depth\n- [Authentication](https://docs.dagu.cloud/features/authentication) - Basic auth, API tokens, TLS\n- [Web UI](https://docs.dagu.cloud/overview/web-ui) - Real-time monitoring and control\n- [REST API](https://docs.dagu.cloud/overview/api) - Full programmatic access\n\n## Quick Start\n\n```bash\n# Install\ncurl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash\n\n# Create dagu configuration directory\nmkdir -p ~/.config/dagu/dags\n\n# Create your first workflow\nmkdir -p ~/.config/dagu/dags\ncat \u003e ~/.config/dagu/dags/hello.yaml \u003c\u003c 'EOF'\nsteps:\n  - name: hello\n    command: echo \"Hello from Dagu!\"\n    \n  - name: world  \n    command: echo \"Running step 2\"\nEOF\n\n# Execute it\ndagu start hello\n\n# Check the status\ndagu status hello\n\n# Start the web UI\ndagu start-all\n# Visit http://localhost:8080\n```\n\n## Installation\n\n### macOS / Linux\n\n```bash\n# Latest\ncurl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash\n\n# Specific version\ncurl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash -s -- --version v1.17.0\n\n# Install to a specific directory\ncurl -L https://raw.githubusercontent.com/dagu-org/dagu/main/scripts/installer.sh | bash -s -- --prefix /path/to/install\n\n# Homebrew\nbrew install dagu-org/brew/dagu\n```\n\n### Docker\n```bash\ndocker run -d \\\n  --name dagu \\\n  -p 8080:8080 \\\n  -v ~/.dagu:/var/lib/dagu \\\n  ghcr.io/dagu-org/dagu:latest dagu start-all\n```\n\n### Manual Download\nDownload from [releases](https://github.com/dagu-org/dagu/releases) and add to PATH.\n\n## Documentation\n\n- [Getting Started](https://docs.dagu.cloud/getting-started/quickstart) - Tutorial and first steps\n- [Core Concepts](https://docs.dagu.cloud/getting-started/concepts) - Architecture and design\n- [Writing Workflows](https://docs.dagu.cloud/writing-workflows/) - Complete authoring guide\n- [CLI Reference](https://docs.dagu.cloud/reference/cli) - Command-line usage\n- [API Reference](https://docs.dagu.cloud/reference/api) - REST API documentation\n- [Configuration](https://docs.dagu.cloud/configurations/server) - Configuration options\n- [Changelog](https://docs.dagu.cloud/reference/changelog) - Release notes and version history\n\n## Examples\n\nFind more in our [examples documentation](https://docs.dagu.cloud/writing-workflows/examples/).\n\n### ETL Pipeline\n```yaml\nname: daily-etl\nschedule: \"0 2 * * *\"\nsteps:\n  - name: extract\n    command: python extract.py\n    output: DATA_FILE\n    \n  - name: validate\n    command: python validate.py ${DATA_FILE}\n    \n  - name: transform\n    command: python transform.py ${DATA_FILE}\n    retryPolicy:\n      limit: 3\n      \n  - name: load\n    command: python load.py ${DATA_FILE}\n```\n\n### Hierarchical Workflows\n```yaml\nsteps:\n  - name: data-pipeline\n    run: etl\n    params: \"ENV=prod REGION=us-west-2\"\n    \n  - name: parallel-jobs\n    run: batch\n    parallel:\n      items: [\"job1\", \"job2\", \"job3\"]\n      maxConcurrency: 2\n    params: \"JOB=${ITEM}\"\n---\nname: etl\nparams:\n  - ENV\n  - REGION\nsteps:\n  - name: process\n    command: python etl.py --env ${ENV} --region ${REGION}\n---\nname: batch\nparams:\n  - JOB\nsteps:\n  - name: process\n    command: python process.py --job ${JOB}\n```\n\n### Container-based Pipeline\n```yaml\nname: ml-pipeline\nsteps:\n  - name: prepare-data\n    executor:\n      type: docker\n      config:\n        image: python:3.11\n        autoRemove: true\n        volumes:\n          - /data:/data\n    command: python prepare.py\n    \n  - name: train-model\n    executor:\n      type: docker\n      config:\n        image: tensorflow/tensorflow:latest-gpu\n    command: python train.py\n    \n  - name: deploy\n    command: kubectl apply -f model-deployment.yaml\n    preconditions:\n      - condition: \"`date +%u`\"\n        expected: \"re:[1-5]\"  # Weekdays only\n```\n\n## Web Interface\n\n[Learn more about the Web UI →](https://docs.dagu.cloud/overview/web-ui)\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/public/dashboard.png\" width=\"720\" alt=\"Dashboard\"\u003e\n  \u003cp\u003e\u003ci\u003eReal-time monitoring of all workflows\u003c/i\u003e\u003c/p\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/public/dag-editor.png\" width=\"720\" alt=\"DAG Editor\"\u003e\n  \u003cp\u003e\u003ci\u003eVisual workflow editor with validation\u003c/i\u003e\u003c/p\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/public/dag-logs.png\" width=\"720\" alt=\"Log Viewer\"\u003e\n  \u003cp\u003e\u003ci\u003eDetailed execution logs with stdout/stderr separation\u003c/i\u003e\u003c/p\u003e\n\u003c/div\u003e\n\n## Use Cases\n\n- **Data Engineering** - ETL pipelines, data validation, warehouse loading\n- **Machine Learning** - Training pipelines, model deployment, experiment tracking\n- **DevOps** - CI/CD workflows, infrastructure automation, deployment orchestration\n- **Media Processing** - Video transcoding, image manipulation, content pipelines\n- **Business Automation** - Report generation, data synchronization, scheduled tasks\n\n## Roadmap\n\n- [ ] Run steps in a DAG across multiple machines (distributed execution)\n- [ ] Track artifacts by dropping files in `$DAGU_ARTIFACTS`\n- [ ] Pause executions for webhooks, approvals, or any event (human-in-the-loop, event-driven workflows)\n- [ ] Integrate with AI agents via MCP (Model Context Protocol)\n\n## Building from Source\n\nPrerequisites: Go 1.24+, Node.js, pnpm\n\n```bash\ngit clone https://github.com/dagu-org/dagu.git \u0026\u0026 cd dagu\nmake build\nmake run\n```\n\n## Contributing\n\nContributions are welcome. See our [documentation](https://docs.dagu.cloud) for development setup.\n\n- [Discord Community](https://discord.gg/gpahPUjGRk)\n- [GitHub Issues](https://github.com/dagu-org/dagu/issues)\n\n## Contributors\n\n\u003ca href=\"https://github.com/dagu-org/dagu/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=dagu-org/dagu\" /\u003e\n\u003c/a\u003e\n\n## License\n\nGNU GPLv3 - See [LICENSE](./LICENSE)\n\n---\n\n\u003cdiv align=\"center\"\u003e\n  \u003cp\u003eIf you find Dagu useful, please ⭐ star this repository\u003c/p\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyohamta%2Fdagu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyohamta%2Fdagu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyohamta%2Fdagu/lists"}