{"id":13542817,"url":"https://github.com/hatchet-dev/hatchet-workflows","last_synced_at":"2025-04-02T11:30:58.845Z","repository":{"id":205651360,"uuid":"714722233","full_name":"hatchet-dev/hatchet-workflows","owner":"hatchet-dev","description":"Declarative workflows for Go apps, for background task processing and event-driven architectures. ","archived":true,"fork":false,"pushed_at":"2024-06-13T15:38:22.000Z","size":135,"stargazers_count":26,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-03T09:33:29.178Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hatchet-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-05T17:26:20.000Z","updated_at":"2024-06-13T15:38:38.000Z","dependencies_parsed_at":"2023-11-12T17:24:31.540Z","dependency_job_id":"9cf34389-d363-456e-a2f3-cf082bbe063d","html_url":"https://github.com/hatchet-dev/hatchet-workflows","commit_stats":null,"previous_names":["hatchet-dev/hatchet-workflows"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hatchet-dev%2Fhatchet-workflows","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hatchet-dev%2Fhatchet-workflows/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hatchet-dev%2Fhatchet-workflows/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hatchet-dev%2Fhatchet-workflows/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hatchet-dev","download_url":"https://codeload.github.com/hatchet-dev/hatchet-workflows/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246806514,"owners_count":20837112,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T11:00:18.232Z","updated_at":"2025-04-02T11:30:58.401Z","avatar_url":"https://github.com/hatchet-dev.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":["Libraries"],"readme":"**Note:** the Hatchet project has been moved [here](https://github.com/hatchet-dev/hatchet).\n\n\u003cdetails\u003e\n\u003csummary\u003eArchived description\u003c/summary\u003e\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-purple.svg)](https://opensource.org/licenses/MIT) [![Go Reference](https://pkg.go.dev/badge/github.com/hatchet-dev/hatchet-workflows.svg)](https://pkg.go.dev/github.com/hatchet-dev/hatchet-workflows)\n\n\n\n## Introduction\n\n_**Note:** Hatchet workflows are in early development. Changes are not guaranteed to be backwards-compatible. If you'd like to run them in production, feel free to reach out on Slack for tips._\n\nHatchet is a declarative workflow builder for Golang apps. Using Hatchet, you can create workers which process a set of background tasks based on different triggers, using a declarative file syntax that draws heavy inspiration from Github actions. Unlike Github actions, code runs inside your Go application, with triggers and actions that you've defined.\n\nAs a simple example, let's say you want to perform 3 actions when a user has signed up for your app:\n\n1. Initialize a set of resources for the user (perhaps a sandbox environment for testing).\n2. Send the user an automated greeting over email\n3. Add the user to a newsletter campaign\n\nWith Hatchet workflows, this would look something like the following:\n\n```yaml\nname: \"Post User Sign Up\"\non:\n  - user:create\njobs:\n  create-resources:\n    steps:\n      - name: Create sandbox environment\n        id: createSandbox\n        actionId: sandbox:create\n        timeout: 60s\n  greet-user:\n    steps:\n      - name: Greet user\n        id: greetUser\n        actionId: postmark:email-from-template\n        timeout: 15s\n        with:\n          firstName: \"{{ .user.firstName }}\"\n          email: \"{{ .user.email }}\"\n  add-to-newsletter:\n    steps:\n      - name: Add to newsletter\n        id: addUserToNewsletter\n        actionId: newsletter:add-user\n        timeout: 15s\n        with:\n          email: \"{{ .user.email }}\"\n```\n\nIn your codebase, you would then create the following integrations (see [Writing an integration](#writing-an-integration)):\n\n- A `sandbox` integration responsible for creating/tearing down a sandbox environment\n- A `postmark` integration for sending an email from a template\n- A `newsletter` integration for adding a user to a newsletter campaign\n\nUltimately, the goal of Hatchet workflows are that you don't need to write these integrations yourself -- creating a robust set of prebuilt integrations is one of the goals of the project.\n\n### Why is this useful?\n\n- No need to build all of your plumbing logic (action 1 -\u003e event 1 -\u003e action 2 -\u003e event 2). Just define your jobs and steps and write your business logic. This is particularly useful the more complex your workflows become.\n- Using prebuilt integrations with a standard interface makes building auxiliary services like notification systems, billing, backups, and auditing much easier. **Please file an issue if you'd like to see an integration supported.** The following are on the roadmap:\n  - Email providers: Sendgrid, Postmark, AWS SES\n  - Stripe\n  - AWS S3\n- Additionally, if you're already familiar with/using Temporal, making workflows declarative provides several benefits:\n  - Makes spec'ing, debugging and visualizing workflows much simpler\n  - Automatically updates triggers, schedules, and timeouts when they change, rather than doing this through a UI/CLI/SDK\n  - Makes monitoring easier to build by logically separating units of work - jobs will automatically correspond to `BeginSpan`. OpenTelemetry support is on the roadmap.\n\n## Getting Started\n\nFor a set of end-to-end examples, see the [examples](./examples) directory.\n\n### Prerequisites\n\n- Go 1.21 installed\n- Taskfile installed (instructions [here](https://taskfile.dev/installation/))\n\n### Setting up Temporal\n\nFirst, you need to get a Temporal cluster running. There are many ways to do this: see [here](https://docs.temporal.io/kb/all-the-ways-to-run-a-cluster) for all options.\n\nTo make things easier, there's a bundled server and UI in `./cmd/temporal-server`. Run the following commands to get it working:\n\n```sh\ntask write-default-env\ntask generate-certs\ntask start-temporal-server\n```\n\nYou can then navigate to 127.0.0.1:8233 to view the Temporal UI.\n\n### Writing an Integration\n\nAn integration needs to satisfy the following interface:\n\n```go\ntype Integration interface {\n\tGetId() string\n\tActions() []string\n\tPerformAction(action types.Action, data map[string]interface{}) (map[string]interface{}, error)\n}\n```\n\nSee the [Slack integration](./pkg/integrations/slack) for an example.\n\n### Writing a Workflow\n\nBy default, Hatchet searches for workflows in the `.hatchet` folder relative to the directory you run your application in. However, you can configure this using `worker.WithWorkflowFiles` and the exported `fileutils` package (`fileutils.ReadAllValidFilesInDir`).\n\nThere are two main sections of a workflow file:\n\n**Triggers (using `on`)**\n\nThis section specifies what triggers a workflow. This can be events or a crontab-like schedule. For example, the following are valid triggers:\n\n```yaml\non:\n  - eventkey1\n  - eventkey2\n```\n\n```yaml\non:\n  cron:\n    schedule: \"*/15 * * * *\"\n```\n\nThere are also a set of keywords `random_15_min`, `random_hourly`, `random_daily` for cron-like schedules. Upon creation of these schedules, a random minute is picked in the given interval - for example, `random_hourly` might result in a schedule `49 * * * *` (the 49th minute of every hour). After creation, these schedules will **not** be updated with a new random schedule.\n\n```yaml\non:\n  cron:\n    schedule: \"random_hourly\"\n```\n\nThe point of this is to avoid burstiness if all jobs have the exact same schedule (i.e. runs at the 0th minute of every hour), you may start to run out of memory on your workers.\n\n**Jobs**\n\nAfter defining your triggers, you define a list of jobs to run based on the triggers. **Jobs run in parallel.** Jobs contain the following fields:\n\n```yaml\n# ...\njobs:\n  my-awesome-job:\n    # (optional) A queue name\n    queue: internal\n    # (optional) A timeout value for the entire job\n    timeout: 60s\n    # (required) A set of steps for the job; see below\n    steps: []\n```\n\nWithin each job, there are a set of **steps** which run sequentially. A step can contain the following fields:\n\n```yaml\n# (required) the name of the step\nname: Step 1\n# (required) a unique id for the step (can be referenced by future steps)\nid: step-1\n# (required) the action id in the form of \"integration_id:action\".\nactionId: \"slack:create-channel\"\n# (required) the timeout of the individual step\ntimeout: 15s\n# (optional or required, depending on integration) input data to the integration\nwith:\n  key: val\n```\n\n### Creating a Worker\n\nWorkers can be created using:\n\n```go\nimport \"github.com/hatchet-dev/hatchet-workflows/pkg/worker\"\n\nfunc main() {\n  // ... application code\n  worker, err := worker.NewWorker(\n    worker.WithIntegrations(\n      myIntegration,\n    ),\n  )\n\n  if err != nil {\n    // TODO: error handling here\n    panic(err)\n  }\n\n  // Start worker in non-blocking fashion\n  worker.Start()\n\n  // Start worker in blocking fashion\n  worker.Run()\n}\n```\n\nYou can configure the worker with your own set of workflow files using the `worker.WithWorkflowFiles` option.\n\n### Triggering Events\n\nTo trigger events from your main application, use the `dispatcher` package:\n\n```go\nimport \"github.com/hatchet-dev/hatchet-workflows/pkg/worker\"\n\nfunc main() {\n  d := dispatcher.NewDispatcher()\n\n  // this would typically be called within a handler\n\terr = d.Trigger(\"user:create\", map[string]any{\n\t\t\"username\": \"testing12345\",\n\t})\n\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n```\n\nYou can configure the dispatcher with your own set of workflow files using the `dispatcher.WithWorkflowFiles` option.\n\n## Why should I care?\n\n**If you're unfamiliar with background task processing**\n\nMany Go APIs start out without a task processing/worker service. You might not need it, but at a certain level of complexity, you probably will. There are a few use-cases where workers start to make sense:\n\n1. You need to run scheduled tasks which that aren't triggered from your core API. For example, this may be a daily cleanup task, like traversing soft-deleted database entries or backing up data to S3.\n2. You need to run tasks which are triggered by API events, but aren't required for the core business logic of the handler. For example, you want to add a user to your CRM after they sign up.\n\nFor both of these cases, it's typical to re-use a lot of core functionality from your API, so the most natural place to start is by adding some automation within your API itself; for example, after returning `201 Created`, you might send a greeting to the user, initialize a sandbox environment, send an internal notification that a user signed up, etc, all within your API handlers. Let's say you've handled this case as following:\n\n```go\n// Hypothetical handler called via a routing package, let's just pretend it returns an error\nfunc MyHandler(ctx context.Context, w http.ResponseWriter, r *http.Request) error {\n    // Boilerplate code to parse the request\n    var newUser User\n    err := json.NewDecoder(r.Body).Decode(\u0026newUser)\n    if err != nil {\n      http.Error(w, \"Invalid user data\", http.StatusBadRequest)\n      return err\n    }\n\n    // Validate email and password fields...\n    // (Add your validation logic here)\n\n    // Create a user in the database\n    user, err := createUser(ctx, newUser.Email, newUser.Password)\n    if err != nil {\n      // Handle database errors, such as unique constraint violation\n      http.Error(w, \"Error creating user\", http.StatusInternalServerError)\n      return err\n    }\n\n    // Return 201 created with user type\n    w.Header().Set(\"Content-Type\", \"application/json\")\n    w.WriteHeader(http.StatusCreated)\n\n    // Send user a greeting\n    err := email.SendGreetingEmail(context.Background(), user)\n\n    if err != nil {\n      // can't return an error, since header is already set\n      fmt.Println(err)\n    }\n\n    // ... other post-signup operations\n}\n```\n\nAt some point, you realize all of these background operations don't really belong in the handler -- when they're part of the handler, they're more difficult to monitor and observe, difficult to retry (especially if a third-party service goes down), and bloat your handlers (which could cause goroutine leakage or memory issues).\n\nThis is where a service (like [Temporal](https://github.com/temporalio/temporal)) suited for background/task processing comes in.\n\n**If you're familiar with/already using Temporal**\n\nIf you're familiar with Temporal, Hatchet utilizes Temporal as a backend for processing workflows and activities, and adds a set of prebuilt workflows and utilities to make Temporal easier to use. For an understanding of how Hatchet works:\n\n- Each Hatchet job corresponds to a different Temporal workflow\n- Each step in a job corresponds to a Temporal activity\n\nHatchet is compatible with both Temporal Cloud and self-hosted versions of Temporal.\n\n## I'd Like to Contribute\n\nHatchet is still in very early development -- as a result, there are no development docs. However, please feel free to reach out on the #contributing channel on [Slack](https://join.slack.com/t/hatchet-co/signup) to shape the direction of the project.\n\n\u003c/details\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhatchet-dev%2Fhatchet-workflows","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhatchet-dev%2Fhatchet-workflows","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhatchet-dev%2Fhatchet-workflows/lists"}