{"id":18768720,"url":"https://github.com/get-convex/workflow","last_synced_at":"2025-04-13T07:30:52.089Z","repository":{"id":257788720,"uuid":"854826740","full_name":"get-convex/workflow","owner":"get-convex","description":"Convex component for durably executing workflows.","archived":false,"fork":false,"pushed_at":"2024-10-18T01:49:42.000Z","size":269,"stargazers_count":2,"open_issues_count":17,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-20T14:10:36.287Z","etag":null,"topics":["convex","determinism","durable-execution","workflow"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/get-convex.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-09T20:57:51.000Z","updated_at":"2024-10-18T01:49:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"94aa368b-4c1f-4906-ac4d-aa036d684ade","html_url":"https://github.com/get-convex/workflow","commit_stats":null,"previous_names":["get-convex/workflow"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/get-convex%2Fworkflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/get-convex%2Fworkflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/get-convex%2Fworkflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/get-convex%2Fworkflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/get-convex","download_url":"https://codeload.github.com/get-convex/workflow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223573807,"owners_count":17167369,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convex","determinism","durable-execution","workflow"],"created_at":"2024-11-07T19:13:42.172Z","updated_at":"2025-04-13T07:30:52.077Z","avatar_url":"https://github.com/get-convex.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Convex Workflow\n\n[![npm version](https://badge.fury.io/js/@convex-dev%2Fworkflow.svg?)](https://badge.fury.io/js/@convex-dev%2Fworkflow)\n\n\u003c!-- START: Include on https://convex.dev/components --\u003e\n\nHave you ever wanted to run a series of functions reliably and durably, where\neach can have its own retry behavior, the overall workflow will survive server\nrestarts, and you can have long-running workflows spanning months that can be\ncanceled? Do you want to observe the status of a workflow reactively, as well as\nthe results written from each step?\n\nAnd do you want to do this with code, instead of a DSL?\n\nWelcome to the world of Convex workflows.\n\n- Run workflows asynchronously, and observe their status reactively via\n  subscriptions, from one or many users simultaneously, even on page refreshes.\n- Workflows can run for months, and survive server restarts. You can specify\n  delays or custom times to run each step.\n- Run steps in parallel, or in sequence.\n- Output from previous steps is available to pass to subsequent steps.\n- Run queries, mutations, and actions.\n- Specify retry behavior on a per-step basis, along with a default policy.\n- Specify how many workflows can run in parallel to manage load.\n- Cancel long-running workflows.\n- Clean up workflows after they're done.\n\n```ts\nimport { WorkflowManager } from \"@convex-dev/workflow\";\nimport { components } from \"./_generated/api\";\n\nexport const workflow = new WorkflowManager(components.workflow);\n\nexport const exampleWorkflow = workflow.define({\n  args: {\n    storageId: v.id(\"_storage\"),\n  },\n  handler: async (step, args): Promise\u003cnumber[]\u003e =\u003e {\n    const transcription = await step.runAction(\n      internal.index.computeTranscription,\n      { storageId: args.storageId },\n    );\n\n    const embedding = await step.runAction(\n      internal.index.computeEmbedding,\n      { transcription },\n      // Run this a month after the transcription is computed.\n      { runAfter: 30 * 24 * 60 * 60 * 1000 },\n    );\n    return embedding;\n  },\n});\n```\n\nThis component adds durably executed _workflows_ to Convex. Combine Convex queries, mutations,\nand actions into long-lived workflows, and the system will always fully execute a workflow\nto completion.\n\nOpen a [GitHub issue](https://github.com/get-convex/workflow/issues) with any feedback or bugs you find.\n\n## Installation\n\nFirst, add `@convex-dev/workflow` to your Convex project:\n\n```sh\nnpm install @convex-dev/workflow\n```\n\nThen, install the component within your `convex/convex.config.ts` file:\n\n```ts\n// convex/convex.config.ts\nimport workflow from \"@convex-dev/workflow/convex.config\";\nimport { defineApp } from \"convex/server\";\n\nconst app = defineApp();\napp.use(workflow);\nexport default app;\n```\n\nFinally, create a workflow manager within your `convex/` folder, and point it\nto the installed component:\n\n```ts\n// convex/index.ts\nimport { WorkflowManager } from \"@convex-dev/workflow\";\nimport { components } from \"./_generated/api\";\n\nexport const workflow = new WorkflowManager(components.workflow);\n```\n\n## Usage\n\nThe first step is to define a workflow using `workflow.define()`. This function\nis designed to feel like a Convex action but with a few restrictions:\n\n1. The workflow runs in the background, so it can't return a value.\n2. The workflow must be _deterministic_, so it should implement most of its logic\n   by calling out to other Convex functions. We will be lifting some of these\n   restrictions over time by implementing `Math.random()`, `Date.now()`, and\n   `fetch` within our workflow environment.\n\nNote: To help avoid type cycles, always annotate the return type of the `handler`\nwith the return type of the workflow.\n\n```ts\nexport const exampleWorkflow = workflow.define({\n  args: { name: v.string() },\n  handler: async (step, args): Promise\u003cstring\u003e =\u003e {\n    const queryResult = await step.runQuery(\n      internal.example.exampleQuery,\n      args,\n    );\n    const actionResult = await step.runAction(\n      internal.example.exampleAction,\n      { queryResult }, // pass in results from previous steps!\n    );\n    return actionResult;\n  },\n});\n\nexport const exampleQuery = internalQuery({\n  args: { name: v.string() },\n  handler: async (ctx, args) =\u003e {\n    return `The query says... Hi ${args.name}!`;\n  },\n});\n\nexport const exampleAction = internalAction({\n  args: { queryResult: v.string() },\n  handler: async (ctx, args) =\u003e {\n    return args.queryResult + \" The action says... Hi back!\";\n  },\n});\n```\n\n### Starting a workflow\n\nOnce you've defined a workflow, you can start it from a mutation or action\nusing `workflow.start()`.\n\n```ts\nexport const kickoffWorkflow = mutation({\n  handler: async (ctx) =\u003e {\n    const workflowId = await workflow.start(\n      ctx,\n      internal.example.exampleWorkflow,\n      { name: \"James\" },\n    );\n  },\n});\n```\n\n### Handling the workflow's result with onComplete\n\nYou can handle the workflow's result with `onComplete`. This is useful for\ncleaning up any resources used by the workflow.\n\nNote: when you return things from a workflow, you'll need to specify the return\ntype of your `handler` to break type cycles due to using `internal.*` functions\nin the body, which then inform the type of the workflow, which is included in\nthe `internal.*` type.\n\nYou can also specify a `returns` validator to do runtime validation on the\nreturn value. If it fails, your `onComplete` handler will be called with an\nerror instead of success. You can also do validation in the `onComplete` handler\nto have more control over handling that situation.\n\n```ts\nimport { vWorkflowId } from \"@convex-dev/workflow\";\nimport { vResultValidator } from \"@convex-dev/workpool\";\n\nexport const foo = mutation({\n  handler: async (ctx) =\u003e {\n    const name = \"James\";\n    const workflowId = await workflow.start(\n      ctx,\n      internal.example.exampleWorkflow,\n      { name },\n      {\n        onComplete: internal.example.handleOnComplete,\n        context: name, // can be anything\n      },\n    );\n  },\n});\n\nexport const handleOnComplete = mutation({\n  args: {\n    workflowId: vWorkflowId,\n    result: vResultValidator,\n    context: v.any(), // used to pass through data from the start site.\n  }\n  handler: async (ctx, args) =\u003e {\n    const name = (args.context as { name: string }).name;\n    if (args.result.kind === \"success\") {\n      const text = args.result.returnValue;\n      console.log(`${name} result: ${text}`);\n    } else if (args.result.kind === \"error\") {\n      console.error(\"Workflow failed\", args.result.error);\n    } else if (args.result.kind === \"canceled\") {\n      console.log(\"Workflow canceled\", args.context);\n    }\n  },\n});\n```\n\n### Running steps in parallel\n\nYou can run steps in parallel by calling `step.runAction()` multiple times in\na `Promise.all()` call.\n\n```ts\nexport const exampleWorkflow = workflow.define({\n  args: { name: v.string() },\n  handler: async (step, args): Promise\u003cvoid\u003e =\u003e {\n    const [result1, result2] = await Promise.all([\n      step.runAction(internal.example.myAction, args),\n      step.runAction(internal.example.myAction, args),\n    ]);\n  },\n});\n```\n\nNote: The workflow will not proceed until all steps fired off at once have completed.\n\n### Specifying retry behavior\n\nSometimes actions fail due to transient errors, whether it was an unreliable\nthird-party API or a server restart. You can have the workflow automatically\nretry actions using best practices (exponential backoff \u0026 jitter).\nBy default there are no retries, and the workflow will fail.\n\nYou can specify default retry behavior for all workflows on the WorkflowManager,\nor override it on a per-workflow basis.\n\nYou can also specify a custom retry behavior per-step, to opt-out of retries\nfor actions that may want at-most-once semantics.\n\nWorkpool options:\n\nIf you specify any of these, it will override the\n[`DEFAULT_RETRY_BEHAVIOR`](./src/component/pool.ts).\n\n- `defaultRetryBehavior`: The default retry behavior for all workflows.\n  - `maxAttempts`: The maximum number of attempts to retry an action.\n  - `initialBackoffMs`: The initial backoff time in milliseconds.\n  - `base`: The base multiplier for the backoff. Default is 2.\n- `retryActionsByDefault`: Whether to retry actions, by default is false.\n  - If you specify a retry behavior at the step level, it will always retry.\n\nAt the step level, you can also specify `true` or `false` to disable or use\nthe default policy.\n\n```ts\nconst workflow = new WorkflowManager(components.workflow, {\n  defaultRetryBehavior: {\n    maxAttempts: 3,\n    initialBackoffMs: 100,\n    base: 2,\n  },\n  // If specified, this sets the defaults, overridden per-workflow or per-step.\n  workpoolOptions: { ... }\n});\n\nexport const exampleWorkflow = workflow.define({\n  args: { name: v.string() },\n  handler: async (step, args): Promise\u003cvoid\u003e =\u003e {\n    // Uses default retry behavior \u0026 retryActionsByDefault\n    await step.runAction(internal.example.myAction, args);\n    // Retries will be attempted with the default behavior\n    await step.runAction(internal.example.myAction, args, { retry: true });\n    // No retries will be attempted\n    await step.runAction(internal.example.myAction, args, { retry: false });\n    // Custom retry behavior will be used\n    await step.runAction(internal.example.myAction, args, {\n      retry: { maxAttempts: 2, initialBackoffMs: 100, base: 2 },\n    });\n  },\n  // If specified, this will override the workflow manager's default\n  workpoolOptions: { ... },\n});\n```\n\n### Specifying how many workflows can run in parallel\n\nYou can specify how many workflows can run in parallel by setting the `maxParallelism`\nworkpool option. It has a reasonable default.\n\n```ts\nconst workflow = new WorkflowManager(components.workflow, {\n  workpoolOptions: {\n    // You must only set this to one value per components.xyz!\n    // You can set different values if you \"use\" multiple different components\n    // in convex.config.ts.\n    maxParallelism: 10,\n  },\n});\n```\n\n### Checking a workflow's status\n\nThe `workflow.start()` method returns a `WorkflowId`, which can then be used for querying\na workflow's status.\n\n```ts\nexport const kickoffWorkflow = action({\n  handler: async (ctx) =\u003e {\n    const workflowId = await workflow.start(\n      ctx,\n      internal.example.exampleWorkflow,\n      { name: \"James\" },\n    );\n    await new Promise((resolve) =\u003e setTimeout(resolve, 1000));\n\n    const status = await workflow.status(ctx, workflowId);\n    console.log(\"Workflow status after 1s\", status);\n  },\n});\n```\n\n### Canceling a workflow\n\nYou can cancel a workflow with `workflow.cancel()`, halting the workflow's execution immmediately.\nIn-progress calls to `step.runAction()`, however, will finish executing.\n\n```ts\nexport const kickoffWorkflow = action({\n  handler: async (ctx) =\u003e {\n    const workflowId = await workflow.start(\n      ctx,\n      internal.example.exampleWorkflow,\n      { name: \"James\" },\n    );\n    await new Promise((resolve) =\u003e setTimeout(resolve, 1000));\n\n    // Cancel the workflow after 1 second.\n    await workflow.cancel(ctx, workflowId);\n  },\n});\n```\n\n### Cleaning up a workflow\n\nAfter a workflow has completed, you can clean up its storage with `workflow.cleanup()`.\nCompleted workflows are not automatically cleaned up by the system.\n\n```ts\nexport const kickoffWorkflow = action({\n  handler: async (ctx) =\u003e {\n    const workflowId = await workflow.start(\n      ctx,\n      internal.example.exampleWorkflow,\n      { name: \"James\" },\n    );\n    try {\n      while (true) {\n        const status = await workflow.status(ctx, workflowId);\n        if (status.type === \"inProgress\") {\n          await new Promise((resolve) =\u003e setTimeout(resolve, 1000));\n          continue;\n        }\n        console.log(\"Workflow completed with status:\", status);\n        break;\n      }\n    } finally {\n      await workflow.cleanup(ctx, workflowId);\n    }\n  },\n});\n```\n\n### Specifying a custom name for a step\n\nYou can specify a custom name for a step by passing a `name` option to the step.\n\nThis allows the events emitted to your logs to be more descriptive.\nBy default it uses the `file/folder:function` name.\n\n```ts\nexport const exampleWorkflow = workflow.define({\n  args: { name: v.string() },\n  handler: async (step, args): Promise\u003cvoid\u003e =\u003e {\n    await step.runAction(internal.example.myAction, args, { name: \"FOO\" });\n  },\n});\n```\n\n## Tips and troubleshooting\n\n### Circular dependencies\n\nHaving the return value of workflows depend on other Convex functions can lead to circular dependencies due to the\n`internal.foo.bar` way of specifying functions. The way to fix this is to explicitly type the return value of the\nworkflow. When in doubt, add return types to more `handler` functions, like this:\n\n```diff\n export const supportAgentWorkflow = workflow.define({\n   args: { prompt: v.string(), userId: v.string(), threadId: v.string() },\n+  handler: async (step, { prompt, userId, threadId }): Promise\u003cstring\u003e =\u003e {\n     // ...\n   },\n });\n\n // And regular functions too:\n export const myFunction = action({\n   args: { prompt: v.string() },\n+  handler: async (ctx, { prompt }): Promise\u003cstring\u003e =\u003e {\n     // ...\n   },\n });\n```\n\n### More concise workflows\n\nTo avoid the noise of `internal.foo.*` syntax, you can use a variable.\nFor instance, if you define all your steps in `convex/steps.ts`, you can do this:\n\n```diff\n const s = internal.steps;\n\n export const myWorkflow = workflow.define({\n   args: { prompt: v.string() },\n   handler: async (step, args): Promise\u003cstring\u003e =\u003e {\n+    const result = await step.runAction(s.myAction, args);\n     return result;\n   },\n });\n```\n\n## Limitations\n\nHere are a few limitations to keep in mind:\n\n- Steps can only take in and return a total of _1 MiB_ of data within a single\n  workflow execution. If you run into journal size limits, you can work around\n  this by storing results in the DB from your step functions and passing IDs\n  around within the the workflow.\n- `console.log()` isn't currently captured, so you may see duplicate log lines\n  within your Convex dashboard if you log within the workflow definition.\n- We currently do not collect backtraces from within function calls from workflows.\n- If you need to use side effects like `fetch`, `Math.random()`, or `Date.now()`,\n  you'll need to do that in a step, not in the workflow definition.\n- If the implementation of the workflow meaningfully changes (steps added,\n  removed, or reordered) then it will fail with a determinism violation.\n  The implementation should stay stable for the lifetime of active workflows.\n  See [this issue](https://github.com/get-convex/workflow/issues/35) for ideas\n  on how to make this better.\n\n\u003c!-- END: Include on https://convex.dev/components --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fget-convex%2Fworkflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fget-convex%2Fworkflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fget-convex%2Fworkflow/lists"}