{"id":21062439,"url":"https://github.com/rnd-soft/burstflow","last_synced_at":"2025-03-14T01:14:22.949Z","repository":{"id":62554740,"uuid":"163874670","full_name":"RND-SOFT/burstflow","owner":"RND-SOFT","description":"ActiveJob based workflow management","archived":false,"fork":false,"pushed_at":"2019-02-10T09:01:24.000Z","size":113,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-03-14T20:22:05.672Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RND-SOFT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-02T18:20:09.000Z","updated_at":"2019-02-10T09:01:26.000Z","dependencies_parsed_at":"2022-11-03T05:15:37.516Z","dependency_job_id":null,"html_url":"https://github.com/RND-SOFT/burstflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RND-SOFT%2Fburstflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RND-SOFT%2Fburstflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RND-SOFT%2Fburstflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RND-SOFT%2Fburstflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RND-SOFT","download_url":"https://codeload.github.com/RND-SOFT/burstflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243505963,"owners_count":20301619,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T17:38:52.576Z","updated_at":"2025-03-14T01:14:22.925Z","avatar_url":"https://github.com/RND-SOFT.png","language":"Ruby","readme":"\u003c!-- # Gush [![Build Status](https://travis-ci.org/chaps-io/gush.svg?branch=master)](https://travis-ci.org/chaps-io/gush) --\u003e\n\n\u003c!-- ## [![](http://i.imgur.com/ya8Wnyl.png)](https://chaps.io) proudly made by [Chaps](https://chaps.io) --\u003e\n\n[![Build Status](https://travis-ci.com/RnD-Soft/burstflow.svg?branch=master)](https://travis-ci.com/RnD-Soft/burstflow) [![Gem Version](https://badge.fury.io/rb/burstflow.svg)](https://badge.fury.io/rb/burstflow)\n\nBurstflow is a parallel workflow runner using [ActiveRecord] and [ActiveJob](https://guides.rubyonrails.org/v4.2/active_job_basics.html) for scheduling and executing jobs.\n\nThis gem is higly inspired by [Gush](https://github.com/chaps-io/gush). But not tied to Reddis or Sidekiq like Gush.\n\nActiveRecord used as persisted store during workflow execution.\n\nAdditional features:\n\n* **susped** and **resume** job(and whole workflow). For example if your job makes asynchronous request and will receive a response some time later. In this case, the job can send request and suspend until some external event resumes it eventually.\n* before/after callbacks on jobs level\n* before/after callbacks on workflow level\n* **Dynamic workflows**. Any job can produce another jobs while executing. This jobs has its parent as incomming jobs, and all parents outgoing jobs as own outgoings.\n\n\n## Installation\n\n### 1. Add `burst` to Gemfile\n\n```ruby\ngem 'burstflow', '~\u003e 0.2.0'\n```\n\n### 2. Run migrations\n\n```rake burstflow:install:migrations```\n\n## Example\n\nThe DSL for defining jobs consists of a single `run` method.\nHere is a complete example of a workflow you can create:\n\n```ruby\n# app/workflows/sample_workflow.rb\nclass SampleWorkflow \u003c Burstflow::Workflow\n  configure do |url_to_fetch_from|\n    run FetchJob1, params: { url: url_to_fetch_from }\n    run FetchJob2, params: { some_flag: true, url: 'http://url.com' }\n\n    run PersistJob1, after: FetchJob1\n    run PersistJob2, after: FetchJob2\n\n    run Normalize,\n        after: [PersistJob1, PersistJob2],\n        before: Index\n\n    run Index\n  end\nend\n```\n\nand this is how the graph will look like:\n\n![SampleWorkflow](https://i.imgur.com/DFh6j51.png)\n\n\n## Defining workflows\n\nLet's start with the simplest workflow possible, consisting of a single job:\n\n```ruby\nclass SimpleWorkflow \u003c Burstflow::Workflow\n  configure do \n    run DownloadJob\n  end\nend\n```\n\nOf course having a workflow with only a single job does not make sense, so it's time to define dependencies:\n\n```ruby\nclass SimpleWorkflow \u003c Burst::Workflow\n  configure do\n    run DownloadJob\n    run SaveJob, after: DownloadJob\n  end\nend\n```\n\nWe just told Burstflow to execute `SaveJob` right after `DownloadJob` finishes **successfully**.\n\nBut what if your job must have multiple dependencies? That's easy, just provide an array to the `after` attribute:\n\n```ruby\nclass SimpleWorkflow \u003c Burstflow::Workflow\n  configure do\n    run FirstDownloadJob\n    run SecondDownloadJob\n\n    run SaveJob, after: [FirstDownloadJob, SecondDownloadJob]\n  end\nend\n```\n\nNow `SaveJob` will only execute after both its parents finish without errors.\n\nWith this simple syntax you can build any complex workflows you can imagine!\n\n#### Alternative way\n\n`run` method also accepts `before:` attribute to define the opposite association. So we can write the same workflow as above, but like this:\n\n```ruby\nclass SimpleWorkflow \u003c Burstflow::Workflow\n  configure do\n    run FirstDownloadJob, before: SaveJob\n    run SecondDownloadJob, before: SaveJob\n\n    run SaveJob\n  end\nend\n```\n\nYou can use whatever way you find more readable or even both at once :)\n\n### Passing arguments to workflows\n\nWorkflows can accept any primitive arguments in their constructor, which then will be available in your\n`configure` method.\n\nLet's assume we are writing a book publishing workflow which needs to know where the PDF of the book is and under what ISBN it will be released:\n\n```ruby\nclass PublishBookWorkflow \u003c Burstflow::Workflow\n  configure do |url, isbn|\n    run FetchBook, params: { url: url }\n    run PublishBook, params: { book_isbn: isbn }, after: FetchBook\n  end\nend\n```\n\nand then create your workflow with those arguments:\n\n```ruby\nPublishBookWorkflow.build(\"http://url.com/book.pdf\", \"978-0470081204\")\n```\n\nand that's basically it for defining workflows, see below on how to define jobs:\n\n## Defining jobs\n\nThe simplest job is a class inheriting from `Burstflow::Job` and responding to `perform` and `resume` method. Much like any other ActiveJob class.\n\n```ruby\nclass FetchBook \u003c Burst::Job\n  def perform\n    # do some fetching from remote APIs\n  end\nend\n```\n\nBut what about those params we passed in the previous step?\n\n## Passing parameters into jobs\n\nTo do that, simply provide a `params:` attribute with a hash of parameters you'd like to have available inside the `perform` method of the job.\n\nSo, inside workflow:\n\n```ruby\n(...)\nrun FetchBook, params: {url: \"http://url.com/book.pdf\"}\n(...)\n```\n\nand within the job we can access them like this:\n\n```ruby\nclass FetchBook \u003c Burst::Job\n  def perform\n    # you can access `params` method here, for example:\n\n    params #=\u003e {url: \"http://url.com/book.pdf\"}\n  end\nend\n```\n\n## Executing workflows\n\nWorkflows are executed by any backend you chose for ActiveJob.\n\n\n### 1. Create the workflow instance\n\n```ruby\nflow = PublishBookWorkflow.build(\"http://url.com/book.pdf\", \"978-0470081204\")\n```\n\n### 2. Start the workflow\n\n```ruby\nflow.start!\n```\n\nNow Burstflow will start processing jobs in the background using ActiveJob and your chosen backend.\n\n### 3. Monitor its progress:\n\n```ruby\nflow.reload\nflow.status\n#=\u003e :running|:finished|:failed|:suspended\n```\n\n`reload` is needed to see the latest status, since workflows are updated asynchronously.\n\n## Advanced features\n\n### Pipelining\n\nBurstflow offers a useful tool to pass results of a job to its dependencies, so they can act differently.\n\n**Example:**\n\nLet's assume you have two jobs, `DownloadVideo`, `EncodeVideo`.\nThe latter needs to know where the first one saved the file to be able to open it.\n\n\n```ruby\nclass DownloadVideo \u003c Burst::Job\n  def perform\n    downloader = VideoDownloader.fetch(\"http://youtube.com/?v=someytvideo\")\n\n    output(downloader.file_path)\n  end\nend\n```\n\n`output` method is used to ouput data from the job to all dependant jobs.\n\nNow, since `DownloadVideo` finished and its dependant job `EncodeVideo` started, we can access that payload inside it:\n\n```ruby\nclass EncodeVideo \u003c Burstflow::Job\n  def perform\n    video_path = payloads.first[:value]\n  end\nend\n```\n\n`payloads` is an array containing outputs from all ancestor jobs. So for our `EncodeVide` job from above, the array will look like:\n\n\n```ruby\n[\n  {\n    id: \"DownloadVideo-41bfb730-b49f-42ac-a808-156327989294\" # unique id of the ancestor job\n    class: \"DownloadVideo\",\n    value: \"https://s3.amazonaws.com/somebucket/downloaded-file.mp4\" #the payload returned by DownloadVideo job using `output()` method\n  }\n]\n```\n\n**Note:** Keep in mind that payloads can only contain data which **can be serialized as JSON**.\n\n### Dynamic workflows\n\nThere might be a case when you have to construct the workflow dynamically depending on the input.\n\nAs an example, let's write a workflow which accepts an array of users and has to send an email to each one. Additionally after it sends the e-mail to every user, it also has to notify the admin about finishing.\n\n\n```ruby\n\nclass ParentJob \u003c Burstflow::Job\n  def perform\n    configure do \n      params[:user_ids].map do |user_id|\n        run NotificationJob, params: {user_id: user_id}\n      end\n    end\n  end\nend\n\nclass NotifyWorkflow \u003c Burst::Workflow\n  configure do |user_ids|\n    run ParentJob, params: {user_ids: user_ids}\n\n    run AdminNotificationJob, after: ParentJob\n  end\nend\n```\n\n\n\n## Original Gush Contributors\n\nhttps://github.com/chaps-io/gush#contributors\n\n## Contributing\n\n1. Fork it ( https://github.com/RnD-Soft/burstflow/fork )\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frnd-soft%2Fburstflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frnd-soft%2Fburstflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frnd-soft%2Fburstflow/lists"}