{"id":33916309,"url":"https://github.com/ductwork/ductwork","last_synced_at":"2026-04-06T07:01:55.522Z","repository":{"id":324274718,"uuid":"889158441","full_name":"ductwork/ductwork","owner":"ductwork","description":"A Ruby pipeline and workflow framework","archived":false,"fork":false,"pushed_at":"2026-04-03T18:29:13.000Z","size":1288,"stargazers_count":67,"open_issues_count":1,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-04-03T19:41:30.174Z","etag":null,"topics":["pipelines","rails","ruby","workflows"],"latest_commit_sha":null,"homepage":"https://www.getductwork.io/","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ductwork.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG-PRO.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-11-15T18:13:26.000Z","updated_at":"2026-04-03T18:29:20.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ductwork/ductwork","commit_stats":null,"previous_names":["ductwork/ductwork"],"tags_count":41,"template":false,"template_full_name":null,"purl":"pkg:github/ductwork/ductwork","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ductwork%2Fductwork","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ductwork%2Fductwork/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ductwork%2Fductwork/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ductwork%2Fductwork/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ductwork","download_url":"https://codeload.github.com/ductwork/ductwork/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ductwork%2Fductwork/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31463015,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T21:22:52.476Z","status":"online","status_checked_at":"2026-04-06T02:00:07.287Z","response_time":112,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["pipelines","rails","ruby","workflows"],"created_at":"2025-12-12T07:16:37.529Z","updated_at":"2026-04-06T07:01:55.509Z","avatar_url":"https://github.com/ductwork.png","language":"Ruby","readme":"# Ductwork\n\n[![CI](https://github.com/ductwork/ductwork/actions/workflows/main.yml/badge.svg)](https://github.com/ductwork/ductwork/actions/workflows/main.yml)\n[![Gem Version](https://badge.fury.io/rb/ductwork.svg?icon=si%3Arubygems)](https://rubygems.org/gems/ductwork)\n\nA Ruby pipeline and workflow framework.\n\nDuctwork lets you build complex pipelines quickly and easily using intuitive Ruby tooling and a natural DSL. No need to learn complicated unified object models or stand up separate runner instances—just write Ruby code and let Ductwork handle the orchestration.\n\nThere is also a paid [Ductwork Pro](https://www.getductwork.io/) version with more features and support. See the [Pricing](https://www.getductwork.io/#pricing) page to buy a license.\n\n**[Full Documentation](https://docs.getductwork.io/)**\n\n## Installation\n\nAdd Ductwork to your application's Gemfile:\n\n```bash\nbundle add ductwork\n```\n\nRun the Rails generator to create the binstub, configuration file, and migrations:\n\n```bash\nbin/rails generate ductwork:install\n```\n\n**NOTE**: run the update generator if you've already installed ductwork to get updates:\n\n```bash\nbin/rails generate ductwork:update\n```\n\nRun migrations and you're ready to start building pipelines!\n\n## Configuration\n\n\nThe only required configuration is specifying which pipelines to run. Edit the default configuration file `config/ductwork.yml`:\n\n```yaml\ndefault: \u0026default\n  pipelines:\n    - EnrichUserDataPipeline\n    - SendMonthlyStatusReportsPipeline\n```\n\nOr use the wildcard to run all pipelines (use cautiously—this can consume significant resources):\n\n```yaml\ndefault: \u0026default\n  pipelines: \"*\"\n```\n\nSee the [Configuration Guide](https://docs.getductwork.io/getting-started/configuration.html) for all available options including thread counts, timeouts, and database settings.\n\n## Usage\n\n### 1. Create a Pipeline Class\n\nPipeline classes live in `app/pipelines` and inherit from `Ductwork::Pipeline`. While the \"Pipeline\" suffix is optional, it can help avoid naming collisions:\n\n```ruby\n# app/pipelines/enrich_user_data_pipeline.rb\nclass EnrichUserDataPipeline \u003c Ductwork::Pipeline\nend\n```\n\n### 2. Define Steps\n\nSteps are Ruby objects that inherit from `Ductwork::Step` and implement two methods:\n- `initialize` - accepts parameters from the trigger call or previous step's return value\n- `execute` - performs the work and returns data for the next step\n\nSteps live in `app/steps`:\n\n```ruby\n# app/steps/users_requiring_enrichment.rb\nclass QueryUsersRequiringEnrichment \u003c Ductwork::Step\n  def initialize(days_outdated)\n    @days_outdated = days_outdated\n  end\n\n  def execute\n    ids = User.where(\"data_last_refreshed_at \u003c ?\", @days_outdated.days.ago).ids\n    Ductwork.logger.info(\"Enriching #{ids.length} users' data\")\n\n    # Return value becomes input to the next step\n    ids\n  end\nend\n```\n\n### 3. Define Transitions\n\nConnect steps together using Ductwork's fluent interface DSL. The key principle: **each step's return value becomes the next step's input**.\n\n```ruby\nclass EnrichUserDataPipeline \u003c Ductwork::Pipeline\n  define do |pipeline|\n    pipeline.start(QueryUsersRequiringEnrichment)  # Start with a single step\n            .expand(to: LoadUserData)              # Fan out to multiple steps\n            .divide(to: [FetchDataFromSourceA,     # Split into parallel branches\n                         FetchDataFromSourceB])\n            .combine(into: CollateUserData)        # Merge branches back together\n            .chain(to: UpdateUserData)             # Sequential processing\n            .collapse(into: ReportSuccess)         # Gather expanded steps\n  end\nend\n```\n\n**Important:** Return values must be JSON-serializable.\n\nSee [Defining Pipelines](https://docs.getductwork.io/getting-started/defining-pipelines.html) for detailed documentation.\n\n### 4. Run Ductwork\n\nStart the Ductwork supervisor, which manages pipeline advancers and job workers for each configured pipeline:\n\n```bash\nbin/ductwork\n```\n\nUse a custom configuration file if needed:\n\n```bash\nbin/ductwork -c config/ductwork.0.yml\n```\n\n### 5. Trigger Your Pipeline\n\nTrigger pipelines from anywhere in your Rails application. The `trigger` method returns a `Ductwork::Pipeline` instance for monitoring:\n\n```ruby\n# In a Rake task\ntask enrich_user_data: :environment do\n  pipeline = EnrichUserDataPipeline.trigger(7)\n  puts \"Pipeline #{pipeline.id} started\"\nend\n\n# In a controller\ndef create\n  pipeline = EnrichUserDataPipeline.trigger(params[:days_outdated])\n\n  render json: { pipeline_id: pipeline.id, status: pipeline.status }\nend\n```\n\n## Development\n\nAfter checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/ductwork/ductwork. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/ductwork/ductwork/blob/main/CODE_OF_CONDUCT.md).\n\n## License\n\nThe gem is available as open source under the terms of the [LGPLv3.0 License](https://github.com/ductwork/ductwork/blob/main/LICENSE.txt).\n\n## Code of Conduct\n\nEveryone interacting in the Ductwork project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/ductwork/ductwork/blob/main/CODE_OF_CONDUCT.md).\n","funding_links":[],"categories":["Ruby"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fductwork%2Fductwork","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fductwork%2Fductwork","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fductwork%2Fductwork/lists"}