{"id":15581291,"url":"https://github.com/molawson/sidekiq-corral","last_synced_at":"2025-03-29T08:23:03.564Z","repository":{"id":182904055,"uuid":"669258801","full_name":"molawson/sidekiq-corral","owner":"molawson","description":"Sidekiq add-on for confining a job and its child jobs to a single queue","archived":false,"fork":false,"pushed_at":"2023-07-24T21:20:01.000Z","size":47,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-04T23:04:12.576Z","etag":null,"topics":["ruby","sidekiq"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/molawson.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-21T18:35:29.000Z","updated_at":"2023-08-17T16:32:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"4df1e1f1-8ef1-4bb5-8668-5c0d106c8ebf","html_url":"https://github.com/molawson/sidekiq-corral","commit_stats":{"total_commits":25,"total_committers":1,"mean_commits":25.0,"dds":0.0,"last_synced_commit":"824d5f4071f3dc92ff643e256c12220cfa3b5719"},"previous_names":["molawson/sidekiq-corral"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/molawson%2Fsidekiq-corral","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/molawson%2Fsidekiq-corral/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/molawson%2Fsidekiq-corral/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/molawson%2Fsidekiq-corral/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/molawson","download_url":"https://codeload.github.com/molawson/sidekiq-corral/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246157872,"owners_count":20732686,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ruby","sidekiq"],"created_at":"2024-10-02T19:42:22.884Z","updated_at":"2025-03-29T08:23:03.548Z","avatar_url":"https://github.com/molawson.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sidekiq::Corral\n\nA [Sidekiq](https://github.com/sidekiq/sidekiq) add-on that makes it easy to keep the processing for a job and all of the jobs it enqueues on a single queue.\n\n## But Why?\nImagine a situation where you need to introduce a new Sidekiq job to do a specific task. But because you're working with a good bit of data, you split up the work into a bunch of small, idempotent jobs, as Sidekiq suggests. And you even go a step further, putting this new job class on its own queue to avoid clogging up one of your standard queues with these specialized jobs.\n\nBut these new jobs hardly ever live in isolation. In your existing application you're probably enqueuing jobs from a wide variety of places. And that's great until you realize that your new job enqueues a handful of these preexisting jobs based on lifecycle events or something else a few call sites away from the new job itself. This can quickly lead to a situation where the new jobs are on their own queue, as you intended, but as you churn through enough of them, the other jobs they enqueue start to fill up your other queues!\n\nWouldn't it be nice to be able to tell a job that not only does it go on a certain queue, but any other job that's enqueued while it's being processed should _also_ go on that same queue, ensuring that all work related to the initial job is processed separately from the other jobs in your application?\n\nThat's exactly where Sidekiq::Corral comes in!\n\n## Installation\n\nInstall the gem and add to the application's Gemfile by executing:\n\n    $ bundle add sidekiq-corral\n\nIf bundler is not being used to manage dependencies, install the gem by executing:\n\n    $ gem install sidekiq-corral\n    \nInstall the middleware on application boot (e.g. in a Rails initializer or wherever you're configuring other parts of Sidekiq):\n\n```ruby\n# config/initializers/sidekiq.rb\n\nSidekiq::Corral.install\n```\n\nThis will register the included middleware in the right spots.\n\n## Usage\n\n### Sidekiq::Job.set\nSet the corral when enqueueing a job:\n\n```ruby\nSomeJob.set(corral: \"backfill\").perform_async(args)\n```\n\nThis will both set the corral and the queue for that job and any job enqueued during processing of that `SomeJob` instance.\n\n### Sidekiq::Corral.confine\nIf you're enqueueing multiple jobs or calling classes that enqueue jobs of their own and you want to confine everything to a single queue, you can use `Sidekiq::Corral.confine`:\n\n```ruby\nSidekiq::Corral.confne(\"backfill\") do\n  SomeJob.perform_async(args)\n  ClassThatEnqueuesJobs.new.call(more_args)\n  AnotherJob.peform_async(even_more_args) \nend\n```\n\nAll jobs enqueued within that block (including those enqueued in `ClassThatEnqueuesJobs`, etc.) will be put in the `\"backfill\"` corral and processed on the `\"backfill\"` queue.\n\n\n### Exempt Queues\n\nSometimes a queue is special enough that you always want jobs destined for it to _always_ be processed there, regardless of Sidekiq::Corral's concerns. You can name those queues on setup:\n\n```ruby\nSidekiq::Corral.install(exempt_queues: [\"notifications\"])\n```\n\nDoing this will ensure anything destined for the `\"notifications\"` queue will be processed there. But those jobs will still pass along the corral set either on the job or further up the chain.  So any jobs enqueued while it's processing will use the corral.\n\nFor example, given this set of jobs:\n\n```ruby\nclass NormalJob\n  include Sidekiq::Job\n  sidekiq_options queue: \"default\"\n  \n  def perform\n    SpecialJob.perform_async\n  end\nend\n\nclass SpecialJob\n  include Sidekiq::Job\n  sidekiq_options queue: \"notifications\"\n  \n  def peform\n    AnotherNormalJob.perform_async\n  end\nend\n\nclass AnotherNormalJob\n  include Sidekiq::Job\n  sidekiq_options queue: \"default\"\n  \n  def perform\n  end\nend\n```\n\nEnqueuing the NormalJob with a corral would get processed like so:\n\n```ruby\nNormalJob.set(corral: \"backfill\").perform_async\n\nNormalJob               # queue: \"backfill\"       corral: \"backfill\"\n-\u003e SpecialJob           # queue: \"notifications\"  corral: \"backfill\"\n   -\u003e AnotherNormalJob  # queue: \"backfill\"       corral: \"backfill\"\n```\n\n\n## Development\n\nAfter checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/molawson/sidekiq-corral. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/molawson/sidekiq-corral/blob/main/CODE_OF_CONDUCT.md).\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n\n## Code of Conduct\n\nEveryone interacting in the Sidekiq::Corral project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/molawson/sidekiq-corral/blob/main/CODE_OF_CONDUCT.md).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmolawson%2Fsidekiq-corral","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmolawson%2Fsidekiq-corral","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmolawson%2Fsidekiq-corral/lists"}