{"id":16937214,"url":"https://github.com/ixti/sidekiq-throttled-clone","last_synced_at":"2026-01-16T08:30:17.545Z","repository":{"id":37942521,"uuid":"502751763","full_name":"ixti/sidekiq-throttled-clone","owner":"ixti","description":"Temporary clone of https://github.com/ixti/sidekiq-throttled","archived":false,"fork":false,"pushed_at":"2023-04-06T22:29:30.000Z","size":450,"stargazers_count":17,"open_issues_count":1,"forks_count":18,"subscribers_count":3,"default_branch":"main","last_synced_at":"2023-04-10T23:08:05.376Z","etag":null,"topics":["concurrency","rate-limiting","sidekiq","threshold","throttling"],"latest_commit_sha":null,"homepage":"https://github.com/ixti/sidekiq-throttled","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ixti.png","metadata":{"files":{"readme":"README.adoc","changelog":"CHANGES.adoc","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-12T23:36:12.000Z","updated_at":"2023-04-10T23:07:51.000Z","dependencies_parsed_at":"2022-07-12T17:04:00.379Z","dependency_job_id":null,"html_url":"https://github.com/ixti/sidekiq-throttled-clone","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ixti%2Fsidekiq-throttled-clone","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ixti%2Fsidekiq-throttled-clone/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ixti%2Fsidekiq-throttled-clone/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ixti%2Fsidekiq-throttled-clone/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ixti","download_url":"https://codeload.github.com/ixti/sidekiq-throttled-clone/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219851161,"owners_count":16556306,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency","rate-limiting","sidekiq","threshold","throttling"],"created_at":"2024-10-13T20:59:02.309Z","updated_at":"2024-10-13T20:59:02.843Z","avatar_url":"https://github.com/ixti.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"= Sidekiq::Throttled\n:ci-link: https://github.com/ixti/sidekiq-throttled/actions/workflows/ci.yml\n:ci-badge: https://img.shields.io/github/workflow/status/ixti/sidekiq-throttled/CI/main?style=for-the-badge\n:gem-link: http://rubygems.org/gems/sidekiq-throttled\n:gem-badge: https://img.shields.io/gem/v/sidekiq-throttled?style=for-the-badge\n:doc-link: http://www.rubydoc.info/gems/sidekiq-throttled\n:doc-badge: https://img.shields.io/badge/Documentation-API-blue?style=for-the-badge\n\n****\n{ci-link}[image:{ci-badge}[CI Status]]\n{gem-link}[image:{gem-badge}[Latest Version]]\n{doc-link}[image:{doc-badge}[API Documentation]]\n****\n\nNOTE: This is the 1.x *development* branch. For the 0.x *stable* branch, please\n  see: https://github.com/ixti/sidekiq-throttled/tree/0-x-stable[0-x-stable]\n\nConcurrency and threshold throttling for https://github.com/mperham/sidekiq[Sidekiq].\n\n\n== Installation\n\nAdd this line to your application's Gemfile:\n\n[source,ruby]\n----\ngem \"sidekiq-throttled\"\n----\n\nAnd then execute:\n\n  $ bundle\n\nOr install it yourself as:\n\n  $ gem install sidekiq-throttled\n\n\n== Usage\n\nAdd somewhere in your app's bootstrap (e.g. `config/initializers/sidekiq.rb` if\nyou are using Rails):\n\n[source,ruby]\n----\nrequire \"sidekiq/throttled\"\nSidekiq::Throttled.setup!\n----\n\nLoad order can be an issue if you are using other Sidekiq plugins and/or middleware.\nTo prevent any problems, add the `.setup!` call to the bottom of your init file.\n\nOnce you've done that you can include `Sidekiq::Throttled::Job` to your\njob classes and configure throttling:\n\n[source,ruby]\n----\nclass MyJob\n  include Sidekiq::Job\n  include Sidekiq::Throttled::Job\n\n  sidekiq_options :queue =\u003e :my_queue\n\n  sidekiq_throttle(\n    # Allow maximum 10 concurrent jobs of this class at a time.\n    :concurrency =\u003e { :limit =\u003e 10 },\n    # Allow maximum 1K jobs being processed within one hour window.\n    :threshold =\u003e { :limit =\u003e 1_000, :period =\u003e 1.hour }\n  )\n\n  def perform\n    # ...\n  end\nend\n----\n\nTIP: `Sidekiq::Throttled::Job` is aliased as `Sidekiq::Throttled::Worker`,\n  thus if you're using `Sidekiq::Worker` naming convention, you can use the\n  alias for consistency:\n\n[source,ruby]\n----\nclass MyWorker\n  include Sidekiq::Worker\n  include Sidekiq::Throttled::Worker\n\n  # ...\nend\n----\n\n\n=== Observer\n\nYou can specify an observer that will be called on throttling. To do so pass an\n`:observer` option with callable object:\n\n[source,ruby]\n----\nclass MyJob\n  include Sidekiq::Job\n  include Sidekiq::Throttled::Job\n\n  MY_OBSERVER = lambda do |strategy, *args|\n    # do something\n  end\n\n  sidekiq_options :queue =\u003e :my_queue\n\n  sidekiq_throttle(\n    :concurrency =\u003e { :limit =\u003e 10 },\n    :threshold   =\u003e { :limit =\u003e 100, :period =\u003e 1.hour },\n    :observer    =\u003e MY_OBSERVER\n  )\n\n  def perform(*args)\n    # ...\n  end\nend\n----\n\nObserver will receive `strategy, *args` arguments, where `strategy` is a Symbol\n`:concurrency` or `:threshold`, and `*args` are the arguments that were passed\nto the job.\n\n\n=== Dynamic throttling\n\nYou can throttle jobs dynamically with `:key_suffix` option:\n\n[source,ruby]\n----\nclass MyJob\n  include Sidekiq::Job\n  include Sidekiq::Throttled::Job\n\n  sidekiq_options :queue =\u003e :my_queue\n\n  sidekiq_throttle(\n    # Allow maximum 10 concurrent jobs per user at a time.\n    :concurrency =\u003e { :limit =\u003e 10, :key_suffix =\u003e -\u003e (user_id) { user_id } }\n  )\n\n  def perform(user_id)\n    # ...\n  end\nend\n----\n\nYou can also supply dynamic values for limits and periods by supplying a proc\nfor these values. The proc will be evaluated at the time the job is fetched\nand will receive the same arguments that are passed to the job.\n\n[source,ruby]\n----\nclass MyJob\n  include Sidekiq::Job\n  include Sidekiq::Throttled::Job\n\n  sidekiq_options :queue =\u003e :my_queue\n\n  sidekiq_throttle(\n    # Allow maximum 1000 concurrent jobs of this class at a time for VIPs and 10 for all other users.\n    :concurrency =\u003e {\n      :limit      =\u003e -\u003e(user_id) { User.vip?(user_id) ? 1_000 : 10 },\n      :key_suffix =\u003e -\u003e(user_id) { User.vip?(user_id) ? \"vip\" : \"std\" }\n    },\n    # Allow 1000 jobs/hour to be processed for VIPs and 10/day for all others\n    :threshold   =\u003e {\n      :limit      =\u003e -\u003e(user_id) { User.vip?(user_id) ? 1_000 : 10 },\n      :period     =\u003e -\u003e(user_id) { User.vip?(user_id) ? 1.hour : 1.day },\n      :key_suffix =\u003e -\u003e(user_id) { User.vip?(user_id) ? \"vip\" : \"std\" }\n    }\n  )\n\n  def perform(user_id)\n    # ...\n  end\nend\n----\n\nYou also can use several different keys to throttle one worker.\n\n[source,ruby]\n----\nclass MyJob\n  include Sidekiq::Job\n  include Sidekiq::Throttled::Job\n\n  sidekiq_options :queue =\u003e :my_queue\n\n  sidekiq_throttle(\n    # Allow maximum 10 concurrent jobs per project at a time and maximum 2 jobs per user\n    :concurrency =\u003e [\n      { :limit =\u003e 10, :key_suffix =\u003e -\u003e (project_id, user_id) { project_id } },\n      { :limit =\u003e 2, :key_suffix =\u003e -\u003e (project_id, user_id) { user_id } }\n    ]\n    # For :threshold it works the same\n  )\n\n  def perform(project_id, user_id)\n    # ...\n  end\nend\n----\n\nIMPORTANT: Don't forget to specify `:key_suffix` and make it return different\n  values if you are using dynamic limit/period options. Otherwise, you risk\n  getting into some trouble.\n\n\n=== Concurrency throttling fine-tuning\n\nConcurrency throttling is based on distributed locks. Those locks have default\ntime to live (TTL) set to 15 minutes. If your job takes more than 15 minutes\nto finish, lock will be released and you might end up with more jobs running\nconcurrently than you expect.\n\nThis is done to avoid deadlocks - when by any reason (e.g. Sidekiq process was\nOOM-killed) cleanup middleware wasn't executed and locks were not released.\n\nIf your job takes more than 15 minutes to complete, you can tune concurrency\nlock TTL to fit your needs:\n\n[source,ruby]\n----\n# Set concurrency strategy lock TTL to 1 hour.\nsidekiq_throttle(:concurrency =\u003e { :limit =\u003e 20, :ttl =\u003e 1.hour.to_i })\n----\n\n\n== Supported Ruby Versions\n\nThis library aims to support and is tested against the following Ruby versions:\n\n* Ruby 2.7.x\n* Ruby 3.0.x\n* Ruby 3.1.x\n\nIf something doesn't work on one of these versions, it's a bug.\n\nThis library may inadvertently work (or seem to work) on other Ruby versions,\nhowever support will only be provided for the versions listed above.\n\nIf you would like this library to support another Ruby version or\nimplementation, you may volunteer to be a maintainer. Being a maintainer\nentails making sure all tests run and pass on that implementation. When\nsomething breaks on your implementation, you will be responsible for providing\npatches in a timely fashion. If critical issues for a particular implementation\nexist at the time of a major release, support for that Ruby version may be\ndropped.\n\n\n== Supported Sidekiq Versions\n\nThis library aims to support and work with following Sidekiq versions:\n\n* Sidekiq 6.4.x\n* Sidekiq 6.5.x\n\n\n== Contributing\n\n* Fork sidekiq-throttled on GitHub\n* Make your changes\n* Ensure all tests pass (`bundle exec rake`)\n* Send a pull request\n* If we like them we'll merge them\n* If we've accepted a patch, feel free to ask for commit access!\n\n\n== Development\n\n  bundle update\n  bundle exec appraisal install   # install dependencies for all gemfiles\n  bundle exec appraisal update    # update dependencies for all gemfiles\n  bundle exec appraisal rspec     # run rspec against each gemfile\n  bundle exec rubocop             # run static code analysis\n\nDon't forget to run `appraisal update` after any changes to `Gemfile`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fixti%2Fsidekiq-throttled-clone","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fixti%2Fsidekiq-throttled-clone","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fixti%2Fsidekiq-throttled-clone/lists"}