{"id":19446050,"url":"https://github.com/stitchfix/resqutils","last_synced_at":"2025-04-25T01:31:58.373Z","repository":{"id":20926683,"uuid":"24214787","full_name":"stitchfix/resqutils","owner":"stitchfix","description":"Handy methods, classes, and test support for applications that use Resque","archived":false,"fork":false,"pushed_at":"2024-11-08T15:41:45.000Z","size":91,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":89,"default_branch":"main","last_synced_at":"2024-11-08T16:38:38.792Z","etag":null,"topics":["gem","opensource","ruby"],"latest_commit_sha":null,"homepage":"http://www.rubydoc.info/github/stitchfix/resqutils/","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stitchfix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-09-19T03:31:09.000Z","updated_at":"2024-11-08T15:41:47.000Z","dependencies_parsed_at":"2023-02-10T12:31:10.268Z","dependency_job_id":"9a58cc5e-0536-4d83-9e37-36b48fe49e4f","html_url":"https://github.com/stitchfix/resqutils","commit_stats":{"total_commits":98,"total_committers":14,"mean_commits":7.0,"dds":0.6122448979591837,"last_synced_commit":"26630b6791e442bb5235acb5e1fb4eae4f6714d9"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stitchfix%2Fresqutils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stitchfix%2Fresqutils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stitchfix%2Fresqutils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stitchfix%2Fresqutils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stitchfix","download_url":"https://codeload.github.com/stitchfix/resqutils/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223721389,"owners_count":17191900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gem","opensource","ruby"],"created_at":"2024-11-10T16:12:42.474Z","updated_at":"2024-11-10T16:12:43.190Z","avatar_url":"https://github.com/stitchfix.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# resqutils - useful stuff when you have Resque in your app\n\nThis is a small library of useful modules and functions that can help dealing with Resque.\n\nCurrently:\n\n* Job that kills stale workers\n* Means to identify stale workers\n* Spec helper `:some_queue.should have_job_queued(class: FooJob)`\n* Methods to introspect queues, including the delayed queue, in your specs\n* Simple `resque:work` task wrapper to better handle exceptions in the worker\n* Marker interface to document jobs which should not be retried\n\nMaybe will have more stuff someday.\n\n## To use\n\nAdd to your `Gemfile`:\n\n```ruby\ngem 'resqutils'\n```\n\n## Stale Workers\n\nIt's possible (and, on Heroku, highly likely) that your jobs will appear to be running for \"too long\".  Usually, this happens\nwhen a worker exits without cleaning up after itself.  Since Resque stores all state in Redis, and is process-based, it's\nactually fairly easy to create this situation.\n\nThe good news is that, if your jobs are idempotent, you can just unregister the \"stale\" workers, which will kick-off the\nfailed handling (which is hopefully to restart your jobs).\n\nYou need a means of identifying these workers, and then killing them.\n\n```ruby\nResqutils::StaleWorkersKiller.kill_stale_workers\n```\n\nThis will queue a `WorkerKillerJob` for each stale worker.  This uses `Resqutils::StaleWorkers` under the covers to identify\nwhich are stale.  You can pass it in to `StaleWorkersKiller`'s constructor, or configure it using the environment.  By\ndefault, a worker running for more than an hour is considered stale.\n\nSetting the `RESQUTILS_SECONDS_TO_BE_CONSIDERED_STALE` environment variable, you can override that.\n\nThe queue that `WorkerKillerJob` will queue to is `worker_killer_job` by default, but can be changed by setting the `RESQUTILS_WORKER_KILLER_JOB_QUEUE` environment variable.\n\n`Resqutils::StaleWorkersKiller` is also itself a resque job, so you can use this class directly in your resque-scheduler\nimplementation to kill stale jobs on a schedule.\n\nYou can, of course, use these building blocks on your own for other purposes.\n\n## Spec Helpers\n\n```ruby\n# in spec_helper.rb\nrequire 'resqutils/spec'\n\n# In one of your spec files\ndescribe SomeProcess do\n  include Resqutils::Spec::ResqueHelpers\n\n  # ...\nend\n```\n\n`require`ing the `resqutils/spec` will also set up the `have_job_queued` matcher, which is likely what you'll want to use.\n\n### Clearing Jobs\n\nThe most important part of using Resque in tests as making sure the queue has what you\nthink it has in it.  To that end, you'll likely need `clear_queue` in a `setup` or\n`before` block.\n\n```ruby\nbefore do\n  clear_queue(MyImportantJob) # clears whatever queue this job is configured to use\n  clear_queue(:foobar)        # clear the \"foobar\" queue\nend\n```\n\n### Checking that Jobs Were Queued\n\n```ruby\n# foo_service.rb\nclass FooService\n  def doit(foo)\n    Resque.enqueue(:foo,FooJob,foo)\n    \"bar\"\n  end\nend\n\n# foo_service_spec.rb\ndescribe FooService do\n  include Resqutils::Spec::ResqueHelpers\n  \n  before do\n    clear_queue(FooJob) # Looks at what queue FooJob uses and clears before each test\n  end\n  \n  it \"queues a job\" do\n    result = FooService.new.doit(\"blah\")\n\n    expect(result).to eq(\"bar\")\n    expect(:foo).to have_job_queued(class: FooJob, args: [ \"blah\" ])\n  end\nend\n```\n\nThis also works with the delayed queue as provided by resque-scheduler:\n\n```ruby\n# foo_service.rb\nclass FooService\n  def doit(foo)\n    Resque.enqueue_in(5.minutes,:foo,FooJob,foo)\n    \"bar\"\n  end\nend\n\n# foo_service_spec.rb\n\ndescribe FooService do\n  include Resqutils::Spec::ResqueHelpers\n  \n  before do\n    clear_queue(:delayed) # Clears all delayed/scheduled queues\n  end\n  it \"queues a job\" do\n    result = FooService.new.doit(\"blah\")\n\n    expect(result).to eq(\"bar\")\n    # :delayed is special and triggers logic to look into the various scheduled queues\n    expect(:delayed).to have_job_queued(class: FooJob, args: [ \"blah\" ])\n  end\nend\n```\n\n### Executing Jobs\n\nIn an integration test, you may wish to execute a job that's on the queue, which will both assert that it's there and perform whatever function it performs.\n\n\n```ruby\n# foo_service.rb\nclass FooService\n  def doit(foo)\n    Resque.enqueue(:foo,FooJob,foo)\n    \"bar\"\n  end\nend\n\nclass FooJob\n  def perform(some_value)\n    Foo.create!(value: some_value)\n  end\nend\n\n# the_foo_service_spec.rb\ndescribe \"the foo service\" do\n  include Resqutils::Spec::ResqueHelpers\n  it \"writes a Foo with the value\" do\n    result = FooService.new.doit(\"blah\")\n\n    process_resque_job(FooJob)\n\n    expect(Foo.last.value).to eq(\"blah\")\n  end\nend\n```\n\nThe `ResqueHelpers` module has many more methods, if you need finer control over your tests with respect to resque.\n\n### Exception Handling in your Worker\n\nThe built-in worker lets exceptions bubble up.\nIn a PaaS setup, or where your Redis is \"over the internet\", you'll get periodic connection issues from your worker.\nThese self-heal when your worker management system (e.g. monit) restarts the worker after it crashes.\nThus, these unhandled exceptions should just be ignored.\n\nSince the built-in resque worker is a rake task, we provide a wrapper rake task to call it and log the exception:\n\n```ruby\nrequire 'resqutils/worker_task'\n```\n\nTo run:\n\n```\nenv TERM_CHILD=1 bundle exec rake environment resqutils:work QUEUE=file_uploads --trace\n```\n\n### Being clear about not retrying\n\nAlthough you should design your jobs to automatically retry, some jobs simply should not be retried.\nInstead of omitting the retry logic or dropping in a comment, you should use a marker interface to communicate intent via code:\n\n```ruby\nclass DangerousJob\n  include Resqutils::DoNotAutoRetry\n\n  def perform\n    # ...\n  end\nend\n```\n\nThis is a more powerful statement that a comment, and communicates intent clearly.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstitchfix%2Fresqutils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstitchfix%2Fresqutils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstitchfix%2Fresqutils/lists"}