{"id":15590097,"url":"https://github.com/bkeepers/qu","last_synced_at":"2025-05-16T04:04:05.149Z","repository":{"id":1684124,"uuid":"2411700","full_name":"bkeepers/qu","owner":"bkeepers","description":"a Ruby library for queuing and processing background jobs.","archived":false,"fork":false,"pushed_at":"2016-10-06T05:23:36.000Z","size":964,"stargazers_count":505,"open_issues_count":9,"forks_count":49,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-08T14:08:33.198Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bkeepers.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":null,"funding":null,"license":"MIT-LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2011-09-18T22:16:04.000Z","updated_at":"2024-11-20T15:39:40.000Z","dependencies_parsed_at":"2022-09-03T12:00:17.673Z","dependency_job_id":null,"html_url":"https://github.com/bkeepers/qu","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bkeepers%2Fqu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bkeepers%2Fqu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bkeepers%2Fqu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bkeepers%2Fqu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bkeepers","download_url":"https://codeload.github.com/bkeepers/qu/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253199524,"owners_count":21870083,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T23:19:51.131Z","updated_at":"2025-05-16T04:04:05.132Z","avatar_url":"https://github.com/bkeepers.png","language":"Ruby","funding_links":[],"categories":["Queue"],"sub_categories":[],"readme":"# Qu\n\nQu is a Ruby library for queuing and processing background jobs. It is heavily inspired by delayed_job and Resque.\n\nQu was created to overcome some shortcomings in the existing queuing libraries that we experienced at [Ordered List](http://orderedlist.com) while building [SpeakerDeck](http://speakerdeck.com), [Gaug.es](http://get.gaug.es) and [Harmony](http://get.harmonyapp.com). The advantages of Qu are:\n\n* Multiple backends (redis, mongo)\n* Jobs are requeued when worker is killed\n* Resque-like API\n\n## Information \u0026 Help\n\n* Find more information on the [Wiki](https://github.com/bkeepers/qu/wiki).\n* Post to the [Google Group](http://groups.google.com/group/qu-users) for help or questions.\n* See the [issue tracker](https://github.com/bkeepers/qu/issues) for known issues or to report an issue.\n\n## Installation\n\n### Rails 3 and 4\n\nDecide which backend you want to use and add the gem to your `Gemfile`.\n\n``` ruby\ngem 'qu-rails'\ngem 'qu-redis'\n```\n\nThat's all you need to do!\n\n### Rails 2\n\nDecide which backend you want to use and add the gem to `config.gems` in `environment.rb`:\n\n``` ruby\nconfig.gem 'qu-redis'\n````\n\nTo load the rake tasks, add the following to your `Rakefile`:\n\n``` ruby\nrequire 'qu/tasks'\n```\n\n## Usage\n\nJobs are defined by extending the `Qu::Job` class:\n\n``` ruby\nclass ProcessPresentation \u003c Qu::Job\n  def initialize(presentation_id)\n    @presentation_id = presentation_id\n  end\n\n  def perform\n    Presentation.find(@presentation_id).process!\n  end\nend\n```\n\nYou can add a job to the queue by calling `create` on your job:\n\n``` ruby\njob = ProcessPresentation.create(@presentation.id)\nputs \"Created job #{job.id}\"\n```\n\nThe job will be initialized with any parameters that are passed to it when it is performed. These parameters will be stored in the backend, so they must be simple types that can easily be serialized and unserialized. Don't try to pass in an ActiveRecord object.\n\nProcessing the jobs on the queue can be done with a Rake task:\n\n``` sh\n$ bundle exec rake qu:work\n```\n\nYou can easily inspect the queue or clear it:\n\n``` ruby\nputs \"Jobs on the queue:\", Qu.size\nQu.clear\n```\n\n### Queues\n\nThe `default` queue is used, um…by default. Jobs that don't specify a queue will be placed in that queue, and workers that don't specify a queue will work on that queue.\n\nHowever, if you have some background jobs that are more or less important, or some that take longer than others, you may want to consider using multiple queues. You can have workers dedicated to specific queues, or simply tell all your workers to work on the most important queue first.\n\nJobs can be placed in a specific queue by setting the queue:\n\n``` ruby\nclass CallThePresident \u003c Qu::Job\n  queue :urgent\n\n  def initialize(message)\n    @message = message\n  end\n\n  def perform\n    # …\n  end\nend\n```\n\nYou can then tell workers to work on this queue by passing an environment variable\n\n``` sh\n$ bundle exec rake qu:work QUEUES=urgent,default\n```\n\nNote that if you still want your worker to process the default queue, you must specify it. Queues will be process in the order they are specified.\n\nYou can also get the size or clear a specific queue:\n\n``` ruby\nQu.size(:urgent)\nQu.clear(:urgent)\n```\n\n## Configuration\n\nMost of the configuration for Qu should be automatic. It will also automatically detect ENV variables from Heroku for backend connections, so you shouldn't need to do anything to configure the backend.\n\nHowever, if you do need to customize it, you can by calling the `Qu.configure`:\n\n``` ruby\nQu.configure do |c|\n  c.logger = Logger.new('log/qu.log')\n  c.graceful_shutdown = true\nend\n```\n\n## Tests\n\nIf you prefer to have jobs processed immediatly in your tests, there is an `Immediate` backend that will perform the job instead of enqueuing it. In your test helper, require qu-immediate:\n\n``` ruby\nrequire 'qu-immediate'\n```\n\n## Why another queuing library?\n\n[Resque](https://github.com/resque/resque) and [delayed_job](https://github.com/collectiveidea/delayed_job) are both great, but both of them have shortcomings that can be frustrating in production applications.\n\ndelayed_job was a brilliantly simple pioneer in the world of database-backed queues. While most asynchronous queuing systems were tending toward overly complex, it made use of your existing database and just worked. But there are a few flaws:\n\n* Occasionally fails silently.\n* Use of priority instead of separate named queues.\n* Contention in the ActiveRecord backend with multiple workers. Occasionally the same job gets performed by multiple workers.\n\nResque, the wiser relative of delayed_job, fixes most of those issues. But in doing so, it forces some of its beliefs on you, and sometimes those beliefs just don't make sense for your environment. Here are some of the flaws of Resque:\n\n* Redis is a great queue backend, but it doesn't make sense for every environment.\n* Forking before each job prevents memory leaks, but it is terribly inefficient in environments with a lot of fast jobs (the resque-jobs-per-fork plugin alleviates this)\n\nThose shortcomings lead us to write Qu. It is not perfect, but we hope to overcome the issues we faced with other queuing libraries.\n\n## Contributing\n\nIf you find what looks like a bug:\n\n1. Search the [issues on GitHub](http://github.com/bkeepers/qu/issues/) to see if anyone else has reported issue.\n2. If you don't see anything, [create an issue](http://github.com/bkeepers/qu/issues/new) with information on how to reproduce it.\n\nIf you want to contribute an enhancement or a fix:\n\n1. [Fork the project](https://github.com/bkeepers/qu/fork) on GitHub.\n2. Make your changes with tests.\n3. Commit the changes without making changes to the Rakefile, Gemfile, gemspec, or any other files that aren't related to your enhancement or fix\n4. Send a pull request.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbkeepers%2Fqu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbkeepers%2Fqu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbkeepers%2Fqu/lists"}