{"id":17680284,"url":"https://github.com/kostya/pgq","last_synced_at":"2025-05-12T21:14:27.513Z","repository":{"id":3235397,"uuid":"4271700","full_name":"kostya/pgq","owner":"kostya","description":"Queues system for AR/Rails based on PgQ Skytools for PostgreSQL, like Resque on Redis. Rails 2.3 and 3 compatible.","archived":false,"fork":false,"pushed_at":"2015-05-29T17:34:34.000Z","size":226,"stargazers_count":12,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-13T04:37:44.138Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kostya.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-05-09T12:59:50.000Z","updated_at":"2022-07-04T08:36:26.000Z","dependencies_parsed_at":"2022-09-01T07:50:40.539Z","dependency_job_id":null,"html_url":"https://github.com/kostya/pgq","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kostya%2Fpgq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kostya%2Fpgq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kostya%2Fpgq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kostya%2Fpgq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kostya","download_url":"https://codeload.github.com/kostya/pgq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249933671,"owners_count":21347725,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-24T09:06:26.265Z","updated_at":"2025-04-20T17:33:55.721Z","avatar_url":"https://github.com/kostya.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"Pgq\n===\n\nQueues system for AR/Rails based on PgQ Skytools for PostgreSQL, like Resque on Redis. Rails 2.3 and 3 compatible.\n \nAbout PgQ\n*  http://wiki.postgresql.org/wiki/SkyTools#PgQ\n\n\nInstall\n-------\n\nInstall skytools:\nUbuntu 11.10: \n\n    # apt-get install postgresql-server postgresql-client\n    # apt-get install skytools\n\nGemfile:\n\n```ruby\ngem 'pgq'\n```\n\nCreate ticker configs from database.yml:\n\n    $ rake pgq:generate_configs \n    edit config: config/pgq_development.ini\n\n\nInstall pgq to database (if test database recreates all the time, should reinstall pgq each time):\n\n    $ rake pgq:install\n    or execute\n    $ pgqadm config/pgq_development.ini install\n\n   \n\nRun ticker daemon (ticker needs on production database, or development if we test the process of consuming):\nDaemon run once, bind to the database. (If worker not consuming check that daemon started).\n\n    $ rake pgq:ticker:start (stop)\n    or execute\n    $ pgqadm config/pgq_development.ini ticker -d\n    \n\nLast, add to config/application.rb\n\n    config.autoload_paths += %W( #{config.root}/app/workers )\n    \n\nCreate Pgq consumer\n-------------------\n\n    $ rails generate pgq:add my\n\n\nThis creates file app/workers/pgq_my.rb and migration\n\n    $ rake db:migrate\n\n\n```ruby\nclass PgqMy \u003c Pgq::Consumer\n\n  def some_method1(a, b, c)\n    logger.info \"async call some_method1 with #{[a, b, c].inspect}\"\n  end\n  \n  def some_method2(x)\n    logger.info \"async called some_method2 with #{x.inspect}\"\n  end\n  \nend\n```\n\nInsert event into queue:\n\n    PgqMy.some_method1(1, 2, 3)\n    \n    or\n        \n    PgqMy.add_event(:some_method2, some_x)\n            \n    or\n    \n    PgqMy.enqueue(:some_method1, 1, 2, 3)\n  \n\nWorkers\n-------\nStart worker for queue:\n\n    $ rake pgq:worker QUEUES=\"my\"\n    $ rake pgq:worker QUEUES=\"my,mailer,other\"\n    $ rake pgq:worker QUEUES=\"all\" RAILS_ENV=production\n    \n\n\nAlso can consume manual, or write [bin_script](http://github.com/kostya/bin_script):\n```ruby\nclass PgqRunnerScript \u003c BinScript\n\n  self.enable_locking = false\n\n  required :q, \"queues separated by ','\"\n  required :w, \"watch file\"\n  \n  def do!\n    worker = Pgq::Worker.new(:logger =\u003e self.logger, :queues =\u003e params(:q), \n                             :watch_file =\u003e params(:w) || \"./tmp/stop_all.txt\")\n    worker.run\n  end\n                                                                     \nend\n```\n\nand run:\n\n    $ ./bin/pgq_runner.rb -q my -l ./log/pgq_my.log\n    $ ./bin/pgq_runner.rb -q all -l ./log/pgq_all.log   # this will consume all queues from config/queues_list.yml\n\n\n\n### Admin interface\nRealized by gem [pgq_web](http://github.com/kostya/pgq_web).\n\n\n### Divide events between workers, for one consumer class\ncreate more queues: my_2, my_3\n\n```ruby\nclass PgqMy \u003c Pgq::Consumer\n\n  QUEUES = %w(my my_2 my_3).freeze\n\n  def self.next_queue_name\n    QUEUES.choice\n  end\n  \nend\n```\nAnd run 3 workers: for my, my_2, my_3\n\n\n\n\n### Consume group of events extracted from pgq(batch): \nUsually batch size is ~500 events.\n\n```ruby\nclass PgqMy \u003c Pgq::ConsumerGroup\n\n  # {'type' =\u003e [events]}\n  def perform_group(events_hash)\n    raise \"realize me\"\n  end\n  \nend\n```\n\n\n\n### Options\ncreate initializer with:\n\n```ruby\nclass Pgq::Consumer\n\n  # specify database with pgq queues\n  def self.database\n    SomeDatabase \n  end\n\n  # specify coder  \n  def self.coder\n    Marshal64 # class/module with self methods dump/load\n  end\n    \nend\n```\n\n### Proxy method to consumer\nUsefull in specs\n\n``` ruby\n  PgqMy.proxy(:some_method1)\n```\n \nWhen code call PgqMy.some_method1(a,b,c) this would be convert into PgqMy.new.some_method1(a,b,c)\n  \n  \n### for Rails 2.3\n\nAdd to Rakefile:\n```\nload 'tasks/pgq.rake'\n```\n\nCopy generators from gem to lib/generators.\n\n```\n./script/generate pgq bla\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkostya%2Fpgq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkostya%2Fpgq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkostya%2Fpgq/lists"}