{"id":13878563,"url":"https://github.com/adamniedzielski/sidekiq-staged_push","last_synced_at":"2025-07-16T14:32:29.697Z","repository":{"id":42366168,"uuid":"441929174","full_name":"adamniedzielski/sidekiq-staged_push","owner":"adamniedzielski","description":"sidekiq-staged_push is an extension to Sidekiq that implements a strategy of pushing jobs to Redis that involves an intermediary database table. This allows you to leverage the power of Sidekiq while still treating your jobs as if they were stored in the database for the purpose of transactions.","archived":false,"fork":false,"pushed_at":"2024-06-14T09:58:27.000Z","size":40,"stargazers_count":32,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-13T16:42:55.095Z","etag":null,"topics":["background-jobs","ruby","sidekiq"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adamniedzielski.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-26T15:57:40.000Z","updated_at":"2024-09-24T01:33:12.000Z","dependencies_parsed_at":"2024-11-13T16:48:11.361Z","dependency_job_id":null,"html_url":"https://github.com/adamniedzielski/sidekiq-staged_push","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamniedzielski%2Fsidekiq-staged_push","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamniedzielski%2Fsidekiq-staged_push/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamniedzielski%2Fsidekiq-staged_push/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamniedzielski%2Fsidekiq-staged_push/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adamniedzielski","download_url":"https://codeload.github.com/adamniedzielski/sidekiq-staged_push/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226138849,"owners_count":17579496,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["background-jobs","ruby","sidekiq"],"created_at":"2024-08-06T08:01:53.197Z","updated_at":"2025-07-16T14:32:29.680Z","avatar_url":"https://github.com/adamniedzielski.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"# `sidekiq-staged_push`\n\n`sidekiq-staged_push` is an extension to [Sidekiq](http://github.com/mperham/sidekiq) that\nimplements a strategy of pushing jobs to Redis that involves an intermediary database table.\nIt is developed based on the approach described in [Transactionally Staged Job Drains in Postgres](https://brandur.org/job-drain). Quoting the author:\n\n\u003e With this pattern, jobs aren’t immediately sent to the job queue. Instead, they’re staged in\n\u003e a table within the relational database itself, and the ACID properties of the running\n\u003e transaction keep them invisible until they’re ready to be worked. A secondary enqueuer\n\u003e process reads the table and sends any jobs it finds to the job queue before removing their\n\u003e rows.\n\nThis allows you to leverage the power of Sidekiq while still treating your jobs as if they\nwere stored in the database for the purpose of transactions.\n[See the demo app](https://github.com/adamniedzielski/sidekiq-staged_push-demo).\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'sidekiq-staged_push'\n```\n\nAnd then execute:\n\n    $ bundle\n\nIn `config/initializers/sidekiq.rb`:\n\n```ruby\nSidekiq::StagedPush.enable!\n```\n\nAnd finally, create the tables:\n\n    $ rails generate sidekiq:staged_push:install\n    $ rails db:migrate\n\n\n## Gotchas\n\n1. `SomeWorker.perform_bulk([[1], [2], [3]])` is inconsistent with other methods, because it\nschedules to Redis immediately.\n\n## Development\n\nYou need Docker with Docker Compose.\n\n```\nmake build\nmake bundle\n```\n\nThen you can run:\n\n```\nmake test\nmake rubocop\n```\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/adamniedzielski/sidekiq-staged_push. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n\n## Code of Conduct\n\nEveryone interacting in the `sidekiq-staged_push` project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/adamniedzielski/sidekiq-staged_push/blob/master/CODE_OF_CONDUCT.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamniedzielski%2Fsidekiq-staged_push","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadamniedzielski%2Fsidekiq-staged_push","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamniedzielski%2Fsidekiq-staged_push/lists"}