{"id":13880180,"url":"https://github.com/kickstarter/replica_pools","last_synced_at":"2025-04-06T16:14:27.785Z","repository":{"id":2413633,"uuid":"3381639","full_name":"kickstarter/replica_pools","owner":"kickstarter","description":"A leader/replica setup with separate replica pools, originally forked from schoefmax/multi_db","archived":false,"fork":false,"pushed_at":"2024-10-15T14:42:49.000Z","size":265,"stargazers_count":24,"open_issues_count":3,"forks_count":5,"subscribers_count":125,"default_branch":"main","last_synced_at":"2025-03-30T15:09:27.280Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kickstarter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2012-02-07T21:33:02.000Z","updated_at":"2024-10-15T14:44:19.000Z","dependencies_parsed_at":"2024-01-13T20:58:12.119Z","dependency_job_id":"70063589-f65d-42d7-9d48-f577408722de","html_url":"https://github.com/kickstarter/replica_pools","commit_stats":{"total_commits":205,"total_committers":17,"mean_commits":"12.058823529411764","dds":0.5902439024390245,"last_synced_commit":"5630567a7167be7fc15236fcf669c581c7ecbf2b"},"previous_names":["kickstarter/slave_pools"],"tags_count":26,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kickstarter%2Freplica_pools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kickstarter%2Freplica_pools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kickstarter%2Freplica_pools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kickstarter%2Freplica_pools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kickstarter","download_url":"https://codeload.github.com/kickstarter/replica_pools/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247509237,"owners_count":20950232,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-06T08:02:50.412Z","updated_at":"2025-04-06T16:14:27.763Z","avatar_url":"https://github.com/kickstarter.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"# ReplicaPools\n\nEasy Single Leader / Multiple Replica Setup for use in Ruby/Rails projects\n\n[![Spec](https://github.com/kickstarter/replica_pools/actions/workflows/spec.yml/badge.svg)](https://github.com/kickstarter/replica_pools/actions/workflows/spec.yml)\n\n## Overview\n\nReplicaPools replaces ActiveRecord's connection with a proxy that routes database interactions to the proper connection. Safe (whitelisted) methods may go to the current replica, and all other methods go to the leader connection.\n\nReplicaPools also provides helpers so you can customize your replica strategy. You can organize replicas into pools and cycle through them (e.g. in a before_filter). You can make the connection default to the leader, or the default replica pool, and then use block helpers to temporarily change the behavior (e.g. in an around_filter).\n\n- Uses a naming convention in database.yml to designate replica pools.\n- Defaults to a given replica pool, but may also be configured to default to leader.\n- Routes database interactions (queries) to the right connection\n  - Whitelisted queries go to the current connection (might be a replica).\n  - All queries inside a transaction run on leader.\n  - All other queries are also sent to the leader connection.\n- Supports ActiveRecord's in-memory query caching.\n- Helper methods can be used to easily load balance replicas, route traffic to different replica pools, or run directly against leader. (examples below)\n\n## Not Supported\n\n- Sharding.\n- Automatic load balancing strategies.\n- Replica weights. You can accomplish this in your own load balancing strategy.\n- Whitelisting models that always use leader.\n- Blacklisting poorly performing replicas. This could cause load spikes on your leader. Whatever provisions your database.yml should make this choice.\n\n## Installation and Setup\n\nAdd to your Gemfile:\n\n    gem 'replica_pools'\n\n### Adding Replicas\n\nAdd entries to your database.yml in the form of `\u003cenvironment\u003e_pool_\u003cpool_name\u003e_name_\u003cdb_name\u003e`\n\nFor example:\n\n    # Leader connection for production environment\n    production:\n      adapter: mysql\n      database: myapp_production\n      username: root\n      password:\n      host: localhost\n\n    # Default pool for production environment\n    production_pool_default_name_replica1:\n      adapter: mysql\n      database: replica_db1\n      username: root\n      password:\n      host: 10.0.0.2\n    production_pool_default_name_replica2:\n      ...\n\n    # Special pool for production environment\n    production_pool_admin_name_replica1:\n      ...\n    production_pool_admin_name_another_replica:\n      ...\n\n### Simulating Replicas\n\nIf you don't have any replicas (e.g. in your development environment), ReplicaPools will create a default pool containing only leader. But if you want to mimic your production environment more closely you can create a read-only mysql user and use it like a replica.\n\n    # Development connection\n    development: \u0026dev\n      adapter: mysql\n      database: myapp_development\n      username: root\n      password:\n      host: localhost\n\n    development_pool_default_name_replica1:\n      username: readonly\n      \u003c\u003c: \u0026dev\n\nDon't do this in your test environment if you use transactional tests though! The replica connections won't be able to see any fixtures or factory data.\n\n### Configuring\n\nAdd a `config/initializers/replica_pools.rb` if you want to change config settings:\n\n    ReplicaPools.config.defaults_to_leader = true\n\n## Usage\n\nToggle to next replica:\n\n    ReplicaPools.next_replica!\n\nSpecify a pool besides the default:\n\n    ReplicaPools.with_pool('other_pool') { #do stuff }\n\nSpecifically use the leader for a call:\n\n    ReplicaPools.with_leader { #do stuff }\n\n### Load Balancing\n\nIf you have multiple replicas in a pool and you'd like to load balance requests between them, you can easily accomplish this with a `before_filter`:\n\n    class ApplicationController \u003c ActionController::Base\n      after_filter    :switch_to_next_replica\n\n      protected\n\n      def switch_to_next_replica\n        ReplicaPools.next_replica!\n      end\n    end\n\n### Specialty Pools\n\nIf you have specialized replica pools and would like to use them for different controllers or actions, you can use an `around_filter`:\n\n    class ApplicationController \u003c ActionController::Base\n      around_filter   :use_special_replicas\n\n      protected\n\n      def use_special_replicas\n        ReplicaPools.with_pool('special'){ yield }\n      end\n    end\n\n### Replica Lag\n\nBy default, writes are sent to the leader and reads are sent to replicas. But replicas might lag behind the leader by seconds or even minutes. So if you write to leader during a request you probably want to read from leader in that request as well. You may even want to read from the leader on the _next_ request, to cover redirects.\n\nHere's one way to accomplish that:\n\n    class ApplicationController \u003c ActionController::Base\n\n      around_filter   :stick_to_leader_for_updates\n      around_filter   :use_leader_for_redirect #goes with above\n\n      def stick_to_leader_for_updates\n        if request.get?\n          yield\n        else\n          ReplicaPools.with_leader { yield }\n          session[:stick_to_leader] = 1\n        end\n      end\n\n      def use_leader_for_redirect\n        if session[:stick_to_leader]\n          session[:stick_to_leader] = nil\n          ReplicaPools.with_leader { yield }\n        else\n          yield\n        end\n      end\n    end\n\n## Disabling Leader\n\nTo disable queries to the leader database -- for instance, in a production\nconsole -- set the disable_leader configuration to false. This will raise\na ReplicaPools::LeaderDisabled error:\n\nReplicaPools.config.disable_leader = false\n\n## Running specs\n\nTests are run against MySQL 5.6 using docker-compose. 🐋\n\nTo get set up, first run:\n\n    docker-compose up\n    bundle install\n    bundle exec rake bootstrap\n\nThen you can run tests with:\n\n    bundle exec rake spec\n\n## Releasing a New Version\n\nFollow these steps to publish a new version to RubyGems:\n\n1. Increment the version in [lib/replica_pools/version.rb](./lib/replica_pools/version.rb). We follow [Semantic Versioning](http://semver.org).\n2. Push changes to GitHub\n3. Create a [new release](https://github.com/kickstarter/replica_pools/releases)\n4. Ensure the [Test and Release](https://github.com/kickstarter/replica_pools/actions/workflows/test-release.yml) GitHub Action succeeds in pushing a new gem to RubyGems\n\n## Authors\n\nAuthor: Dan Drabik, Lance Ivy\n\nCopyright (c) 2012-2021, Kickstarter\n\nReleased under the MIT license\n\n## See also\n\n### MultiDb\n\nThe project is based on:\n\n- [github.com/schoefmax/multi_db](https://github.com/schoefmax/multi_db)\n\n### Masochism\n\nThe original leader/replica plugin:\n\n- [github.com/technoweenie/masochism](http://github.com/technoweenie/masochism)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkickstarter%2Freplica_pools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkickstarter%2Freplica_pools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkickstarter%2Freplica_pools/lists"}