{"id":15145734,"url":"https://github.com/soundcloud/master_slave_adapter","last_synced_at":"2025-09-29T15:31:01.534Z","repository":{"id":56882917,"uuid":"2065323","full_name":"soundcloud/master_slave_adapter","owner":"soundcloud","description":"An ActiveRecord database adapter that allows you to setup a \"master/slave\" environment","archived":true,"fork":true,"pushed_at":"2013-07-08T11:09:27.000Z","size":312,"stargazers_count":43,"open_issues_count":1,"forks_count":9,"subscribers_count":132,"default_branch":"master","last_synced_at":"2024-04-25T03:21:49.685Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"tcurdt/master_slave_adapter","license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/soundcloud.png","metadata":{"files":{"readme":"Readme.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2011-07-18T09:04:44.000Z","updated_at":"2023-01-27T20:56:22.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/soundcloud/master_slave_adapter","commit_stats":null,"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soundcloud%2Fmaster_slave_adapter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soundcloud%2Fmaster_slave_adapter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soundcloud%2Fmaster_slave_adapter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soundcloud%2Fmaster_slave_adapter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/soundcloud","download_url":"https://codeload.github.com/soundcloud/master_slave_adapter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234634974,"owners_count":18863978,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-26T11:42:21.657Z","updated_at":"2025-09-29T15:30:56.273Z","avatar_url":"https://github.com/soundcloud.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Replication Aware Master Slave Adapter [![Build Status](https://secure.travis-ci.org/soundcloud/master_slave_adapter.png)][6]\n\nImproved version of the [master_slave_adapter plugin][1], packaged as a gem.\n\n## Features\n\n1. automatic selection of master or slave connection: `with_consistency`\n2. manual selection of master or slave connection: `with_master`, `with_slave`\n3. handles master unavailable scenarios gracefully\n4. transaction callbacks: `on_commit`, `on_rollback`\n5. also:\n  * support for multiple slaves\n  * (partial) support for [database_cleaner][2]\n\n### Automatic Selection of Master or Slave\n\nThe adapter will run all reads against a slave database, unless a) the read is inside an open transaction or b) the\nadapter determines that the slave lags behind the master _relative to the last write_. For this to work, an initial\ninitial consistency requirement, a Clock, must be passed to the adapter. Based on this clock value, the adapter\ndetermines if a (randomly chosen) slave meets this requirement. If not, all statements are executed against master,\notherwise, the slave connection is used until either a transaction is opened or a write occurs. After a successful write\nor transaction, the adapter determines a new consistency requirement, which is returned and can be used for subsequent\noperations. Note that after a write or transaction, the adapter keeps using the master connection.\n\nAs an example, a Rails application could run the following function as an `around_filter`:\n\n```ruby\ndef with_consistency_filter\n  if logged_in?\n    clock = cached_clock_for(current_user)\n\n    new_clock = ActiveRecord::Base.with_consistency(clock) do\n      # inside the controller, ActiveRecord models can be used just as normal.\n      # The adapter will take care of choosing the right connection.\n      yield\n    end\n\n    [ new_clock, clock ].compact.max.tap do |c|\n      cache_clock_for(current_user, c)\n    end if new_clock != clock\n  else\n    # anonymous users will have to wait until the slaves have caught up\n    with_slave { yield }\n  end\nend\n```\n\nNote that we use the last seen consistency for a given user as reference point. This will give the user a recent view of the data,\npossibly reading from master, and if no write occurs inside the `with_consistency` block, we have a reasonable value to\ncache and reuse on subsequent requests.\nIf no cached clock is available, this indicates that no particular consistency is required. Any slave connection will do.\nSince `with_consistency` blocks can be nested, the controller code could later decide to require a more recent view on\nthe data.\n\n_See also this [blog post][3] for a more detailed explanation._\n\n### Manual Selection of Master or Slave\n\nThe original functionality of the adapter has been preserved:\n\n```ruby\nActiveRecord::Base.with_master do\n  # everything inside here will go to master\nend\n\nActiveRecord::Base.with_slave do\n  # everything inside here will go to one of the slaves\n  # opening a transaction or writing will switch to master\n  # for the rest of the block\nend\n```\n\n`with_master`, `with_slave` as well as `with_consistency` can be nested deliberately.\n\n### Handles master unavailable scenarios gracefully\n\nDue to scenarios when the master is possibly down (e.g., maintenance), we try\nto delegate as much as possible to the active slaves. In order to accomplish\nthis we have added the following functionalities.\n\n * We ignore errors while connecting to the master server.\n * ActiveRecord::MasterUnavailable exceptions are raised in cases when we need to use\n   a master connection, but the server is unavailable. This exception is propagated\n   to the application.\n * We have introduced the circuit breaker pattern in the master reconnect logic\n   to prevent excessive reconnection attempts. We block any queries which require\n   a master connection for a given timeout (by default, 30 seconds). After the\n   timeout has expired, any attempt of using the master connection will trigger\n   a reconnection.\n * The master slave adapter is still usable for any queries that require only\n   slave connections.\n\n### Transaction Callbacks\n\nThis feature was originally developed at [SoundCloud][4] for the standard `MysqlAdapter`. It allows arbitrary blocks of\ncode to be deferred for execution until the next transaction completes (or rolls back).\n\n```irb\nirb\u003e ActiveRecord::Base.on_commit { puts \"COMMITTED!\" }\nirb\u003e ActiveRecord::Base.on_rollback { puts \"ROLLED BACK!\" }\nirb\u003e ActiveRecord::Base.connection.transaction do\nirb*   # ...\nirb\u003e end\nCOMMITTED!\n=\u003e nil\nirb\u003e ActiveRecord::Base.connection.transaction do\nirb*   # ...\nirb*   raise \"failed operation\"\nirb\u003e end\nROLLED BACK!\n# stack trace omitted\n=\u003e nil\n```\n\nNote that a transaction callback will be fired only *once*, so you might want to do:\n\n```ruby\nclass MyModel\n  after_save do\n    connection.on_commit do\n      # ...\n    end\n  end\nend\n```\n\n### Support for Multiple Slaves\n\nThe adapter keeps a list of slave connections (see *Configuration*) and chooses randomly between them. The selection is\nmade at the beginning of a `with_slave` or `with_consistency` block and doesn't change until the block returns. Hence, a\nnested `with_slave` or `with_consistency` might run against a different slave.\n\n### Database Cleaner\n\nAt [SoundCloud][4], we're using [database_cleaner][2]'s 'truncation strategy' to wipe the database between [cucumber][5]\n'feature's. As our cucumber suite proved valuable while testing the `with_consistency` feature, we had to support\n`truncate_table` as an `ActiveRecord::Base.connection` instance method. We might add other strategies if there's enough\ninterest.\n\n## Requirements\n\nMasterSlaveAdapter requires ActiveRecord with a version \u003e= 2.3, is compatible\nwith at least Ruby 1.8.7, 1.9.2, 1.9.3 and comes with built-in support for mysql\nand mysql2 libraries.\n\nYou can check the versions it's tested against at [Travis CI](http://travis-ci.org/#!/soundcloud/master_slave_adapter).\n\n## Installation\n\nUsing plain rubygems:\n\n    $ gem install master_slave_adapter\n\nUsing bundler, just include it in your Gemfile:\n\n    gem 'master_slave_adapter'\n\n## Configuration\n\nExample configuration for the development environment in `database.yml`:\n\n```yaml\ndevelopment:\n  adapter: master_slave          # use master_slave adapter\n  connection_adapter: mysql      # actual adapter to use (only mysql is supported atm)\n  disable_connection_test: false # when an instance is checked out from the connection pool,\n                                 # we check if the connections are still alive, reconnecting if necessary\n\n  # these values are picked up as defaults in the 'master' and 'slaves' sections:\n  database: aweapp_development\n  username: aweappuser\n  password: s3cr3t\n\n  master:\n    host: masterhost\n    username: readwrite_user     # override default value\n\n  slaves:\n    - host: slave01\n    - host: slave02\n```\n\n## Testing\n\nYou can execute all tests against your current ruby version via:\n\n    rake spec\n\nIn case you have `rvm` installed, you can test against 1.8.7, 1.9.2 and 1.9.3 as well as ActiveRecord 2 and 3 via:\n\n    bash spec/all.sh\n\n## Credits\n\n* Maurício Lenhares - _original master_slave_adapter plugin_\n* Torsten Curdt     - _with_consistency, maintainership \u0026 open source licenses_\n* Sean Treadway     - _chief everything \u0026 transaction callbacks_\n* Kim Altintop      - _strong lax monoidal endofunctors_\n* Omid Aladini      - _chief operator \u0026 everything else_\n* Tiago Loureiro    - _review expert \u0026 master unavailable handling_\n* Tobias Schmidt    - _typo master \u0026 activerecord ranter_\n\n\n[1]: https://github.com/mauricio/master_slave_adapter\n[2]: https://github.com/bmabey/database_cleaner\n[3]: http://www.yourdailygeekery.com/2011/06/14/master-slave-consistency.html\n[4]: http://backstage.soundcloud.com\n[5]: http://cukes.info\n[6]: http://travis-ci.org/soundcloud/master_slave_adapter","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoundcloud%2Fmaster_slave_adapter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoundcloud%2Fmaster_slave_adapter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoundcloud%2Fmaster_slave_adapter/lists"}