{"id":15490826,"url":"https://github.com/hsgubert/rails-sharding","last_synced_at":"2025-04-14T00:38:28.314Z","repository":{"id":11623207,"uuid":"70149770","full_name":"hsgubert/rails-sharding","owner":"hsgubert","description":"Simple and robust sharding for Rails, including     Migrations and ActiveRecord extensions","archived":false,"fork":false,"pushed_at":"2022-05-10T23:26:17.000Z","size":107,"stargazers_count":41,"open_issues_count":4,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-27T14:52:26.112Z","etag":null,"topics":["activerecord","ruby-on-rails","shard-databases","sharding","shards"],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hsgubert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-10-06T11:59:29.000Z","updated_at":"2024-08-05T17:13:59.000Z","dependencies_parsed_at":"2022-08-08T21:30:14.677Z","dependency_job_id":null,"html_url":"https://github.com/hsgubert/rails-sharding","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hsgubert%2Frails-sharding","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hsgubert%2Frails-sharding/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hsgubert%2Frails-sharding/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hsgubert%2Frails-sharding/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hsgubert","download_url":"https://codeload.github.com/hsgubert/rails-sharding/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248803904,"owners_count":21164122,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activerecord","ruby-on-rails","shard-databases","sharding","shards"],"created_at":"2024-10-02T07:38:50.970Z","updated_at":"2025-04-14T00:38:28.289Z","avatar_url":"https://github.com/hsgubert.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Rails::Sharding\n\n[![Build Status](https://travis-ci.org/hsgubert/rails-sharding.svg?branch=master)](https://travis-ci.org/hsgubert/rails-sharding)\n[![Code Climate](https://codeclimate.com/github/hsgubert/rails-sharding/badges/gpa.svg)](https://codeclimate.com/github/hsgubert/rails-sharding)\n[![Test Coverage](https://codeclimate.com/github/hsgubert/rails-sharding/badges/coverage.svg)](https://codeclimate.com/github/hsgubert/rails-sharding/coverage)\n[![Gem Version](https://badge.fury.io/rb/rails-sharding.svg)](https://badge.fury.io/rb/rails-sharding)\n[![Dependency Status](https://gemnasium.com/badges/github.com/hsgubert/rails-sharding.svg)](https://gemnasium.com/github.com/hsgubert/rails-sharding)\n\nSimple and robust sharding gem for Rails, including Migrations and ActiveRecord extensions\n\nThis gems allows you to easily create extra databases to your rails application, and freely allocate ActiveRecord instances to any of the databases.\n\nAccessing shards is as simple as:\n\n```ruby\n  # creating a user to a specific shard\n  new_user = User.using_shard(:shard_group1, :shard1).create(username: 'x')\n\n  # retrieving a user from a specific shard\n  loaded_user = User.using_shard(:shard_group1, :shard1).where(username: 'x').first\n```\n\nYou can also use the block syntax:\n\n```ruby\n  Rails::Sharding.using_shard(:shard_group1, :shard1) do\n    # All statements inside this block will go to the selected shard\n\n    # Do some queries\n    new_user = User.create(username: 'x')\n    loaded_user = User.where(username: 'x').first\n    billing_infos = loaded_user.billing_infos.all\n  end\n```\n\nYou can also pick and choose which models will be shardable. Non shardable models will be retrieved from the master database, even if inside a `using_shard` block.\n\n## Compatibility\n\nGem version 1.x.x:\n\n- Rails 5.0, 5.1 and 5.2\n- Databases: MySQL, MariaDB, Postgres\n\nGem version 0.x.x:\n\n- Rails 4.2\n- Databases: MySQL, MariaDB\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'rails-sharding'\n```\n\nAnd then execute:\n\n```\nbundle\n```\n\n## Creating Shards\n\nTo start with the rails-sharding gem, run the command\n\n```\nrails g rails_sharding:scaffold\n```\n\nThis will generate a `config/shards.yml.example` like this:\n\n```ruby\ndefault: \u0026default\n  adapter: mysql2\n  encoding: utf8\n  reconnect: false\n  pool: 5\n  username: \u003c%= ENV[\"MYSQL_USERNAME\"] %\u003e\n  password: \u003c%= ENV[\"MYSQL_PASSWORD\"] %\u003e\n  socket: /var/run/mysqld/mysqld.sock\n\ndevelopment:\n  shard_group1:\n    shard1:\n      \u003c\u003c: *default\n      database: group1_shard1_development\n    shard2:\n      \u003c\u003c: *default\n      database: group1_shard2_development\n...\n```\n\nRename it to `config/shards.yml` and change it to your database configuration. This example file defines a single shard group (named `shard_group1`) containing two shards (`shard1` and `shard2`).\n\n**A shard group is a set of shards that should have the same schema.**\n\nWhen you're ready to create the shards run\n\n```\nrake shards:create\n```\n\n## Migrating Shards\n\nGo to the directory `db/shards_migrations/shard_group1` and add all migrations that you want to run on the shards of `shard_group1`. By design, all shards in a same group should always have the same schema.\n\nAs of now, there is no generator for migrations. You can use the regular rails generator and move the migrations to the `shards_migration` folder.\n\nFor example, add the following migration to your `db/shards_migrations/shard_group1`:\n\n```ruby\n# 20160808000000_create_users.rb\nclass CreateClients \u003c ActiveRecord::Migration[5.0]\n  def up\n    create_table :users do |t|\n      t.string :username, :limit =\u003e 100\n      t.timestamps\n    end\n  end\n\n  def down\n    drop_table :users\n  end\nend\n```\n\nThen run:\n\n```\nrake shards:migrate\n```\n\nAll the shards will be migrated, and one schema file will be dumped for each of the shards (just like rails would do for your master database). You can see the schema of the shards in `db/shards_schemas/shard_group1/`, and it will be something like:\n\n```ruby\nActiveRecord::Schema.define(version: 20160808000000) do\n\n  create_table \"users\", force: :cascade do |t|\n    t.string   \"username\",       limit: 100\n    t.datetime \"created_at\"\n    t.datetime \"updated_at\"\n  end\n\nend\n```\n\n## Other rake tasks\n\nThe rails-sharding gem offers several rake tasks analogous to the ones offered by ActiveRecord:\n\n```\nrake shards:create\nrake shards:drop\nrake shards:migrate\nrake shards:migrate:down\nrake shards:migrate:redo\nrake shards:migrate:reset\nrake shards:migrate:up\nrake shards:rollback\nrake shards:schema:dump\nrake shards:schema:load\nrake shards:test:load_schema\nrake shards:test:prepare\nrake shards:test:purge\nrake shards:version\n```\n\nThey work just the same as the tasks `rake:db:...` but they operate on all shards of all shard groups. If you want to run a rake task just to a specific shard group or shard you can use the `SHARD_GROUP` and `SHARD` options:\n\n```\nrake shards:migrate SHARD_GROUP=shard_group_1\nrake shards:migrate SHARD_GROUP=shard_group_1 SHARD=shard1\n```\n\n## Gem Options\n\nRunning the `rails g rails_sharding:scaffold` will create an initializer at `config/initializers/rails-sharding.rb`. You can pass additional configurations on this initializer to control the gem behavior. You can see below all available options and their default values:\n\n```ruby\n# config/initializers/rails-sharding.rb\n\nRails::Sharding.setup do |config|\n  # If true one connection will be established per shard (in every shard group) on startup.\n  # If false the user must call Shards::ConnectionHandler.establish_connection(shard_group, shard_name) manually at least once before using each shard.\n  config.establish_all_connections_on_setup = true\n\n  # If true the method #using_shard will be mixed in ActiveRecord scopes. Put this to false if you don't want the gem to modify ActiveRecord\n  config.extend_active_record_scope = true\n\n  # If true the query logs of ActiveRecord will be tagged with the corresponding shard you're querying\n  config.add_shard_tag_to_query_logs = true\n\n  # Specifies where to find the definition of the shards configurations\n  config.shards_config_file = 'config/shards.yml'\n\n  # Specifies where to find the migrations for each shard group\n  config.shards_migrations_dir = 'db/shards_migrations'\n\n  # Specifies where to find the schemas for each shard group\n  config.shards_schemas_dir = 'db/shards_schemas'\nend\n```\n\n## Wiki\n\nWant to know more? How to integrate with RSpec, Capistrano, etc? Take a look at our [wiki](https://github.com/hsgubert/rails-sharding/wiki).\n\n## Development and Contributing\n\nAfter checking out the repo:\n\n1. Run `bundle` to install gems\n\n1. Create your `spec/fixtures/shards.yml` based on the example on this same folder (you need MySQL and Postgres)\n\n1. Create your `.env` based on the example on this same folder.\n\n1. Run `rake db:test:prepare` to create the test shards.\n\n1. Run `rspec` to run the tests.\n\nBug reports and pull requests are welcome on GitHub at https://github.com/hsgubert/rails-sharding.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).\n\n## Acknowledgements\n\nThis gem was inspired on several other gems like: [octopus](https://github.com/thiagopradi/octopus), [shard_handler](https://github.com/locaweb/shard_handler) and [active_record_shards](https://github.com/zendesk/active_record_shards).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhsgubert%2Frails-sharding","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhsgubert%2Frails-sharding","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhsgubert%2Frails-sharding/lists"}