{"id":13695817,"url":"https://github.com/PNixx/clickhouse-activerecord","last_synced_at":"2025-05-03T13:33:28.701Z","repository":{"id":37444851,"uuid":"101983502","full_name":"PNixx/clickhouse-activerecord","owner":"PNixx","description":"A Ruby database ActiveRecord driver for ClickHouse","archived":false,"fork":false,"pushed_at":"2024-10-23T13:24:41.000Z","size":309,"stargazers_count":199,"open_issues_count":24,"forks_count":100,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-10T22:31:45.890Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PNixx.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-08-31T09:23:00.000Z","updated_at":"2024-11-05T04:31:16.000Z","dependencies_parsed_at":"2023-02-10T08:15:33.102Z","dependency_job_id":"301a5916-be21-48aa-af18-898da3c312f6","html_url":"https://github.com/PNixx/clickhouse-activerecord","commit_stats":null,"previous_names":[],"tags_count":36,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PNixx%2Fclickhouse-activerecord","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PNixx%2Fclickhouse-activerecord/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PNixx%2Fclickhouse-activerecord/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PNixx%2Fclickhouse-activerecord/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PNixx","download_url":"https://codeload.github.com/PNixx/clickhouse-activerecord/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224364395,"owners_count":17299065,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T18:00:33.792Z","updated_at":"2025-05-03T13:33:28.693Z","avatar_url":"https://github.com/PNixx.png","language":"Ruby","funding_links":[],"categories":["Ruby","Language bindings"],"sub_categories":["Ruby"],"readme":"# Clickhouse::Activerecord\n\nA Ruby database ActiveRecord driver for ClickHouse. Support Rails \u003e= 7.1.\nSupport ClickHouse version from 22.0 LTS.\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'clickhouse-activerecord'\n```\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n    $ gem install clickhouse-activerecord\n    \n## Available database connection parameters\n```yml\ndefault: \u0026default\n  adapter: clickhouse\n  database: database\n  host: localhost\n  port: 8123\n  username: username\n  password: password\n  ssl: true # optional for using ssl connection\n  debug: true # use for showing in to log technical information\n  migrations_paths: db/clickhouse # optional, default: db/migrate_clickhouse\n  cluster_name: 'cluster_name' # optional for creating tables in cluster \n  replica_name: '{replica}' # replica macros name, optional for creating replicated tables\n  read_timeout: 300 # change network timeouts, by default 60 seconds\n  write_timeout: 300\n  keep_alive_timeout: 300\n```\n\nAlternatively if you wish to pass a custom `Net::HTTP` transport (or any other\nobject which supports a `.post()` function with the same parameters as\n`Net::HTTP`'s), you can do this directly instead of specifying\n`host`/`port`/`ssl`:\n\n```ruby\nclass ActionView \u003c ActiveRecord::Base\n  establish_connection(\n    adapter: 'clickhouse',\n    database: 'database',\n    connection: Net::HTTP.start('http://example.org', 8123)\n  )\nend\n```\n\n## Usage in Rails\n\nAdd your `database.yml` connection information with postfix `_clickhouse` for you environment:\n\n```yml\ndevelopment:\n  adapter: clickhouse\n  database: database\n```\n\nYour model example:\n\n```ruby\nclass Action \u003c ActiveRecord::Base\nend\n```\n\nFor materialized view model add:\n```ruby\nclass ActionView \u003c ActiveRecord::Base\n  self.is_view = true\nend\n```\n\n## Usage in Rails with second database\n\nAdd your `database.yml` connection information for you environment:\n\n```yml\ndevelopment:\n  primary:\n    ...\n    \n  clickhouse:\n    adapter: clickhouse\n    database: database\n```\n\nConnection [Multiple Databases with Active Record](https://guides.rubyonrails.org/active_record_multiple_databases.html) or short example:\n\n```ruby\nclass Action \u003c ActiveRecord::Base\n  establish_connection :clickhouse\nend\n```\n\n### Rake tasks\n\nCreate / drop / purge / reset database:\n \n    $ rake db:create\n    $ rake db:drop\n    $ rake db:purge\n    $ rake db:reset\n\nOr with multiple databases:\n\n    $ rake db:create:clickhouse\n    $ rake db:drop:clickhouse\n    $ rake db:purge:clickhouse\n    $ rake db:reset:clickhouse\n    \nMigration:\n\n    $ rails g clickhouse_migration MIGRATION_NAME COLUMNS\n    $ rake db:migrate\n    $ rake db:rollback\n\n### Dump / Load for multiple using databases\n\nIf you using multiple databases, for example: PostgreSQL, Clickhouse.\n\nSchema dump to `db/clickhouse_schema.rb` file:\n\n    $ rake db:schema:dump:clickhouse\n    \nSchema load from `db/clickhouse_schema.rb` file:\n\n    $ rake db:schema:load:clickhouse\n\nFor export schema to PostgreSQL, you need use:\n\n    $ rake clickhouse:schema:dump -- --simple\n\nSchema will be dump to `db/clickhouse_schema_simple.rb`. If default file exists, it will be auto update after migration.\n    \nStructure dump to `db/clickhouse_structure.sql` file:\n\n    $ rake clickhouse:structure:dump\n    \nStructure load from `db/clickhouse_structure.sql` file:\n\n    $ rake clickhouse:structure:load\n\n### Dump / Load for only Clickhouse database using\n\n    $ rake db:schema:dump  \n    $ rake db:schema:load  \n    $ rake db:structure:dump  \n    $ rake db:structure:load\n\n### RSpec\n\nFor auto truncate tables before each test add to `spec/rails_helper.rb` file:\n\n```\nrequire 'clickhouse-activerecord/rspec'\n```\n    \n### Insert and select data\n\n```ruby\nAction.where(url: 'http://example.com', date: Date.current).where.not(name: nil).order(created_at: :desc).limit(10)\n# Clickhouse Action Load (10.3ms)  SELECT actions.* FROM actions WHERE actions.date = '2017-11-29' AND actions.url = 'http://example.com' AND (actions.name IS NOT NULL)  ORDER BY actions.created_at DESC LIMIT 10\n#=\u003e #\u003cActiveRecord::Relation [#\u003cAction *** \u003e]\u003e\n\nAction.create(url: 'http://example.com', date: Date.yesterday)\n# Clickhouse Action Load (10.8ms)  INSERT INTO actions (url, date) VALUES ('http://example.com', '2017-11-28')\n#=\u003e true\n \nActionView.maximum(:date)\n# Clickhouse (10.3ms)  SELECT maxMerge(actions.date) FROM actions\n#=\u003e 'Wed, 29 Nov 2017'\n\nAction.where(date: Date.current).final.limit(10)\n# Clickhouse Action Load (10.3ms)  SELECT actions.* FROM actions FINAL WHERE actions.date = '2017-11-29' LIMIT 10\n#=\u003e #\u003cActiveRecord::Relation [#\u003cAction *** \u003e]\u003e\n\nAction.settings(optimize_read_in_order: 1).where(date: Date.current).limit(10)\n# Clickhouse Action Load (10.3ms)  SELECT actions.* FROM actions FINAL WHERE actions.date = '2017-11-29' LIMIT 10 SETTINGS optimize_read_in_order = 1\n#=\u003e #\u003cActiveRecord::Relation [#\u003cAction *** \u003e]\u003e\n\nUser.joins(:actions).using(:group_id)\n# Clickhouse User Load (10.3ms)  SELECT users.* FROM users INNER JOIN actions USING group_id\n#=\u003e #\u003cActiveRecord::Relation [#\u003cUser *** \u003e]\u003e\n\nUser.window('x', order: 'date', partition: 'name', rows: 'UNBOUNDED PRECEDING').select('sum(value) OVER x')\n# SELECT sum(value) OVER x FROM users WINDOW x AS (PARTITION BY name ORDER BY date ROWS UNBOUNDED PRECEDING)\n#=\u003e #\u003cActiveRecord::Relation [#\u003cUser *** \u003e]\u003e\n```\n\n\n### Migration Data Types\n\nInteger types are unsigned by default. Specify signed values with `:unsigned =\u003e\nfalse`. The default integer is `UInt32`\n\n| Type (bit size) |                    Range                     | :limit (byte size) |\n|:----------------|:--------------------------------------------:|-------------------:|\n| Int8            |                 -128 to 127                  |                  1 | \n| Int16           |               -32768 to 32767                |                  2 |\n| Int32           |         -2147483648 to 2,147,483,647         |                3,4 |\n| Int64           | -9223372036854775808 to 9223372036854775807] |            5,6,7,8 |\n| Int128          |                     ...                      |             9 - 15 |\n| Int256          |                     ...                      |                16+ |\n| UInt8           |                   0 to 255                   |                  1 |\n| UInt16          |                 0 to 65,535                  |                  2 |\n| UInt32          |              0 to 4,294,967,295              |                3,4 |\n| UInt64          |          0 to 18446744073709551615           |            5,6,7,8 |\n| UInt256         |                   0 to ...                   |                 8+ |\n| Array           |                     ...                      |                ... |\n| Map             |                     ...                      |                ... |\n\nExample:\n\n```ruby\nclass CreateDataItems \u003c ActiveRecord::Migration[7.1]\n  def change\n    create_table \"data_items\", id: false, options: \"VersionedCollapsingMergeTree(sign, version) PARTITION BY toYYYYMM(day) ORDER BY category\", force: :cascade do |t|\n      t.date \"day\", null: false\n      t.string \"category\", null: false\n      t.integer \"value_in\", null: false\n      t.integer \"sign\", limit: 1, unsigned: false, default: -\u003e { \"CAST(1, 'Int8')\" }, null: false\n      t.integer \"version\", limit: 8, default: -\u003e { \"CAST(toUnixTimestamp(now()), 'UInt64')\" }, null: false\n    end\n    \n    create_table \"with_index\", id: false, options: 'MergeTree PARTITION BY toYYYYMM(date) ORDER BY (date)' do |t|\n      t.integer :int1, null: false\n      t.integer :int2, null: false\n      t.date :date, null: false\n\n      t.index '(int1 * int2, date)', name: 'idx', type: 'minmax', granularity: 3\n    end\n    \n    remove_index :some, 'idx'\n    \n    add_index :some, 'int1 * int2', name: 'idx2', type: 'set(10)', granularity: 4\n  end\nend\n```\n\nCreate table with custom column structure and codec compression:\n\n```ruby\nclass CreateDataItems \u003c ActiveRecord::Migration[7.1]\n  def change\n    create_table \"data_items\", id: false, options: \"MergeTree PARTITION BY toYYYYMM(timestamp) ORDER BY timestamp\", force: :cascade do |t|\n      t.integer :user_id, limit: 8, codec: 'DoubleDelta, LZ4'\n      t.column \"timestamp\", \"DateTime('UTC') CODEC(DoubleDelta, LZ4)\"\n    end\n  end\nend\n```\n\nCreate Buffer table with connection database name:\n\n```ruby\nclass CreateDataItems \u003c ActiveRecord::Migration[7.1]\n  def change\n    create_table :some_buffers, as: :some, options: \"Buffer(#{connection.database}, some, 1, 10, 60, 100, 10000, 10000000, 100000000)\"\n  end\nend\n```\n\n\n### Using replica and cluster params in connection parameters\n\n```yml\ndefault: \u0026default\n  ***\n  cluster_name: 'cluster_name'\n  replica_name: '{replica}'\n```\n\n`ON CLUSTER cluster_name` will be attach to all queries create / drop.\n\nEngines `MergeTree` and all support replication engines will be replaced to `Replicated***('/clickhouse/tables/cluster_name/database.table', '{replica}')`\n\n## Donations\n\nDonations to this project are going directly to [PNixx](https://github.com/PNixx), the original author of this project:\n\n* BTC address: `1H3rhpf7WEF5JmMZ3PVFMQc7Hm29THgUfN`\n* ETH address: `0x6F094365A70fe7836A633d2eE80A1FA9758234d5`\n* XMR address: `42gP71qLB5M43RuDnrQ3vSJFFxis9Kw9VMURhpx9NLQRRwNvaZRjm2TFojAMC8Fk1BQhZNKyWhoyJSn5Ak9kppgZPjE17Zh`\n* TON address: `UQBt0-s1igIpJoEup0B1yAUkZ56rzbpruuAjNhQ26MVCaNlC`\n\n## Development\n\nAfter checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\nTesting github actions:\n\n```bash\nact\n```\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at [https://github.com/pnixx/clickhouse-activerecord](https://github.com/pnixx/clickhouse-activerecord). This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPNixx%2Fclickhouse-activerecord","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPNixx%2Fclickhouse-activerecord","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPNixx%2Fclickhouse-activerecord/lists"}