{"id":13695833,"url":"https://github.com/shlima/click_house","last_synced_at":"2025-05-16T16:01:48.228Z","repository":{"id":35997969,"uuid":"220491817","full_name":"shlima/click_house","owner":"shlima","description":"Modern Ruby database driver for ClickHouse","archived":false,"fork":false,"pushed_at":"2024-06-18T10:12:56.000Z","size":313,"stargazers_count":183,"open_issues_count":12,"forks_count":25,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-12-18T12:42:57.893Z","etag":null,"topics":["clickhouse","gem","ruby"],"latest_commit_sha":null,"homepage":"https://clickhouse.yandex/docs/en/","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shlima.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-08T15:11:37.000Z","updated_at":"2024-10-29T09:22:24.000Z","dependencies_parsed_at":"2023-02-10T17:15:20.104Z","dependency_job_id":"b553b073-d87f-4136-b3a9-132dd2d96981","html_url":"https://github.com/shlima/click_house","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shlima%2Fclick_house","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shlima%2Fclick_house/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shlima%2Fclick_house/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shlima%2Fclick_house/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shlima","download_url":"https://codeload.github.com/shlima/click_house/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248578875,"owners_count":21127714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clickhouse","gem","ruby"],"created_at":"2024-08-02T18:00:33.945Z","updated_at":"2025-04-12T14:18:40.736Z","avatar_url":"https://github.com/shlima.png","language":"Ruby","funding_links":[],"categories":["Language bindings"],"sub_categories":["Ruby"],"readme":"![](./doc/logo.svg?sanitize=true)\n\n# ClickHouse Ruby driver\n\n![CI](https://github.com/shlima/click_house/workflows/CI/badge.svg)\n[![Code Climate](https://codeclimate.com/github/shlima/click_house/badges/gpa.svg)](https://codeclimate.com/github/shlima/click_house)\n[![Gem Version](https://badge.fury.io/rb/click_house.svg)](https://badge.fury.io/rb/click_house)\n\n```bash\ngem install click_house\n```\n\nA modern Ruby database driver for ClickHouse. [ClickHouse](https://clickhouse.yandex)\nis a high-performance column-oriented database management system developed by\n[Yandex](https://yandex.com/company) which operates Russia's most popular search engine.\n\n\u003e This development was inspired by currently [unmaintainable alternative](https://github.com/archan937/clickhouse)\n\u003e but rewritten and well tested\n\n### Why use the HTTP interface and not the TCP interface?\n\nWell, the developers of ClickHouse themselves [discourage](https://github.com/yandex/ClickHouse/issues/45#issuecomment-231194134) using the TCP interface.\n\n\u003e TCP transport is more specific, we don't want to expose details.\nDespite we have full compatibility of protocol of different versions of client and server, we want to keep the ability to \"break\" it for very old clients. And that protocol is not too clean to make a specification.\n\nYandex uses HTTP interface for working from Java and Perl, Python and Go as well as shell scripts.\n\n# TOC\n\n* [Configuration](#configuration)\n* [Usage](#usage)\n* [Queries](#queries)\n* [Insert](#insert)\n* [Create a table](#create-a-table)\n* [Alter table](#alter-table)\n* [Type casting](#type-casting)\n* [Using with a connection pool](#using-with-a-connection-pool)\n* [Using with Rails](#using-with-rails)\n* [Using with ActiveRecord](#using-with-activerecord)\n* [Using with RSpec](#using-with-rspec)\n* [Development](#development)\n\n## Configuration\n\n```ruby\nClickHouse.config do |config|\n  config.logger = Logger.new(STDOUT)\n  config.adapter = :net_http\n  config.database = 'metrics'\n  config.url = 'http://localhost:8123'\n  config.timeout = 60\n  config.open_timeout = 3\n  config.ssl_verify = false\n  # set to true to symbolize keys for SELECT and INSERT statements (type casting)\n  config.symbolize_keys = false\n  config.headers = {}\n\n  # or provide connection options separately\n  config.scheme = 'http'\n  config.host = 'localhost'\n  config.port = 'port'\n\n  # if you use HTTP basic Auth\n  config.username = 'user'\n  config.password = 'password'\n\n  # if you want to add settings to all queries\n  config.global_params = { mutations_sync: 1 }\n  \n  # choose a ruby JSON parser (default one)\n  config.json_parser = ClickHouse::Middleware::ParseJson\n  # or Oj parser\n  config.json_parser = ClickHouse::Middleware::ParseJsonOj\n\n  # JSON.dump (default one)\n  config.json_serializer = ClickHouse::Serializer::JsonSerializer\n  # or Oj.dump\n  config.json_serializer = ClickHouse::Serializer::JsonOjSerializer\nend\n```\n\nAlternative, you can assign configuration parameters via a hash\n\n```ruby\nClickHouse.config.assign(logger: Logger.new(STDOUT))\n```\n\nNow you are able to communicate with ClickHouse:\n\n```ruby\nClickHouse.connection.ping #=\u003e true\n```\nYou can easily build a new raw connection and override any configuration parameter\n(such as database name, connection address)\n\n```ruby\n@connection = ClickHouse::Connection.new(ClickHouse::Config.new(logger: Rails.logger))\n@connection.ping\n```\n\n## Usage\n\n```ruby\nClickHouse.connection.ping #=\u003e true\nClickHouse.connection.replicas_status #=\u003e true\n\nClickHouse.connection.databases #=\u003e [\"default\", \"system\"]\nClickHouse.connection.create_database('metrics', if_not_exists: true, engine: nil, cluster: nil)\nClickHouse.connection.drop_database('metrics', if_exists: true, cluster: nil)\n\nClickHouse.connection.tables #=\u003e [\"visits\"]\nClickHouse.connection.describe_table('visits') #=\u003e [{\"name\"=\u003e\"id\", \"type\"=\u003e\"FixedString(16)\", \"default_type\"=\u003e\"\"}]\nClickHouse.connection.table_exists?('visits', temporary: nil) #=\u003e true\nClickHouse.connection.drop_table('visits', if_exists: true, temporary: nil, cluster: nil)\nClickHouse.connection.create_table(*) # see \u003cCreate a table\u003e section\nClickHouse.connection.truncate_table('name', if_exists: true, cluster: nil)\nClickHouse.connection.truncate_tables(['table_1', 'table_2'], if_exists: true, cluster: nil)\nClickHouse.connection.truncate_tables # will truncate all tables in database\nClickHouse.connection.rename_table('old_name', 'new_name', cluster: nil)\nClickHouse.connection.rename_table(%w[table_1 table_2], %w[new_1 new_2], cluster: nil)\nClickHouse.connection.alter_table('table', 'DROP COLUMN user_id', cluster: nil)\nClickHouse.connection.add_index('table', 'ix', 'has(b, a)', type: 'minmax', granularity: 2, cluster: nil)\nClickHouse.connection.drop_index('table', 'ix', cluster: nil)\n\nClickHouse.connection.select_all('SELECT * FROM visits')\nClickHouse.connection.select_one('SELECT * FROM visits LIMIT 1')\nClickHouse.connection.select_value('SELECT ip FROM visits LIMIT 1')\nClickHouse.connection.explain('SELECT * FROM visits CROSS JOIN visits')\n```\n\n## Queries\n### Select All\n\nSelect all type-casted result set\n\n```ruby\n@result = ClickHouse.connection.select_all('SELECT * FROM visits')\n\n# all enumerable methods are delegated like #each, #map, #select etc\n# results of #to_a is TYPE CASTED\n@result.to_a #=\u003e [{\"date\"=\u003e#\u003cDate: 2000-01-01\u003e, \"id\"=\u003e1}]\n\n# raw results (WITHOUT type casting)\n# much faster if selecting a large amount of data\n@result.data #=\u003e [{\"date\"=\u003e\"2000-01-01\", \"id\"=\u003e1}, {\"date\"=\u003e\"2000-01-02\", \"id\"=\u003e2}]\n\n# you can access raw data\n@result.meta #=\u003e [{\"name\"=\u003e\"date\", \"type\"=\u003e\"Date\"}, {\"name\"=\u003e\"id\", \"type\"=\u003e\"UInt32\"}]\n@result.statistics #=\u003e {\"elapsed\"=\u003e0.0002271, \"rows_read\"=\u003e2, \"bytes_read\"=\u003e12}\n@result.summary #=\u003e ClickHouse::Response::Summary\n@result.headers #=\u003e {\"x-clickhouse-query-id\"=\u003e\"9bf5f604-31fc-4eff-a4b5-277f2c71d199\"}\n@result.types #=\u003e [Hash\u003cString|Symbol, ClickHouse::Ast::Statement\u003e]\n```\n\n### Select Value\n\nSelect value returns exactly one type-casted value\n\n```ruby\nClickHouse.connection.select_value('SELECT COUNT(*) from visits') #=\u003e 0\nClickHouse.connection.select_value(\"SELECT toDate('2019-01-01')\") #=\u003e #\u003cDate: 2019-01-01\u003e\nClickHouse.connection.select_value(\"SELECT toDateOrZero(NULL)\") #=\u003e nil\n```\n\n### Select One\n\nReturns a record hash with the column names as keys and column values as values.\n\n```ruby\nClickHouse.connection.select_one('SELECT date, SUM(id) AS sum FROM visits GROUP BY date')\n#=\u003e {\"date\"=\u003e#\u003cDate: 2000-01-01\u003e, \"sum\"=\u003e1}\n```\n\n### Execute Raw SQL\n\nBy default, gem provides parser for `JSON` and `CSV` response formats. Type conversion\navailable for the `JSON`.\n\n```ruby\n# format not specified\nresponse = ClickHouse.connection.execute \u003c\u003c~SQL\n  SELECT count(*) AS counter FROM rspec\nSQL\n\nresponse.body #=\u003e \"2\\n\"\n\n# JSON\nresponse = ClickHouse.connection.execute \u003c\u003c~SQL\n  SELECT count(*) AS counter FROM rspec FORMAT JSON\nSQL\n\nresponse.body #=\u003e {\"meta\"=\u003e[{\"name\"=\u003e\"counter\", \"type\"=\u003e\"UInt64\"}], \"data\"=\u003e[{\"counter\"=\u003e\"2\"}], \"rows\"=\u003e1, \"statistics\"=\u003e{\"elapsed\"=\u003e0.0002412, \"rows_read\"=\u003e2, \"bytes_read\"=\u003e4}}\n\n# CSV\nresponse = ClickHouse.connection.execute \u003c\u003c~SQL\n  SELECT count(*) AS counter FROM rspec FORMAT CSV\nSQL\n\nresponse.body #=\u003e [[\"2\"]]\n\n# You may use any format supported by ClickHouse\nresponse = ClickHouse.connection.execute \u003c\u003c~SQL\n  SELECT count(*) AS counter FROM rspec FORMAT RowBinary\nSQL\n\nresponse.body #=\u003e \"\\u0002\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\\u0000\"\n```\n\n## Insert\n\nWhen column names and values are transferred separately, data sends to the server \nusing `JSONCompactEachRow` format by default.\n\n```ruby\nClickHouse.connection.insert('table', columns: %i[id name]) do |buffer|\n  buffer \u003c\u003c [1, 'Mercury']\n  buffer \u003c\u003c [2, 'Venus']\nend\n\n# or\nClickHouse.connection.insert('table', columns: %i[id name], values: [[1, 'Mercury'], [2, 'Venus']])\n```\n\nWhen rows are passed as an Array or a Hash, data sends to the server\nusing `JSONEachRow` format by default.\n\n```ruby\nClickHouse.connection.insert('table', [{ name: 'Sun', id: 1 }, { name: 'Moon', id: 2 }])\n\n# or\nClickHouse.connection.insert('table', { name: 'Sun', id: 1 })\n\n# for ruby \u003c 3.0 provide an extra argument\nClickHouse.connection.insert('table', { name: 'Sun', id: 1 }, {})\n\n# or\nClickHouse.connection.insert('table') do |buffer|\n  buffer \u003c\u003c { name: 'Sun', id: 1 }\n  buffer \u003c\u003c { name: 'Moon', id: 2 }\nend\n```\n\nSometimes it's needed to use other format than `JSONEachRow` For example if you want to send BigDecimal's \nyou could use `JSONStringsEachRow` format so string representation of `BigDecimal` will be parsed:\n\n```ruby\nClickHouse.connection.insert('table', { name: 'Sun', id: '1' }, format: 'JSONStringsEachRow')\n# or\nClickHouse.connection.insert_rows('table', { name: 'Sun', id: '1' }, format: 'JSONStringsEachRow')\n# or\nClickHouse.connection.insert_compact('table', columns: %w[name id], values: %w[Sun 1], format: 'JSONCompactStringsEachRow')\n```\n\nSee the [type casting](#type-casting) section to insert the data in a proper way.\n\n## Create a table\n### Create table using DSL\n\n```ruby\nClickHouse.connection.create_table('visits', if_not_exists: true, engine: 'MergeTree(date, (year, date), 8192)') do |t|\n  t.FixedString :id, 16\n  t.UInt16      :year, low_cardinality: true\n  t.Date        :date\n  t.DateTime    :time, 'UTC'\n  t.Decimal     :money, 5, 4\n  t.String      :event\n  t.UInt32      :user_id\n  t.IPv4        :ipv4\n  t.IPv6        :ipv6\nend\n```\n\n### Create nullable columns\n\n```ruby\nClickHouse.connection.create_table('visits', engine: 'TinyLog') do |t|\n  t.UInt16 :id, 16, nullable: true\nend\n```\n\n### Set column options\n\n```ruby\nClickHouse.connection.create_table('visits', engine: 'MergeTree(date, (year, date), 8192)') do |t|\n  t.UInt16  :year\n  t.Date    :date\n  t.UInt16  :id, 16, default: 0, ttl: 'date + INTERVAL 1 DAY'\nend\n```\n\n### Define column with custom SQL\n\n```ruby\nClickHouse.connection.create_table('visits', engine: 'TinyLog') do |t|\n  t \u003c\u003c \"vendor Enum('microsoft' = 1, 'apple' = 2)\"\n  t \u003c\u003c \"tags Array(String)\"\nend\n```\n\n### Define nested structures\n\n```ruby\nClickHouse.connection.create_table('visits', engine: 'TinyLog') do |t|\n  t.UInt8 :id\n  t.Nested :json do |n|\n    n.UInt8 :cid\n    n.Date  :created_at\n    n.Date  :updated_at\n  end\nend\n```\n\n### Set table options\n\n```ruby\nClickHouse.connection.create_table('visits',\n  order: 'year',\n  ttl: 'date + INTERVAL 1 DAY',\n  sample: 'year',\n  settings: 'index_granularity=8192',\n  primary_key: 'year',\n  engine: 'MergeTree') do |t|\n  t.UInt16  :year\n  t.Date    :date\nend\n```\n\n### Create table with raw SQL\n\n```ruby\nClickHouse.connection.execute \u003c\u003c~SQL\n  CREATE TABLE visits(int Nullable(Int8), date Nullable(Date)) ENGINE TinyLog\nSQL\n```\n\n## Alter table\n### Alter table with DSL\n```ruby\nClickHouse.connection.add_column('table', 'column_name', :UInt64, default: nil, if_not_exists: nil, after: nil, cluster: nil)\nClickHouse.connection.drop_column('table', 'column_name', if_exists: nil, cluster: nil)\nClickHouse.connection.clear_column('table', 'column_name', partition: 'partition_name', if_exists: nil, cluster: nil)\nClickHouse.connection.modify_column('table', 'column_name', type: :UInt64, default: nil, if_exists: false, cluster: nil)\n```\n\n### Alter table with SQL\n\n```ruby\n# By SQL in argument\nClickHouse.connection.alter_table('table', 'DROP COLUMN user_id', cluster: nil)\n\n# By SQL in a block\nClickHouse.connection.alter_table('table', cluster: nil) do\n  \u003c\u003c~SQL\n    MOVE PART '20190301_14343_16206_438' TO VOLUME 'slow'\n  SQL\nend\n```\n\n## Type casting\n\nBy default gem provides all necessary type casting, but you may overwrite or define\nyour own logic. if you need to redefine all built-in types with your implementation,\njust clear the default type system:\n\n```ruby\nClickHouse.types.clear\nClickHouse.types # =\u003e {}\nClickHouse.types.default #=\u003e #\u003cClickHouse::Type::UndefinedType\u003e\n```\n\nType casting works automatically when fetching data, when inserting data, you must serialize the types yourself\n\n```sql\nCREATE TABLE assets(visible Boolean, tags Array(Nullable(String))) ENGINE Memory\n```\n\n```ruby\n# cache table schema in a class variable\n@schema = ClickHouse.connection.table_schema('assets')\n\n# Json each row\nClickHouse.connection.insert('assets', @schema.serialize({'visible' =\u003e true, 'tags' =\u003e ['ruby']}))\n\n# Json compact\nClickHouse.connection.insert('assets', columns: %w[visible tags]) do |buffer|\n  buffer \u003c\u003c [\n    @schema.serialize_column(\"visible\", true),\n    @schema.serialize_column(\"tags\", ['ruby']),\n  ]\nend\n```\n\n## Using with a connection pool\n\n```ruby\nrequire 'connection_pool'\n\nClickHouse.connection = ConnectionPool.new(size: 2) do\n  ClickHouse::Connection.new(ClickHouse::Config.new(url: 'http://replica.example.com'))\nend\n\nClickHouse.connection.with do |conn|\n  conn.tables\nend\n```\n\n## Using with Rails\n\n```yml\n# config/click_house.yml\n\ndefault: \u0026default\n  url: http://localhost:8123\n  timeout: 60\n  open_timeout: 3\n\ndevelopment:\n  database: ecliptic_development\n  \u003c\u003c: *default\n\ntest:\n  database: ecliptic_test\n  \u003c\u003c: *default\n\nproduction:\n  \u003c\u003c: *default\n  database: ecliptic_production\n```\n\n```ruby\n# config/initializers/click_house.rb\n\nClickHouse.config do |config|\n  config.logger = Rails.logger\n  config.assign(Rails.application.config_for('click_house'))\nend\n```\n\n```ruby\n# lib/tasks/click_house.rake\nnamespace :click_house do\n  task prepare: :environment do\n    @environments = Rails.env.development? ? %w[development test] : [Rails.env]\n  end\n\n  task drop: :prepare do\n    @environments.each do |env|\n      config = ClickHouse.config.clone.assign(Rails.application.config_for('click_house', env: env))\n      connection = ClickHouse::Connection.new(config)\n      connection.drop_database(config.database, if_exists: true)\n    end\n  end\n\n  task create: :prepare do\n    @environments.each do |env|\n      config = ClickHouse.config.clone.assign(Rails.application.config_for('click_house', env: env))\n      connection = ClickHouse::Connection.new(config)\n      connection.create_database(config.database, if_not_exists: true)\n    end\n  end\nend\n```\n\nPrepare the ClickHouse database:\n\n```bash\nrake click_house:drop click_house:create\n```\n\nIf your are using SQL Database in Rails, you can manage ClickHouse migrations\nusing `ActiveRecord::Migration` mechanism\n\n```ruby\nclass CreateAdvertVisits \u003c ActiveRecord::Migration[6.0]\n  def up\n    ClickHouse.connection.create_table('visits', engine: 'MergeTree(date, (account_id, advert_id), 512)') do |t|\n      t.UInt16   :account_id\n      t.UInt16   :user_id\n      t.Date     :date\n    end\n  end\n\n  def down\n    ClickHouse.connection.drop_table('visits')\n  end\nend\n```\n\n## Using with ActiveRecord\n\nif you use `ActiveRecord`, you can use the ORM query builder by using fake models\n(empty tables must be present in the SQL database `create_table :visits`)\n\n```ruby\nclass ClickHouseRecord \u003c ActiveRecord::Base\n  self.abstract_class = true\n\n  class \u003c\u003c self\n    def agent\n      ClickHouse.connection\n    end\n\n    def insert(*argv, \u0026block)\n      agent.insert(table_name, *argv, \u0026block)\n    end\n\n    def select_one\n      agent.select_one(current_scope.to_sql)\n    end\n\n    def select_value\n      agent.select_value(current_scope.to_sql)\n    end\n\n    def select_all\n      agent.select_all(current_scope.to_sql)\n    end\n\n    def explain\n      agent.explain(current_scope.to_sql)\n    end\n  end\nend\n````\n\n````ruby\n# FAKE MODEL FOR ClickHouse\nclass Visit \u003c ClickHouseRecord\n  scope :with_os, -\u003e { where.not(os_family_id: nil) }\nend\n\nVisit.with_os.select('COUNT(*) as counter').group(:ipv4).select_all\n#=\u003e [{ 'ipv4' =\u003e 1455869, 'counter' =\u003e 104 }]\n\nVisit.with_os.select('COUNT(*)').select_value\n#=\u003e 20_345_678\n\nVisit.where(user_id: 1).select_one\n#=\u003e { 'ipv4' =\u003e 1455869, 'user_id' =\u003e 1 }\n````\n\n## Using with RSpec\n\nYou can clear the data table before each test with RSpec\n\n```ruby\nRSpec.configure do |config|\n  config.before(:each, truncate_click_house: true) do\n    ClickHouse.connection.truncate_tables\n  end\nend\n```\n\n```ruby\nRSpec.describe Api::MetricsCountroller, truncate_click_house: true do\n  it { }\n  it { }\nend\n```\n\n## Development\n\n```bash\nmake dockerize\nrspec\nrubocop\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshlima%2Fclick_house","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshlima%2Fclick_house","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshlima%2Fclick_house/lists"}