{"id":19032152,"url":"https://github.com/plausible/ch","last_synced_at":"2025-04-15T14:45:50.944Z","repository":{"id":103947053,"uuid":"570480442","full_name":"plausible/ch","owner":"plausible","description":"HTTP ClickHouse driver for Elixir","archived":false,"fork":false,"pushed_at":"2024-05-03T20:44:22.000Z","size":395,"stargazers_count":33,"open_issues_count":15,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-05-03T21:37:04.298Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/plausible.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-25T09:44:27.000Z","updated_at":"2024-05-30T04:12:50.902Z","dependencies_parsed_at":"2023-11-11T07:27:05.610Z","dependency_job_id":"9dcc1a2d-9ddd-4ef1-8bf7-57c3aa0cf639","html_url":"https://github.com/plausible/ch","commit_stats":null,"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plausible%2Fch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plausible%2Fch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plausible%2Fch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plausible%2Fch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/plausible","download_url":"https://codeload.github.com/plausible/ch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249093468,"owners_count":21211706,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T21:27:05.107Z","updated_at":"2025-04-15T14:45:50.937Z","avatar_url":"https://github.com/plausible.png","language":"Elixir","funding_links":[],"categories":["Language bindings"],"sub_categories":["Elixir"],"readme":"# Ch\n\n[![Documentation badge](https://img.shields.io/badge/Documentation-ff69b4)](https://hexdocs.pm/ch)\n[![Hex.pm badge](https://img.shields.io/badge/Package%20on%20hex.pm-informational)](https://hex.pm/packages/ch)\n\nMinimal HTTP [ClickHouse](https://clickhouse.com) client for Elixir.\n\nUsed in [Ecto ClickHouse adapter.](https://github.com/plausible/ecto_ch)\n\n### Key features\n\n- RowBinary\n- Native query parameters\n- Per query settings\n- Minimal API\n\nYour ideas are welcome [here.](https://github.com/plausible/ch/issues/82)\n\n## Installation\n\n```elixir\ndefp deps do\n  [\n    {:ch, \"~\u003e 0.3.0\"}\n  ]\nend\n```\n\n## Usage\n\n#### Start [DBConnection](https://github.com/elixir-ecto/db_connection) pool\n\n```elixir\ndefaults = [\n  scheme: \"http\",\n  hostname: \"localhost\",\n  port: 8123,\n  database: \"default\",\n  settings: [],\n  pool_size: 1,\n  timeout: :timer.seconds(15)\n]\n\n# note that starting in ClickHouse 25.1.3.23 `default` user doesn't have\n# network access by default in the official Docker images\n# see https://github.com/ClickHouse/ClickHouse/pull/75259\n{:ok, pid} = Ch.start_link(defaults)\n```\n\n#### Select rows\n\n```elixir\n{:ok, pid} = Ch.start_link()\n\n{:ok, %Ch.Result{rows: [[0], [1], [2]]}} =\n  Ch.query(pid, \"SELECT * FROM system.numbers LIMIT 3\")\n\n{:ok, %Ch.Result{rows: [[0], [1], [2]]}} =\n  Ch.query(pid, \"SELECT * FROM system.numbers LIMIT {$0:UInt8}\", [3])\n\n{:ok, %Ch.Result{rows: [[0], [1], [2]]}} =\n  Ch.query(pid, \"SELECT * FROM system.numbers LIMIT {limit:UInt8}\", %{\"limit\" =\u003e 3})\n```\n\nNote on datetime encoding in query parameters:\n\n- `%NaiveDateTime{}` is encoded as text to make it assume the column's or ClickHouse server's timezone\n- `%DateTime{}` is encoded as unix timestamp and is treated as UTC timestamp by ClickHouse\n\n#### Insert rows\n\n```elixir\n{:ok, pid} = Ch.start_link()\n\nCh.query!(pid, \"CREATE TABLE IF NOT EXISTS ch_demo(id UInt64) ENGINE Null\")\n\n%Ch.Result{num_rows: 2} =\n  Ch.query!(pid, \"INSERT INTO ch_demo(id) VALUES (0), (1)\")\n\n%Ch.Result{num_rows: 2} =\n  Ch.query!(pid, \"INSERT INTO ch_demo(id) VALUES ({$0:UInt8}), ({$1:UInt32})\", [0, 1])\n\n%Ch.Result{num_rows: 2} =\n  Ch.query!(pid, \"INSERT INTO ch_demo(id) VALUES ({a:UInt16}), ({b:UInt64})\", %{\"a\" =\u003e 0, \"b\" =\u003e 1})\n\n%Ch.Result{num_rows: 2} =\n  Ch.query!(pid, \"INSERT INTO ch_demo(id) SELECT number FROM system.numbers LIMIT {limit:UInt8}\", %{\"limit\" =\u003e 2})\n```\n\n#### Insert rows as [RowBinary](https://clickhouse.com/docs/en/interfaces/formats/RowBinary) (efficient)\n\n```elixir\n{:ok, pid} = Ch.start_link()\n\nCh.query!(pid, \"CREATE TABLE IF NOT EXISTS ch_demo(id UInt64) ENGINE Null\")\n\ntypes = [\"UInt64\"]\n# or\ntypes = [Ch.Types.u64()]\n# or\ntypes = [:u64]\n\n%Ch.Result{num_rows: 2} =\n  Ch.query!(pid, \"INSERT INTO ch_demo(id) FORMAT RowBinary\", [[0], [1]], types: types)\n```\n\nNote that RowBinary format encoding requires `:types` option to be provided.\n\nSimilarly, you can use [RowBinaryWithNamesAndTypes](https://clickhouse.com/docs/en/interfaces/formats/RowBinaryWithNamesAndTypes) which would additionally do something like a type check.\n\n```elixir\nsql = \"INSERT INTO ch_demo FORMAT RowBinaryWithNamesAndTypes\"\nopts = [names: [\"id\"], types: [\"UInt64\"]]\nrows = [[0], [1]]\n\n%Ch.Result{num_rows: 2} = Ch.query!(pid, sql, rows, opts)\n```\n\n#### Insert rows in custom [format](https://clickhouse.com/docs/en/interfaces/formats)\n\n```elixir\n{:ok, pid} = Ch.start_link()\n\nCh.query!(pid, \"CREATE TABLE IF NOT EXISTS ch_demo(id UInt64) ENGINE Null\")\n\ncsv = [0, 1] |\u003e Enum.map(\u0026to_string/1) |\u003e Enum.intersperse(?\\n)\n\n%Ch.Result{num_rows: 2} =\n  Ch.query!(pid, \"INSERT INTO ch_demo(id) FORMAT CSV\", csv, encode: false)\n```\n\n#### Insert rows as chunked RowBinary stream\n\n```elixir\n{:ok, pid} = Ch.start_link()\n\nCh.query!(pid, \"CREATE TABLE IF NOT EXISTS ch_demo(id UInt64) ENGINE Null\")\n\nstream = Stream.repeatedly(fn -\u003e [:rand.uniform(100)] end)\nchunked = Stream.chunk_every(stream, 100)\nencoded = Stream.map(chunked, fn chunk -\u003e Ch.RowBinary.encode_rows(chunk, _types = [\"UInt64\"]) end)\nten_encoded_chunks = Stream.take(encoded, 10)\n\n%Ch.Result{num_rows: 1000} =\n  Ch.query(pid, \"INSERT INTO ch_demo(id) FORMAT RowBinary\", ten_encoded_chunks, encode: false)\n```\n\nThis query makes a [`transfer-encoding: chunked`](https://en.wikipedia.org/wiki/Chunked_transfer_encoding) HTTP request while unfolding the stream resulting in lower memory usage.\n\n#### Query with custom [settings](https://clickhouse.com/docs/en/operations/settings/settings)\n\n```elixir\n{:ok, pid} = Ch.start_link()\n\nsettings = [async_insert: 1]\n\n%Ch.Result{rows: [[\"async_insert\", \"Bool\", \"0\"]]} =\n  Ch.query!(pid, \"SHOW SETTINGS LIKE 'async_insert'\")\n\n%Ch.Result{rows: [[\"async_insert\", \"Bool\", \"1\"]]} =\n  Ch.query!(pid, \"SHOW SETTINGS LIKE 'async_insert'\", [], settings: settings)\n```\n\n## Caveats\n\n#### NULL in RowBinary\n\nIt's the same as in [ch-go](https://clickhouse.com/docs/en/integrations/go#nullable)\n\n\u003e At insert time, Nil can be passed for both the normal and Nullable version of a column. For the former, the default value for the type will be persisted, e.g., an empty string for string. For the nullable version, a NULL value will be stored in ClickHouse.\n\n```elixir\n{:ok, pid} = Ch.start_link()\n\nCh.query!(pid, \"\"\"\nCREATE TABLE ch_nulls (\n  a UInt8 NULL,\n  b UInt8 DEFAULT 10,\n  c UInt8 NOT NULL\n) ENGINE Memory\n\"\"\")\n\ntypes = [\"Nullable(UInt8)\", \"UInt8\", \"UInt8\"]\ninserted_rows = [[nil, nil, nil]]\nselected_rows = [[nil, 0, 0]]\n\n%Ch.Result{num_rows: 1} =\n  Ch.query!(pid, \"INSERT INTO ch_nulls(a, b, c) FORMAT RowBinary\", inserted_rows, types: types)\n\n%Ch.Result{rows: ^selected_rows} =\n  Ch.query!(pid, \"SELECT * FROM ch_nulls\")\n```\n\nNote that in this example `DEFAULT 10` is ignored and `0` (the default value for `UInt8`) is persisted instead.\n\nHowever, [`input()`](https://clickhouse.com/docs/en/sql-reference/table-functions/input) can be used as a workaround:\n\n```elixir\nsql = \"\"\"\nINSERT INTO ch_nulls\n  SELECT * FROM input('a Nullable(UInt8), b Nullable(UInt8), c UInt8')\n  FORMAT RowBinary\\\n\"\"\"\n\nCh.query!(pid, sql, inserted_rows, types: [\"Nullable(UInt8)\", \"Nullable(UInt8)\", \"UInt8\"])\n\n%Ch.Result{rows: [[0], [10]]} =\n  Ch.query!(pid, \"SELECT b FROM ch_nulls ORDER BY b\")\n```\n\n#### UTF-8 in RowBinary\n\nWhen decoding [`String`](https://clickhouse.com/docs/en/sql-reference/data-types/string) columns non UTF-8 characters are replaced with `�` (U+FFFD). This behaviour is similar to [`toValidUTF8`](https://clickhouse.com/docs/en/sql-reference/functions/string-functions#tovalidutf8) and [JSON format.](https://clickhouse.com/docs/en/interfaces/formats#json)\n\n```elixir\n{:ok, pid} = Ch.start_link()\n\nCh.query!(pid, \"CREATE TABLE ch_utf8(str String) ENGINE Memory\")\n\nbin = \"\\x61\\xF0\\x80\\x80\\x80b\"\nutf8 = \"a�b\"\n\n%Ch.Result{num_rows: 1} =\n  Ch.query!(pid, \"INSERT INTO ch_utf8(str) FORMAT RowBinary\", [[bin]], types: [\"String\"])\n\n%Ch.Result{rows: [[^utf8]]} =\n  Ch.query!(pid, \"SELECT * FROM ch_utf8\")\n\n%Ch.Result{rows: %{\"data\" =\u003e [[^utf8]]}} =\n  pid |\u003e Ch.query!(\"SELECT * FROM ch_utf8 FORMAT JSONCompact\") |\u003e Map.update!(:rows, \u0026Jason.decode!/1)\n```\n\nTo get raw binary from `String` columns use `:binary` type that skips UTF-8 checks.\n\n```elixir\n%Ch.Result{rows: [[^bin]]} =\n  Ch.query!(pid, \"SELECT * FROM ch_utf8\", [], types: [:binary])\n```\n\n#### Timezones in RowBinary\n\nDecoding non-UTC datetimes like `DateTime('Asia/Taipei')` requires a [timezone database.](https://hexdocs.pm/elixir/DateTime.html#module-time-zone-database)\n\n```elixir\nMix.install([:ch, :tz])\n\n:ok = Calendar.put_time_zone_database(Tz.TimeZoneDatabase)\n\n{:ok, pid} = Ch.start_link()\n\n%Ch.Result{rows: [[~N[2023-04-25 17:45:09]]]} =\n  Ch.query!(pid, \"SELECT CAST(now() as DateTime)\")\n\n%Ch.Result{rows: [[~U[2023-04-25 17:45:11Z]]]} =\n  Ch.query!(pid, \"SELECT CAST(now() as DateTime('UTC'))\")\n\n%Ch.Result{rows: [[%DateTime{time_zone: \"Asia/Taipei\"} = taipei]]} =\n  Ch.query!(pid, \"SELECT CAST(now() as DateTime('Asia/Taipei'))\")\n\n\"2023-04-26 01:45:12+08:00 CST Asia/Taipei\" = to_string(taipei)\n```\n\nEncoding non-UTC datetimes works but might be slow due to timezone conversion:\n\n```elixir\nMix.install([:ch, :tz])\n\n:ok = Calendar.put_time_zone_database(Tz.TimeZoneDatabase)\n\n{:ok, pid} = Ch.start_link()\n\nCh.query!(pid, \"CREATE TABLE ch_datetimes(name String, datetime DateTime) ENGINE Memory\")\n\nnaive = NaiveDateTime.utc_now()\nutc = DateTime.utc_now()\ntaipei = DateTime.shift_zone!(utc, \"Asia/Taipei\")\n\nrows = [[\"naive\", naive], [\"utc\", utc], [\"taipei\", taipei]]\n\nCh.query!(pid, \"INSERT INTO ch_datetimes(name, datetime) FORMAT RowBinary\", rows, types: [\"String\", \"DateTime\"])\n\n%Ch.Result{\n  rows: [\n    [\"naive\", ~U[2024-12-21 05:24:40Z]],\n    [\"utc\", ~U[2024-12-21 05:24:40Z]],\n    [\"taipei\", ~U[2024-12-21 05:24:40Z]]\n  ]\n} =\n  Ch.query!(pid, \"SELECT name, CAST(datetime as DateTime('UTC')) FROM ch_datetimes\")\n```\n\n## [Benchmarks](./bench)\n\nSee nightly [CI runs](https://github.com/plausible/ch/actions/workflows/bench.yml) for latest results.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplausible%2Fch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplausible%2Fch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplausible%2Fch/lists"}