{"id":15688046,"url":"https://github.com/mrcrypster/clickhousy","last_synced_at":"2025-05-07T20:22:02.465Z","repository":{"id":46604148,"uuid":"515149086","full_name":"mrcrypster/clickhousy","owner":"mrcrypster","description":"High performance Clickhouse PHP lib with progress tracking, parametric queries and compression support","archived":false,"fork":false,"pushed_at":"2023-10-19T12:32:19.000Z","size":45,"stargazers_count":10,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-26T04:22:28.957Z","etag":null,"topics":["clickhouse","clickhouse-client","php","php-clickhouse"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mrcrypster.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-18T11:07:34.000Z","updated_at":"2024-03-28T09:07:26.000Z","dependencies_parsed_at":"2024-10-23T20:44:05.821Z","dependency_job_id":"f845c795-03eb-4440-ab45-91b237e23dc8","html_url":"https://github.com/mrcrypster/clickhousy","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrcrypster%2Fclickhousy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrcrypster%2Fclickhousy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrcrypster%2Fclickhousy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrcrypster%2Fclickhousy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mrcrypster","download_url":"https://codeload.github.com/mrcrypster/clickhousy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243025440,"owners_count":20223801,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clickhouse","clickhouse-client","php","php-clickhouse"],"created_at":"2024-10-03T17:53:59.285Z","updated_at":"2025-03-11T11:32:11.536Z","avatar_url":"https://github.com/mrcrypster.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Clickhousy\nHigh performance Clickhouse PHP library featuring:\n- Tiny memory footprint based on static class (hundreds of times less consumption than [smi2 client](#memory-usage-and-performance)).\n- Curl based.\n- High level methods to fetch rows, single row, array of scalar values or single value.\n- Parametric queries (native Clickhouse SQL injection protection).\n- Long queries progress update through callback function.\n- Large result sets processing without memory overflow.\n- HTTP native compression.\n- Error handling.\n- Batch data writing.\n\n\n## Quick start\nClone or download library:\n\n```bash\ngit clone https://github.com/mrcrypster/clickhousy.git\n```\n\nImport lib and use it:\n\n```php\nrequire 'clickhousy/clickhousy.php';\n$data = clickhousy::rows('SELECT * FROM table LIMIT 5');\n```\n\n\n## Connection \u0026 database\n`Clickhousy` works over [Clickhouse HTTP protocol](https://clickhouse.com/docs/en/interfaces/http/), so you just need to set connection url (by default it's `localhost:8123`):\n```php\nclickhousy::set_url('http://host:port/'); # no auth\nclickhousy::set_url('http://user:password@host:port/'); # auth\n```\n\nAnd then select database (`default` by default):\n```php\nclickhousy::set_db('my_db');\n```\n\n\n## Data fetch\nUse predefined methods to quickly fetch needed data:\n```php\nclickhousy::rows('SELECT * FROM table');         # -\u003e returns array or associative arrays\nclickhousy::row ('SELECT * FROM table LIMIT 1'); # -\u003e returns single row associative array\nclickhousy::cols('SELECT id FROM table');        # -\u003e returns array of scalar values\nclickhousy::col ('SELECT count(*) FROM table');  # -\u003e returns single scalar value\n```\n\n\n## Reading large datasets\nIf your query returns many rows, use reading callback in order not to run out of memory:\n```php\nclickhousy::query('SELECT * FROM large_table', [], null, null, function($packet) {\n  // $packet will contain small portion of returning data\n  foreach ( $packet as $row ) {\n    // do something with $row (array of each result row values)\n    print_r($row) # [0 =\u003e 'col1 value', 1 =\u003e 'col2 value', ...]\n  }\n});\n```\n\n\n## Safe query execution\nIn order to fight SQL injections, you can use [parametric queries](https://clickhouse.com/docs/en/interfaces/http/#cli-queries-with-parameters):\n\n```php\n$count = clickhousy::col(\n  'SELECT count(*) FROM table WHERE age \u003e {age:UInt32} AND category = {cat:String}',\n  ['age' =\u003e 30, 'cat' =\u003e 'Engineering']\n);\n```\n\n\n## Writing data\nIt's not a good idea to insert large amounts of data into Clickhouse using single row inserts.\nConsider using other technologies, like Kafka to ingest data into Clockhouse efficiently.\nAvoid using client-library inserts in production environments.\n\nWith Clickhousy you can insert either named rows:\n```php\nclickhousy::insert('my_table', [\n    ['id' =\u003e 1, 'date' =\u003e date('Y-m-d')],\n    ['id' =\u003e 2, 'date' =\u003e date('Y-m-d')]\n]);\n```\n\nOr unnamed rows (in this case, make sure that values order is the same as table columns order):\n```php\nclickhousy::insert('my_table', [\n    [1, date('Y-m-d')],\n    [2, date('Y-m-d')]\n]);\n```\n\n### Inserting large datasets\nIf you still need (which you should avoid please) to insert large amounts of data using Clickhousy,\ndo it in batches of at least couple of thousands rows (but be sure to monitor RAM usage by PHP):\n```php\n$f = fopen('large.csv', 'r');\n$batch = [];\nwhile ( $row = fgetcsv($f) ) {\n    $batch[] = $row;\n    if ( count($batch) \u003e= 5000 ) {\n        clickhousy::insert('my_table', $batch);\n        $batch = [];\n    }\n}\n\nif ( $batch ) {\n    clickhousy::insert('my_table', $batch);\n}\n```\n\n\n## Custom queries\nGeneric `query` method is available for any read or write query:\n```php\nclickhousy::query('INSERT INTO table(id) VALUES(1)');\nclickhousy::query('TRUNCATE TABLE table');\nclickhousy::query('SELECT NOW()');\n```\n\nFor `SELECT` queries it will return resultset together with system information:\n```php\n$result_set = clickhousy::query('SELECT * FROM numbers(5)');\nprint_r($result_set);\n```\n\nWill output:\n```\nArray\n(\n    [meta] =\u003e Array\n        (\n            [0] =\u003e Array\n                (\n                    [name] =\u003e number\n                    [type] =\u003e UInt64\n                )\n\n        )\n\n    [data] =\u003e Array\n        (\n            [0] =\u003e Array\n                (\n                    [number] =\u003e 0\n                )\n            ...\n        )\n\n    [rows] =\u003e 2\n    [rows_before_limit_at_least] =\u003e 2\n    [statistics] =\u003e Array\n        (\n            [elapsed] =\u003e 0.000237397\n            [rows_read] =\u003e 2\n            [bytes_read] =\u003e 16\n        )\n\n)\n```\n\nMethod also supports parametric queries:\n```php\n$res = clickhousy::query('SELECT * FROM table WHERE age \u003e {age:Int32}', ['age' =\u003e $_GET['age']]);\n```\n\n\n## Inserting data from files\nYou can insert data from files using `$post_buffer` argument pointing to a file:\n```php\n$res = clickhousy::query('INSERT INTO table', [], '/path/to/tsv.gz');\n```\n\nFile should be of gzipped TSV format.\n\n\n## Long query progress tracking\n`$progress_callback` allows specifying callback function which will be called when query execution progress updates:\n```php\nclickhousy::query('SELECT uniq(id) FROM huge_table', [], null, function($progress) {\n  print_r($progress);\n});\n```\nIf the query is long enough, progress function will be called multiple times:\n```\nArray\n(\n    [read_rows] =\u003e 4716360              # -\u003e currently read rows \n    [read_bytes] =\u003e 37730880\n    [written_rows] =\u003e 0\n    [written_bytes] =\u003e 0\n    [total_rows_to_read] =\u003e 50000000    # -\u003e total rows to read\n)\n...\nArray\n(\n    [read_rows] =\u003e 47687640\n    [read_bytes] =\u003e 381501120\n    [written_rows] =\u003e 0\n    [written_bytes] =\u003e 0\n    [total_rows_to_read] =\u003e 50000000\n)\n```\n\nIt's easy to calculate and print query execution progress:\n```php\nclickhousy::query('SELECT count(*) FROM large_table WHERE heavy_condition', [], null, function($progress) {\n  echo round(100 * $progress['read_rows']/$progress['total_rows_to_read']) . '%' . \"\\n\";\n});\n```\nWhich will result in:\n```\n7%\n17%\n28%\n39%\n50%\n61%\n72%\n83%\n94%\n```\n\n\n## Query execution summary\nLatest query summary is available through:\n```php\nclickhousy::rows('SELECT * FROM numbers(100000)');\nprint_r(clickhousy::last_summary());\n```\nWhich is an associative array of multiple metrics:\n```\nArray\n(\n    [read_rows] =\u003e 65505\n    [read_bytes] =\u003e 524040\n    [written_rows] =\u003e 0\n    [written_bytes] =\u003e 0\n    [total_rows_to_read] =\u003e 100000\n)\n```\n\n\n## Errors handling and debug\nBy default, `clickhousy` will return response with `error` key on error, but will not throw any exceptions:\n```php\n$response = clickhousy::query('SELECT bad_query');  # sample query with error\nprint_r($response);\n```\nWhich contains original Clickhouse error message:\n```\nArray\n(\n    [error] =\u003e Code: 47. DB::Exception: Missing columns: 'bad_query' while processing query: 'SELECT bad_query', required columns: 'bad_query'. (UNKNOWN_IDENTIFIER) (version 22.6.1.1696 (official build))\n\n)\n```\n\nIf you want exceptions functionality, you can extend `clickhousy` with your own class and override `error()` method:\n```php\nclass my_clickhousy_exception extends Exception {};\nclass my_clickhousy extends clickhousy {\n  public static function error($msg, $request_info) {\n    throw new my_clickhousy_exception($msg);\n  }\n}\n```\n\nThen use your own class to get exceptions working:\n```php\nmy_clickhousy::query('SELECT bad_query');   # bad query example\n# PHP Fatal error:  Uncaught my_clickhousy_exception: Code: 47. DB::Exception: Missing columns: 'bad_query' ...\n```\n\n\n### Debugging response\nIf you need to access raw response from Clickhouse, you can find it here: \n```php\nclickhouse::last_response();  # raw text response from Clickhouse after latest query\n```\n\nCurl (which is used for HTTP communication) response info can be accessed via:\n```php\nclickhouse::last_info();  # response from curl_getinfo() after latest query\n```\n\n\n### Logging\nThere is also `log()` method which can be overrided to allow custom logging:\n```php\nclass my_clickhousy extends clickhousy {\n  public static function log($raw_response, $curl_info) {\n    error_log('Received response from Clickhouse: ' . $raw_response);\n  }\n}\n```\n\n\n## Memory usage and performance\nBased on [performance testing](tests/smi2-test.php), `Clickhousy` lib is times faster\nand hundreds of times less memory consuming than [smi2 lib](https://github.com/smi2/phpClickHouse):\n\n```\nSmi2: \nmem:                565       \ntime:               10.3      \n\nClickhousy: \nmem:                0.8       688.8x  better\ntime:               2.3       4.4x  better\n``` \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrcrypster%2Fclickhousy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmrcrypster%2Fclickhousy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrcrypster%2Fclickhousy/lists"}