{"id":18853725,"url":"https://github.com/bytehouse-cloud/driver-go","last_synced_at":"2025-04-14T10:24:00.146Z","repository":{"id":45118476,"uuid":"395309759","full_name":"bytehouse-cloud/driver-go","owner":"bytehouse-cloud","description":"High Performance Go Driver for Bytehouse","archived":false,"fork":false,"pushed_at":"2024-03-11T08:08:39.000Z","size":973,"stargazers_count":14,"open_issues_count":0,"forks_count":11,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T23:41:46.794Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bytehouse-cloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-12T12:22:52.000Z","updated_at":"2025-03-27T07:38:26.000Z","dependencies_parsed_at":"2024-03-01T09:28:18.589Z","dependency_job_id":"54e91d80-3f4a-416c-baec-0718a230d1bf","html_url":"https://github.com/bytehouse-cloud/driver-go","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytehouse-cloud%2Fdriver-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytehouse-cloud%2Fdriver-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytehouse-cloud%2Fdriver-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytehouse-cloud%2Fdriver-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bytehouse-cloud","download_url":"https://codeload.github.com/bytehouse-cloud/driver-go/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248860701,"owners_count":21173485,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T03:45:24.909Z","updated_at":"2025-04-14T10:24:00.119Z","avatar_url":"https://github.com/bytehouse-cloud.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ByteHouse Driver Go: High Performance Golang Driver and SDK for connecting to ByteHouse\n\n## Usage Guide\n\n### Connect to ByteHouse\n\n#### 1. Connect with API Key\n\n*For ByteHouse Global/China version, users can create and download credentials from [console](https://console.bytehouse.cloud/account/details)\n\n*For ByteHouse Volcano Cloud Version, users need to create and download credentials from Volcano Cloud's [Account Details page](https://console.volcengine.com/auth/login?redirectURI=%2Fbytehouse%2Fregion%3Abytehouse%2Bcn-beijing%2Faccount%2Fdetails) \n\nCreate the API Key and save it in the local environment. \n\n\u003cimg width=\"830\" alt=\"Screenshot 2023-08-31 at 6 01 19 PM\" src=\"https://github.com/rexlionz/driver-go/assets/87936033/d23777f2-1aa9-4e37-a1bd-5580ed80c7d3\"\u003e\n\n### Test Connectivity\n\n```go\ndb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\n//If user wishes to specify the database in url\ndb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\u0026database={DATABASE}\")\n\nif err != nil {\n    fmt.Printf(\"error = %v\", err)\n    return\n}\ndefer db.Close()\n```\n\nReplace the Host:Port and API key placeholders in the dsn below as specified in the [ByteHouse doc](https://docs.bytehouse.cloud/en/bytehouse/docs/supported-regions-and-providers).\n\n### DDL\n\nAll DDL queries should be done with db.ExecContext\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"log\"\n\n\t_ \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n\tdsn := fmt.Sprintf(\"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\t//If user wishes to specify the database in url \n\tdsn := fmt.Sprintf(\"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\u0026database={DATABASE}\")\n\n\tdb, err := sql.Open(\"bytehouse\", dsn)\n\tif err != nil {\n\t\tfmt.Println(\"failed to open db\", err)\n\t\treturn\n\t}\n\n\tdefer db.Close()\n\n\tctx := context.Background()\n\t// create database query\n\tcreateDBQuery := `\n       CREATE database if not exists my_db`\n\t_, err = db.ExecContext(ctx, createDBQuery)\n\tif err != nil {\n\t\tlog.Fatal(\"failed to execute create database query\\n\", err)\n\t\treturn\n\t}\n\t\n\t// create table query\n\tcreateTableQuery := `\n       CREATE TABLE IF NOT EXISTS my_db.animal\n                  (\n                      dog Int64,\n                      cat Int64\n                  )\n                  ENGINE=CnchMergeTree ORDER BY dog\n    `\n\t_, err = db.ExecContext(ctx, createTableQuery)\n\tif err != nil {\n\t\tlog.Fatal(\"failed to execute create table query\\n\", err)\n\t\treturn\n\t}\n} \n```\n\n### Data Insertion\n\nYou can specify the columns to be inserted, if no column is specified, all columns will be chosen\n\n- with select columns `INSERT INTO sample_table (col1, col2) VALUES`\n- without selected columns `INSERT INTO sample_table VALUES`\n\n#### Insertion\n\n```go\npackage main\n\nimport (\n        \"context\"\n        \"database/sql\"\n        \"fmt\"\n\n        \"github.com/bytehouse-cloud/driver-go\"\n        \"github.com/bytehouse-cloud/driver-go/sdk\"\n        sql2 \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n        db, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n        if err != nil {\n                fmt.Printf(\"error = %v\", err)\n                return\n        }\n        defer db.Close()\n\n        // set the insert block size if needed\n        ctx := bytehouse.NewQueryContext(context.Background())\n        batchSize := 1000\n        if err != ctx.AddClientSetting(bytehouse.InsertBlockSize, batchSize) {\n                panic(err)\n        }\n\n        if err = sql2.RunConn(ctx, db, func(conn sdk.Conn) error {\n                stmt, err := conn.PrepareContext(ctx, \"INSERT INTO sample_table VALUES (?, ?)\")\n                if err != nil {\n                        return err\n                }\n                rows := 1000000\n                for i := 0; i \u003c rows; i++ {\n                        if err := stmt.ExecContext(ctx, 1, 2); err != nil {\n                                return err\n                        }\n                }\n\n                return stmt.Close() // Remember to close the stmt! This step is a must for the query to go through!\n        }); err != nil {\n                fmt.Printf(\"error = %v\", err)\n        }\n\n}\n```\n\n#### Insert from select\n\nYou can insert from SELECT statements. Output from select statement with be inserted into your table\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\n\t_ \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n\tdb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer db.Close()\n\n\tctx := context.Background()\n\t// Note first return value is sql.Result, which can be discarded since it is not implemented in the driver\n\tif _, err := db.ExecContext(ctx, \"INSERT INTO sample_table SELECT * FROM sample_table\"); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t}\n}\n```\n\n#### Insertion from local file\n\n##### CSV\n\nFollowing shows how it can be done with csv file format\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"database/sql\"\n    \"fmt\"\n    \"os\"\n\n    \"github.com/bytehouse-cloud/driver-go/sdk\"\n    _ \"github.com/bytehouse-cloud/driver-go/sql\"\n    driverSql \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n    db, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n    if err != nil {\n        fmt.Printf(\"error = %v\", err)\n        return\n    }\n    defer db.Close()\n\n    ctx := context.Background()\n\n    file, err := os.Open(\"./testdata/insert.csv\") // path to your .csv file\n    if err != nil {\n        fmt.Printf(\"error = %v\", err)\n        return\n    }\n    defer file.Close()\n\n    if err = driverSql.RunConn(ctx, db, func(conn sdk.Conn) error {\n     _,e :=conn.InsertFromReader(ctx, \"INSERT INTO sample_table FORMAT CSV\", file)\n        return e\n     }); err != nil {\n         fmt.Printf(\"error = %v\", err)\n     }\n}\n```\n\nExample CSV Format\n\nFormat should not have headers\n\n```\n1,2\n1,2\n```\n\nUsing custom delimiter for your csv file rather than default `,`\n\nAdd to query setting map your custom delimiter The setting name is `format_csv_delimiter`\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"database/sql\"\n    \"fmt\"\n    \"os\"\n\n    \"github.com/bytehouse-cloud/driver-go/sdk\"\n    _ \"github.com/bytehouse-cloud/driver-go/sql\"\n    driverSql \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n    db, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n    if err != nil {\n        fmt.Printf(\"error = %v\", err)\n        return\n    }\n    defer db.Close()\n\n    ctx := context.Background()\n\n    file, err := os.Open(\"./testdata/insert_with_pipes.csv\")\n    if err != nil {\n        fmt.Printf(\"error = %v\", err)\n        return\n    }\n    defer file.Close()\n\n    if err = driverSql.RunConn(ctx, db, func(conn sdk.Conn) error {\n _,e :=conn.InsertFromReader(ctx, \"INSERT INTO sample_table FORMAT CSV\", file)\n return e\n        }); err != nil {\n fmt.Printf(\"error = %v\", err)\n        }\n}\n```\n\n##### CSVWithNames\n\nUse format if your csv file has column headers. Note that this options simply skip the first line of your CSV We do not\nread your CSV column headers and match them to the corresponding row You have to make sure that your CSV column ordering\nis the same as that defined in your table\n\n```go\npackage main\n\nimport (\n        \"context\"\n        \"database/sql\"\n        \"fmt\"\n        \"os\"\n\n        \"github.com/bytehouse-cloud/driver-go/sdk\"\n        _ \"github.com/bytehouse-cloud/driver-go/sql\"\n        driverSql \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n        db, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n        if err != nil {\n                fmt.Printf(\"error = %v\", err)\n                return\n        }\n        defer db.Close()\n\n        ctx := context.Background()\n\n        file, err := os.Open(\"./testdata/insert_with_names.csv\")\n        if err != nil {\n                fmt.Printf(\"error = %v\", err)\n                return\n        }\n        defer file.Close()\n\n        if err = driverSql.RunConn(ctx, db, func(conn sdk.Conn) error {\n     _,e :=conn.InsertFromReader(ctx, \"INSERT INTO sample_table FORMAT CSV\", file)\n     return e\n            }); err != nil {\n     fmt.Printf(\"error = %v\", err)\n            }\n}\n```\n\nExample CSVWithNames Format\n\n- Note: contents of the first line doesn't matter as it will be skipped\n\n```\na, b \n1, 2\n1, 2\n```\n\n##### JSON\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"database/sql\"\n    \"fmt\"\n    \"os\"\n\n    \"github.com/bytehouse-cloud/driver-go/sdk\"\n    _ \"github.com/bytehouse-cloud/driver-go/sql\"\n    driverSql \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n    db, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n    if err != nil {\n        fmt.Printf(\"error = %v\", err)\n        return\n    }\n    defer db.Close()\n\n    ctx := context.Background()\n\n    file, err := os.Open(\"insert.json\")\n    if err != nil {\n        fmt.Printf(\"error = %v\", err)\n        return\n    }\n    defer file.Close()\n\n    if err = driverSql.RunConn(ctx, db, func(conn sdk.Conn) error {\n        _,e :=conn.InsertFromReader(ctx, \"INSERT INTO sample_table FORMAT JSON\", file)\n        return e\n        }); err != nil {\n        fmt.Printf(\"error = %v\", err)\n    }\n}\n```\n\nExample JSON Format\n\n- JSON field name must match with your clickhouse table field name\n- Example: for data below your table should be of this structure `a Int, b Int`\n\n```json\n{\n  \"data\": [\n    {\n      \"a\": 1,\n      \"b\": 2\n    },\n    {\n      \"a\": 1,\n      \"b\": 2\n    }\n  ]\n}\n```\n\n### Select\n\n#### To Golang struct\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\n\t_ \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n\tdb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer db.Close()\n\n\t// Use your own types here depending on your table\n\ttype sample struct {\n\t\tcat int\n\t\tdog int\n\t}\n\n\tctx := context.Background()\n\trows, err := db.QueryContext(ctx, \"SELECT * FROM sample_table LIMIT 5\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\n\tout := make([]sample, 5)\n\ti := 0\n\tfor rows.Next() {\n\t\tif err := rows.Scan(\u0026out[i].dog, \u0026out[i].cat); err != nil {\n\t\t\tfmt.Printf(\"error = %v\", err)\n\t\t}\n\t\ti++\n\t}\n\n\tfmt.Println(out)\n\n\t// Remember to close your rows when you are done! This is a must!\n\tif err := rows.Close(); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t}\n}\n```\n\n##### Single Row\n\nIf you are selecting just a single row, you can use db.QueryRowContext which is much more convenient!\nMake sure that your query only returns one row!\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\n\t_ \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n\tdb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer db.Close()\n\n\t// Use your own types here depending on your table value \n\ttype sample struct {\n\t\tdog int\n\t\tcat int\n\t}\n\n\tsampleData := \u0026sample{}\n\tctx := context.Background()\n\trow := db.QueryRowContext(ctx, \"SELECT * FROM sample_table LIMIT 1\")\n\n\tif err := row.Scan(\u0026sampleData.dog, \u0026sampleData.dog); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\n\tfmt.Printf(\"%+v\\n\", sampleData)\n}\n```\n\n#### To local file\n\n- Use this when you want to export your query results into a file\n\n##### CSV\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/bytehouse-cloud/driver-go/sdk\"\n\t_ \"github.com/bytehouse-cloud/driver-go/sql\"\n\tdriverSql \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n\tdb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer db.Close()\n\n\tctx := context.Background()\n\n\tfile, err := os.OpenFile(\"./testdata/select_temp.csv\", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer file.Close()\n\n\tvar reader io.Reader\n\tif err = driverSql.RunConn(ctx, db, func(conn sdk.Conn) error {\n\t\tqr, err := conn.QueryContext(ctx, `\n\t\tSELECT * FROM\n\t\tsample_table\n\t\t`)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer qr.Close()\n\n\t\treader = qr.ExportToReader(\"CSV\")\n\t\treturn nil\n\n\t}); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\n\tif _, err = io.Copy(file, reader); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t}\n}\n```\n\nOutput: `1,2 3,4 5,6`\n\n###### JSON\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/bytehouse-cloud/driver-go/sdk\"\n\t_ \"github.com/bytehouse-cloud/driver-go/sql\"\n\tdriverSql \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n\tdb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer db.Close()\n\n\tctx := context.Background()\n\n\tfile, err := os.OpenFile(\"./testdata/select_temp.json\", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer file.Close()\n\n\tvar reader io.Reader\n\tif err = driverSql.RunConn(ctx, db, func(conn sdk.Conn) error {\n\t\tqr, err := conn.QueryContext(ctx, \"SELECT * FROM sample_table\")\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer qr.Close()\n\n\t\treader = qr.ExportToReader(\"JSON\")\n\t\treturn nil\n\n\t}); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\n\tif _, err = io.Copy(file, reader); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t}\n}\n```\n\nOutput\n\n```json\n{\n  \"meta\": [\n    {\n      \"name\": \"dog\",\n      \"type\": \"UInt32\"\n    },\n    {\n      \"name\": \"cat\",\n      \"type\": \"UInt32\"\n    }\n  ],\n  \"data\": [\n    {\n      \"dog\": 1,\n      \"cat\": 2\n    },\n    {\n      \"dog\": 3,\n      \"cat\": 4\n    },\n    {\n      \"dog\": 5,\n      \"cat\": 6\n    }\n  ],\n  \"rows\": 3\n}\n```\n\n##### VALUES\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/bytehouse-cloud/driver-go/sdk\"\n\t_ \"github.com/bytehouse-cloud/driver-go/sql\"\n\tdriverSql \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n\tdb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer db.Close()\n\n\tctx := context.Background()\n\n\tfile, err := os.OpenFile(\"./testdata/select_temp.txt\", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer file.Close()\n\n\tvar reader io.Reader\n\tif err = driverSql.RunConn(ctx, db, func(conn sdk.Conn) error {\n\t\tqr, err := conn.QueryContext(ctx, \"SELECT * FROM sample_table\")\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer qr.Close()\n\n\t\treader = qr.ExportToReader(\"VALUES\")\n\t\treturn nil\n\n\t}); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\n\tif _, err = io.Copy(file, reader); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t}\n}\n```\n\nOutput\n\n```\n(1, 2),\n(3, 4),\n(5, 6)\n```\n\n### Query with external tables (local file system)\n\n- For more info on external tables: https://clickhouse.tech/docs/en/engines/table-engines/special/external-data/\n\nExternal tables refer to data you want to reference in your query that is not in your database\n\nExternal table in file CSV ./test_data/external_table.csv\n\n```\n1,4 2,5 3,6\n```\n\nCode Example\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/bytehouse-cloud/driver-go/driver/lib/data/column\"\n\t\"github.com/bytehouse-cloud/driver-go/sdk\"\n\tsqlDriver \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n\tdb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer db.Close()\n\n\tctx := context.Background()\n\n\t// Open file \n\tfile, err := os.Open(\"./testdata/external_table.csv\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer file.Close()\n\n\t// Run insert query \n\tvar qr *sdk.QueryResult\n\tif err = sqlDriver.RunConn(ctx, db, func(conn sdk.Conn) error {\n\t\tqr, err =\n\t\t\tconn.QueryContextWithExternalTableReader(\n\t\t\t\tctx, // External table name used \"fish\" must match that in the ExternalTableReader\n\t\t\t\t\"SELECT a, b FROM fish\", sdk.NewExternalTableReader(\n\t\t\t\t\t// Table name\n\t\t\t\t\t\"fish\",\n\t\t\t\t\t// File path\n\t\t\t\t\tfile,\n\t\t\t\t\t// Column names\n\t\t\t\t\t[]string{\"a\", \"b\"},\n\t\t\t\t\t// Column types\n\t\t\t\t\t[]column.CHColumnType{column.UINT32, column.UINT32},\n\t\t\t\t\t// File format\n\t\t\t\t\t\"CSV\",\n\t\t\t\t),\n\t\t\t)\n\n\t\treturn err\n\n\t}); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\n\tdefer qr.Close()\n\tout := make([][]interface{}, 0, 5)\n\tfor {\n\t\trowValues, ok := qr.NextRow()\n\t\tif !ok {\n\t\t\tbreak\n\t\t}\n\n\t\tout = append(out, rowValues)\n\n\t}\n\n\tfmt.Println(out) // [[1 4] [2 5] [3 6]]\n}\n```\n\n### Query settings\n\nUsage Example\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\n\t\"github.com/bytehouse-cloud/driver-go\"\n\t_ \"github.com/bytehouse-cloud/driver-go/sql\"\n)\n\nfunc main() {\n\tdb, err := sql.Open(\"bytehouse\", \"tcp://{HOST}:{PORT}?secure=true\u0026user=bytehouse\u0026password={API_KEY}\")\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\tdefer db.Close()\n\n\tctx := context.Background()\n\tqueryCtx := bytehouse.NewQueryContext(ctx)\n\tif err := queryCtx.AddQuerySetting(\"Query Setting Name\", \"Query Setting Value\"); err != nil {\n\t\tfmt.Printf(\"error = %v\",\n\t\t\terr)\n\t\treturn\n\t}\n\n\tif _, err := db.ExecContext(queryCtx, \"INSERT INTO sample_table VALUES (?, ?)\", 1, 2); err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t}\n}\n```\n\n### Multi threading and Connection Pooling\nThe SQL interface that Go provides uses a connection pool by default. Connection pool configuration can be customized during runtime.\n\nThe returned pool is thread safe so it can be used by multiple goroutines at the same time. We only call the `sql.Open` once.\nYou must call the `Ping()` method to test for connectivity. `sql.Open` does not establish connections until a request is sent.\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\tbytehouse \"github.com/bytehouse-cloud/driver-go\"\n\t_ \"github.com/bytehouse-cloud/driver-go/sql\" //this is required, otherwise \"bytehouse\" driver will not be registered\n\t\"golang.org/x/sync/errgroup\"\n\t\"log\"\n\t\"time\"\n)\n\nfunc main() {\n\thost := \"\u003c\u003chostname\u003e\u003e\"\n\tport := \"\u003c\u003cport\u003e\u003e\"\n\tapiToken := \"\u003c\u003cYOUR_API_TOKEN\u003e\u003e\"\n\t\n\tdsn := fmt.Sprintf(\"tcp://%s:%s?secure=true\u0026user=bytehouse\u0026password=%s\", host, port, apiToken)\n\t\n\tqueryString := \"select * from numbers(100)\"\n\t\n\tpool, err := sql.Open(\"bytehouse\", dsn)\n\tif err != nil {\n\t\tfmt.Printf(\"error = %v\", err)\n\t\treturn\n\t}\n\t\n\tdefer pool.Close()\n\n\t//setup the database configuration here\n\tpool.SetMaxOpenConns(10)\n\tpool.SetConnMaxIdleTime(time.Minute)\n\tpool.SetConnMaxLifetime(time.Minute * 5)\n\tpool.SetMaxIdleConns(10)\n\n\tthreadCount := 10\n\teg := errgroup.Group{}\n\n\tfor i := 0; i \u003c threadCount; i++ {\n\t\ti := i\n\t\teg.Go(func() error {\n\t\t\tthreadID := fmt.Sprintf(\"thread-%v\", i)\n\t\t\t//we need to create bytehouse.NewQueryContext() to assign a query ID and add query settings\n\t\t\tqueryCtx := bytehouse.NewQueryContext(context.Background())\n\t\t\t\n\t\t\t//set the query ID here, duplicate query IDs will be rejected\n\t\t\tqueryCtx.SetQueryID(fmt.Sprintf(\"my_query_id%v\", time.Now().String()))\n\t\t\t\n\t\t\t//add optional query settings\n\t\t\tif err := queryCtx.AddQuerySetting(\"max_block_size\", \"2000\"); err != nil {\n\t\t\t\tlog.Fatalf(\"thread %v failed to add query setting err = %v\", threadID, err)\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\t//test if the connection is valid\n\t\t\tif err := pool.Ping(); err != nil {\n\t\t\t\tlog.Fatalf(\"thread %v failed to ping err = %v\", threadID, err)\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t\n\t\t\t//send the query\n\t\t\trows, err := pool.QueryContext(queryCtx, queryString)\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatalf(\"thread %v failed to query err = %v\", threadID, err)\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\t//we must close the row\n\t\t\tdefer rows.Close()\n\t\t\tvar num int64\n\n\t\t\t//iterate and retrieve the query result\n\t\t\tfor rows.Next() {\n\t\t\t\tif err := rows.Scan(\u0026num); err != nil {\n\t\t\t\t\tlog.Fatalf(\"thread %v failed to scan row err = %v\", threadID, err)\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tfmt.Printf(\"\\nID:%v Value: %v\", threadID, num)\n\t\t\t}\n\t\t\tfmt.Printf(\"\\n%v Done!\", threadID)\n\t\t\treturn nil\n\t\t})\n\t}\n\n\t//wait for all threads to finish\n\tif err := eg.Wait(); err != nil {\n\t\tlog.Fatalf(\"one of the threads have an error:= %v\", err)\n\t}\n\n\tfmt.Println(\"\\nAll threads done!\")\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbytehouse-cloud%2Fdriver-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbytehouse-cloud%2Fdriver-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbytehouse-cloud%2Fdriver-go/lists"}