{"id":27933437,"url":"https://github.com/timescale/bigquery-go-client","last_synced_at":"2025-05-07T04:58:42.951Z","repository":{"id":225234874,"uuid":"765426795","full_name":"timescale/bigquery-go-client","owner":"timescale","description":"A database/sql driver for BigQuery","archived":false,"fork":false,"pushed_at":"2025-04-23T17:53:23.000Z","size":70,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-05-07T04:58:39.814Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timescale.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-02-29T22:30:49.000Z","updated_at":"2024-04-17T14:14:10.000Z","dependencies_parsed_at":"2024-03-04T18:14:46.218Z","dependency_job_id":"cb68e376-ac31-4f13-9ea1-db4c64f43bee","html_url":"https://github.com/timescale/bigquery-go-client","commit_stats":null,"previous_names":["popsql/bigquery-go-client","timescale/bigquery-go-client"],"tags_count":26,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fbigquery-go-client","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fbigquery-go-client/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fbigquery-go-client/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timescale%2Fbigquery-go-client/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timescale","download_url":"https://codeload.github.com/timescale/bigquery-go-client/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252816948,"owners_count":21808704,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-07T04:58:42.352Z","updated_at":"2025-05-07T04:58:42.929Z","avatar_url":"https://github.com/timescale.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BigQuery Driver\n\nA [database/sql](https://pkg.go.dev/database/sql) driver for BigQuery.\n\n## Features\n\n- Implements all modern [database/sql/driver](https://pkg.go.dev/database/sql/driver) interfaces.\n- Supports query cancellation and timeouts via [context.Context](https://pkg.go.dev/context).\n- Supports sessions (each [sql.Conn](https://pkg.go.dev/database/sql#Conn) maps\n  to a single [BigQuery session](https://cloud.google.com/bigquery/docs/sessions-intro)).\n- Supports transactions via [sql.DB.BeginTx](https://pkg.go.dev/database/sql#DB.BeginTx)\n  and related methods. Note that only the default [sql.IsolationLevel](https://pkg.go.dev/database/sql#IsolationLevel)\n  is supported, and read-only transactions are not supported.\n- Compliant with the [database/sql](https://pkg.go.dev/database/sql) package\n  interface. In particular, only valid [driver.Value](https://pkg.go.dev/database/sql/driver#Value)\n  types are returned. The driver therefore behaves as documented in the\n  [Rows.Scan](https://pkg.go.dev/database/sql#Rows.Scan) documentation.\n- Support for accessing the underlying [bigquery.Query](https://pkg.go.dev/cloud.google.com/go/bigquery#Query)\n  and [bigquery.Job](https://pkg.go.dev/cloud.google.com/go/bigquery#Job) types.\n  See [Accessing the Underlying Query/Job](#accessing-the-underlying-queryjob).\n\n## DSN\n\nTo connect via the [sql.Open](https://pkg.go.dev/database/sql#Open) method, use\n`bigquery` as the driver name and a DSN (Data Source Name) of the following form:\n\n```\nbigquery://projectID[/location][/dataset]?key=val\n```\n\nBoth the `location` and `dataset` are optional.\n\nCredentials can be passed via the `GOOGLE_APPLICATION_CREDENTIALS` environment\nvariable (see [Application Default\nCredentials](https://cloud.google.com/docs/authentication/application-default-credentials)\nfor more information).\n\nAlternatively, you can configure authentication via one of the `apiKey`,\n`credentials`, or `credentialsFile` options (see below).\n\n### Options\n\nOptions can be passed as key/value pairs in the query string. Supported options\ninclude:\n\n- `apiKey` - API Key to be used for authentication.\n- `credentials` - Base64-encoded service account or refresh token credentials\n  JSON object.\n- `credentialsFile` - Path to a file containing a service account or refresh\n  token credentials JSON object.\n- `scopes` - Overrides the default OAuth2 scopes to be used.\n- `endpoint` - Overrides the default endpoint to be used.\n- `userAgent` - Sets the User-Agent that is used when making requests to the\n  BigQuery API.\n- `disableAuth` - Set to `true` to disable all authentication methods. Primarily\n  useful in testing, or when accessing publicly accessible resources.\n\nIf you would like any other [option.ClientOption](https://pkg.go.dev/google.golang.org/api/option#ClientOption)\noptions to be supported via the DSN, feel free to a pull request or submit an\nissue.\n\n### Example\n\n```go\npackage main\n\nimport (\n\t\"database/sql\"\n\n\t_ \"github.com/timescale/bigquery-go-client\"\n)\n\nvar db, _ = sql.Open(\"bigquery\", \"bigquery://PROJECT_ID/LOCATION/DATASET?credentialsFile=/path/to/credentials.json\")\n```\n\n## Connector\n\nThe [NewConnector](https://pkg.go.dev/github.com/timescale/bigquery-go-client#NewConnector)\nmethod can be used to create a [driver.Connector](https://pkg.go.dev/database/sql/driver#Connector)\ninstance which can be passed to [sql.OpenDB](https://pkg.go.dev/database/sql#OpenDB).\nThis provides a more flexible and powerful way of opening a [sql.DB](https://pkg.go.dev/database/sql#DB)\ninstance, and supports all of the available [option.ClientOption](https://pkg.go.dev/google.golang.org/api/option#ClientOption)\noptions.\n\n### Example\n\n```go\npackage main\n\nimport (\n\t\"database/sql\"\n\n\tbigquery \"github.com/timescale/bigquery-go-client\"\n\t\"google.golang.org/api/option\"\n)\n\nvar db = sql.OpenDB(bigquery.NewConnector(bigquery.Config{\n\tProjectID: \"PROJECT_ID\",\n\tDataset:   \"DATASET\",\n\tLocation:  \"LOCATION\",\n\tOptions: []option.ClientOption{\n\t\toption.WithCredentialsFile(\"/path/to/credentials.json\"),\n\t},\n}))\n```\n\n## Data Types\n\nThe driver supports all [BigQuery data types](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types),\nwhich are scanned into the following Go types by default:\n\n| BigQuery Type | Go Type |\n| ------------- | ------- |\n| STRING | string |\n| BYTES | []byte |\n| INT64 | int64 |\n| FLOAT64 | float64 |\n| BOOL | bool |\n| TIMESTAMP | time.Time |\n| DATE | string |\n| TIME | string |\n| DATETIME | string |\n| NUMERIC | string |\n| BIGNUMERIC | string |\n| GEOGRAPHY | string |\n| INTERVAL | string |\n| RANGE | string |\n| JSON | []byte |\n| ARRAY | []byte |\n| STRUCT | []byte |\n\nNote that several types are returned as `string`/`[]byte` values instead of as\nthe more specific types that the underlying [bigquery.RowIterator.Next](https://pkg.go.dev/cloud.google.com/go/bigquery#RowIterator.Next)\nmethod returns by default. While this may seem inconvenient, it is necessary to\nsatisfy the [driver.Value](https://pkg.go.dev/database/sql/driver#Value)\nrequirements, which only allow a narrow range of predefined types. Adhering to\nthese requirements ensures that [sql.Rows.Scan](https://pkg.go.dev/database/sql#Rows.Scan)\nalways functions as described in the documentation.\n\nNote that the `ARRAY` and `STRUCT` types are returned as a `[]byte` value\ncontaining a JSON representation of the `ARRAY`/`STRUCT`. `STRUCT` types are\nmarshalled as JSON objects, where each field is represented as a key/value in\nthe JSON object. It should therefore be possible to call\n[json.Unmarshal](https://pkg.go.dev/encoding/json#Unmarshal) to convert the\n`[]byte` returned into a Go map, slice, or struct of your choosing.\n\nTo scan directly into a more complex type, create your own\n[sql.Scanner](https://pkg.go.dev/database/sql#Scanner) implementation (for\nexample, one that wraps one of the [civil](https://pkg.go.dev/cloud.google.com/go/civil)\ntypes, for `DATE`/`TIME`/`DATETIME`). Such types might be added to this package in\nthe future.\n\n## Accessing the Underlying Query/Job\n\nThis driver is a relatively thin wrapper around [cloud.google.com/go/bigquery](https://pkg.go.dev/cloud.google.com/go/bigquery),\nwhich offers a lot of functionality that cannot easily be exposed via the\n[database/sql](https://pkg.go.dev/database/sql) API. In particular, you may want\nto access the underlying [bigquery.Query](https://pkg.go.dev/cloud.google.com/go/bigquery#Query)\nor [bigquery.Job](https://pkg.go.dev/cloud.google.com/go/bigquery#Job) types\n(for example, to modify the query config before executing it, or to check the\njob statistics afterwards).\n\nThis can be achieved via the [GetQuery](https://pkg.go.dev/github.com/timescale/bigquery-go-client#GetQuery)\nand [GetJob](https://pkg.go.dev/github.com/timescale/bigquery-go-client#GetJob)\nfunction types, respectively. These functions are callbacks that provide access\nto the underlying query/job when passed to Query/Exec as arguments. They will\nbe called at the appropriate time in the query life cycle, providing access to\nthe underlying types, which can modified or inspected as needed.\n\nFor example, these functions can be used to execute a [dry\nrun](https://cloud.google.com/bigquery/docs/running-queries#dry-run) and print\nout the estimated total bytes processed:\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\n\tbq \"cloud.google.com/go/bigquery\"\n\t\"github.com/timescale/bigquery-go-client\"\n)\n\nfunc main() {\n\tdb, _ := sql.Open(\"bigquery\", \"bigquery://PROJECT_ID/LOCATION/DATASET?credentialsFile=/path/to/credentials.json\")\n\n\tqueryOpt := bigquery.GetQuery(func(q *bq.Query) {\n\t\tq.DryRun = true\n\t})\n\n\tjobOpt := bigquery.GetJob(func(j *bq.Job) {\n\t\tfmt.Printf(\"Total Bytes Processed: %d\\n\", j.LastStatus().Statistics.TotalBytesProcessed)\n\t})\n\n\tdb.QueryContext(context.Background(), \"SELECT * FROM my_table;\", queryOpt, jobOpt)\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimescale%2Fbigquery-go-client","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimescale%2Fbigquery-go-client","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimescale%2Fbigquery-go-client/lists"}