https://github.com/hckhanh/pg2bigquery
A CLI tool to convert query from PostgreSQL to BigQuery
https://github.com/hckhanh/pg2bigquery
big bigquery google pg pgsql postgres postgres-tool postgresql postgresql-database postgressql query query-parser querybuilder sql sql-toolkit sql-tools tool toolbox toolkit utility
Last synced: 9 months ago
JSON representation
A CLI tool to convert query from PostgreSQL to BigQuery
- Host: GitHub
- URL: https://github.com/hckhanh/pg2bigquery
- Owner: hckhanh
- Created: 2020-12-08T02:55:06.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2023-01-24T17:03:57.000Z (almost 3 years ago)
- Last Synced: 2025-03-18T11:24:59.069Z (9 months ago)
- Topics: big, bigquery, google, pg, pgsql, postgres, postgres-tool, postgresql, postgresql-database, postgressql, query, query-parser, querybuilder, sql, sql-toolkit, sql-tools, tool, toolbox, toolkit, utility
- Language: TypeScript
- Homepage:
- Size: 1.53 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 43
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# pg2bigquery
A CLI tool to convert query from PostgreSQL to BigQuery
[](https://www.deepcode.ai/app/gh/hckhanh/pg2bigquery/_/dashboard?utm_content=gh%2Fhckhanh%2Fpg2bigquery)
[](https://lgtm.com/projects/g/hckhanh/pg2bigquery/context:javascript)
[](https://oclif.io)
[](https://npmjs.org/package/pg2bigquery)
[](https://npmjs.org/package/pg2bigquery)
[](https://github.com/hckhanh/pg2bigquery/blob/master/package.json)
# Installation
For npm:
```shell
npm i -g pg2bigquery
```
or yarn:
```shell
yarn global add pg2bigquery
```
# Usage
## Requirements
```sh-session
$ pg2bigquery -d dataset -t tables.json input output
```
You need these things to use:
- **input**: input folder that contains PostgresSQL query files
- **output**: output folder that contains BigQuery query files
- **dataset**: destination dataset which is used to run BigQuery query
- **tables**: list of tables of pg database in json file
> If you want to know about the files format, go to [samples](samples)
### Tables JSON file
For now, you need to put the list of tables in a json file like this:
```json
["forms", "roles", "steps", "wards", "people", "staffs", "products"]
```
You can easily get the list of tables in your sql editor by this query:
```sql
select table_name from information_schema.tables;
```
Run `pg2bigquery` with `--help` option to get more details:
```sh-session
$ pg2bigquery --help
```
A success run will be:
```sh-session
$ pg2bigquery -d dataset -t tables.json input output
check output folder... done
get input files... 10 files
get tables... 55 tables
convert 10 files... done
```
# Limitations
> [No Silver Bullet](https://en.wikipedia.org/wiki/No_Silver_Bullet).
>
> The converted files might not be perfect.
When you use my tool, you will see these cases:
### `No matching signature for operator` (two different types)
```
No matching signature for operator = for argument types: INT64, STRING. Supported signature: ANY = ANY at [382:44]
```
BigQuery don't implicit cast one type to another when using with operators (+, -, <, >,...)
so you have to do it for yourself.