{"id":15222448,"url":"https://github.com/takegue/bigquery-porter","last_synced_at":"2026-02-12T19:33:57.881Z","repository":{"id":57700761,"uuid":"502597134","full_name":"takegue/bigquery-porter","owner":"takegue","description":"BigQuery Deployment and Metadata Management tool","archived":false,"fork":false,"pushed_at":"2024-11-04T18:11:58.000Z","size":4154,"stargazers_count":2,"open_issues_count":17,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-09T02:30:32.861Z","etag":null,"topics":["bigquery"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/takegue.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-12T11:31:05.000Z","updated_at":"2024-08-18T11:46:22.000Z","dependencies_parsed_at":"2024-06-16T23:27:37.432Z","dependency_job_id":"266bfc60-01b5-44f9-b1e0-001f30860bf9","html_url":"https://github.com/takegue/bigquery-porter","commit_stats":{"total_commits":517,"total_committers":3,"mean_commits":"172.33333333333334","dds":"0.15087040618955516","last_synced_commit":"c7cf86c4c5ae6c9a6dde75e65e89da1399727f7c"},"previous_names":[],"tags_count":46,"template":false,"template_full_name":null,"purl":"pkg:github/takegue/bigquery-porter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takegue%2Fbigquery-porter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takegue%2Fbigquery-porter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takegue%2Fbigquery-porter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takegue%2Fbigquery-porter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/takegue","download_url":"https://codeload.github.com/takegue/bigquery-porter/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takegue%2Fbigquery-porter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29378819,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-12T19:05:20.189Z","status":"ssl_error","status_checked_at":"2026-02-12T19:01:44.216Z","response_time":55,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery"],"created_at":"2024-09-28T15:12:04.300Z","updated_at":"2026-02-12T19:33:57.862Z","avatar_url":"https://github.com/takegue.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# BigQuery Porter\n\n[![CI - testing](https://github.com/takegue/bigquery-porter/actions/workflows/ci.yml/badge.svg)](https://github.com/takegue/bigquery-porter/actions/workflows/ci.yml)\n\nBigQuery Porter is a tool to manage BigQuery Resource and its metadata by filesystem.\n\n![Demo](./docs/assets/demo.gif)\n\n## Get Started\n\n### Installation\n\n```\nnpm i bigquery-porter\n```\n\nSet up your OAuth and GCP Default Project.\n\n### Clone BigQuery resources from your GCP Project\n\nFirst step to start to use is cloning BigQuery resources as files in remote.\nNext comamnd, you'll get BigQuery metadata files and DDL codes in local filesystem.\n\n```sh\nnpx bqport pull --all --with-ddl\n```\n\nTypical directory strucuture follows:\n\n```\n(Root Directory)\n`-- @default: Your Default GCP Project\n    `-- sandbox\n        |-- ddl.sql\n        |-- metadata.json\n        |-- @models\n        |   `-- mymodel\n        |       |-- ddl.sql\n        |       `-- metadata.json\n        |-- @routines\n        |   |-- sample_function\n        |   |   |-- ddl.sql\n        |   |   `-- metadata.json\n        |   |-- sample_procedure\n        |   |   |-- ddl.sql\n        |   |   `-- metadata.json\n        |   `-- sample_tvf\n        |       |-- ddl.sql\n        |       `-- metadata.json\n        |-- sample_clone_table\n        |   |-- ddl.sql\n        |   |-- metadata.json\n        |   `-- schema.json\n        |-- sample_materialized_view\n        |   |-- ddl.sql\n        |   |-- metadata.json\n        |   `-- schema.json\n        |-- sample_partition_table\n        |   |-- ddl.sql\n        |   |-- metadata.json\n        |   `-- schema.json\n        |-- sample_snapshot_table\n        |   |-- ddl.sql\n        |   |-- metadata.json\n        |   `-- schema.json\n        |-- sample_table\n        |   |-- ddl.sql\n        |   |-- metadata.json\n        |   `-- schema.json\n        `-- sample_view\n            |-- metadata.json\n            |-- schema.json\n            `-- view.sql\n\n`-- \u003cother_projects\u003e\n```\n\n#### Deploy Your BigQuery Resources\n\nAfter you modifeid files in locals, you can deploy them by next command:\nThe following command will deploy to the default project in GCP.\n\n```\nnpx bqport push\n```\n\n`bqport` deploys codes in prallel and topological order by SQL's DAG dependencies.\n\n\n#### Partial Deployment from STDIN\n\nYou can also deploy your BigQuery queries from a list of SQL files.\nThis is useful for cases where you want to execute only specific SQL.\nFor example, a diff execution by git.\n\n```\nfind ./bigquery -name '*.sql' | npx bqport push\n```\n\n## Dependency Resolution Behavior\n\n`bigquery-porter` resolves dependencies in following order\n\n0. DDL(DROP): `DROP TABLE` statement \n0. DDL(CREATE): `CREATE TABLE` statement\n0. DDL(ALTER): `ALTER TABLE` statement\n0. DML: `INSERT INTO`, `DELETE`,`TRUNCATE`, `MERGE` statements\n0. QUERY\n\n## Tips\n\n### Vendoring BigQuery Datasets\n\nFor Example, Let's copy public datasets in `bqutil` project .\n\n1. Fetch Dataset Metadata and DDL SQL file\n\n```\nnpx bqport pull bqutil --all --with-ddl\n```\n\n2. Make your new destination project directory and copy files\n\n```\nmkdir -p bigquery/@default\ncp -av bigquery/bqutil/fn bigquery/@default/fn\n```\n\n3. Fix old destination in SQL files to new one\n\n```\nnpx bqport format\n```\n\n4. Deploy\n\n```\nnpx bqport push @default\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftakegue%2Fbigquery-porter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftakegue%2Fbigquery-porter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftakegue%2Fbigquery-porter/lists"}