Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/esanchezros/bigquery-maven-plugin
Maven plugin for managing BigQuery datasets, tables and views
https://github.com/esanchezros/bigquery-maven-plugin
bigquery java maven maven-plugin
Last synced: about 1 month ago
JSON representation
Maven plugin for managing BigQuery datasets, tables and views
- Host: GitHub
- URL: https://github.com/esanchezros/bigquery-maven-plugin
- Owner: esanchezros
- Archived: true
- Created: 2018-07-28T16:13:11.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2022-11-16T09:21:26.000Z (almost 2 years ago)
- Last Synced: 2024-09-25T19:24:11.899Z (about 1 month ago)
- Topics: bigquery, java, maven, maven-plugin
- Language: Java
- Homepage:
- Size: 58.6 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 4
-
Metadata Files:
- Readme: README.adoc
Awesome Lists containing this project
README
image:https://travis-ci.org/esanchezros/bigquery-maven-plugin.svg?branch=master["Build Status", link="https://travis-ci.org/esanchezros/bigquery-maven-plugin"]
image:https://codecov.io/gh/esanchezros/bigquery-maven-plugin/branch/master/graph/badge.svg["codecov", link="https://codecov.io/gh/esanchezros/bigquery-maven-plugin"]
image:https://img.shields.io/hexpm/l/plug.svg["Apache 2", link="http://www.apache.org/licenses/LICENSE-2.0"]
= BigQuery Maven plugin
This Maven plugin provides goals to create datasets, tables and views in Google BigQuery.
== How to use
In your application, add the following plugin to your pom.xml:
[source, xml]
----
io.allune
bigquery-maven-plugin
1.0.0
...
----
== Supported goals
|===
|Goal | Description
|bigquery:create|Creates the dataset, tables and views defined in the plugin configuration.
|bigquery:create-dataset|Creates the dataset defined in the plugin configuration.
|bigquery:clean|Removes the dataset, tables and views defined in the plugin configuration.
|bigquery:help|Displays help information on the plugin. Use `mvn bigquery:help -Ddetail=true -Dgoal=[goal]` for detailed goal documentation.
|===
== Example plugin configuration
[source, xml]
----
io.allune
bigquery-maven-plugin
1.0.0
your_project_id
/credentials.json
your_dataset
EU
create
create
pre-integration-test
${skipTests}
true
gs://folder/data.json
CSV
file://${project.basedir}/src/main/resources/bigquery/schemas/dir1
classpath:/bigquery/schemas/dir2
file://${project.basedir}/src/main/resources/bigquery/views
clean
clean
post-integration-test
${skipTests}
true
true
----
== Schema definition
A JSON schema file consists of a JSON array that contains the following:
* (Optional) The column's description
* The column name
* The column's data type
* (Optional) The column's mode (if unspecified, mode defaults to NULLABLE)
Reference: https://cloud.google.com/bigquery/docs/schemas
=== Table name
The schema file name is used as the table name.
=== Example
==== test_table
[source, json]
----
[
{
"name": "id",
"mode": "NULLABLE",
"type": "STRING"
},
{
"name": "subject",
"mode": "NULLABLE",
"type": "STRING"
},
{
"name": "from",
"mode": "NULLABLE",
"type": "STRING"
},
{
"name": "to",
"mode": "NULLABLE",
"type": "STRING"
},
{
"name": "cc",
"mode": "NULLABLE",
"type": "STRING"
},
{
"name": "body",
"mode": "NULLABLE",
"type": "STRING"
},
{
"name": "time",
"mode": "NULLABLE",
"type": "TIME"
},
{
"name": "timestamp",
"mode": "NULLABLE",
"type": "TIMESTAMP"
},
{
"name": "fields",
"type": "RECORD",
"mode": "REPEATED",
"fields": [
{
"name": "field1",
"type": "STRING",
"mode": "REQUIRED"
},
{
"name": "moreFields",
"type": "RECORD",
"mode": "REPEATED",
"fields": [
{
"name": "field1",
"type": "STRING",
"mode": "REQUIRED"
},
{
"name": "field2",
"type": "STRING",
"mode": "REQUIRED"
}
]
}
]
}
]
----