https://github.com/bgpat/glue2bq

Convert the table schema from AWS Glue to Google BigQuery
https://github.com/bgpat/glue2bq

Last synced: 5 days ago
JSON representation

Convert the table schema from AWS Glue to Google BigQuery

• Host: GitHub
• URL: https://github.com/bgpat/glue2bq
• Owner: bgpat
• License: mit
• Created: 2019-10-23T15:21:23.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2024-04-24T23:38:19.000Z (8 months ago)
• Last Synced: 2024-04-25T00:51:48.059Z (8 months ago)
• Language: Ruby
• Size: 7.81 KB
• Stars: 1
• Watchers: 3
• Forks: 0
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# glue2bq

Convert the table schema from AWS Glue to Google BigQuery

## example

```console

$ aws glue get-table --database-name=default --name=cloudtrail_logs | ruby main.rb

[{"name":"eventversion","type":"STRING","mode":"NULLABLE"},{"name":"useridentity","type":"RECORD","mode":"NULLABLE","fields":[{"name":"type","type":"STRING","mode":"NULLABLE"},{"name":"principalid","type":"STRING","mode":"NULLABLE"},{"name":"arn","type":"STRING","mode":"NULLABLE"},{"name":"accountid","type":"STRING","mode":"NULLABLE"},{"name":"invokedby","type":"STRING","mode":"NULLABLE"},{"name":"accesskeyid","type":"STRING","mode":"NULLABLE"},{"name":"username","type":"STRING","mode":"NULLABLE"},{"name":"sessioncontext","type":"RECORD","mode":"NULLABLE","fields":[{"name":"attributes","type":"RECORD","mode":"NULLABLE","fields":[{"name":"mfaauthenticated","type":"STRING","mode":"NULLABLE"},{"name":"creationdate","type":"STRING","mode":"NULLABLE"}]},{"name":"sessionissuer","type":"RECORD","mode":"NULLABLE","fields":[{"name":"type","type":"STRING","mode":"NULLABLE"},{"name":"principalid","type":"STRING","mode":"NULLABLE"},{"name":"arn","type":"STRING","mode":"NULLABLE"},{"name":"accountid","type":"STRING","mode":"NULLABLE"},{"name":"username","type":"STRING","mode":"NULLABLE"}]}]}]},{"name":"eventtime","type":"STRING","mode":"NULLABLE"},{"name":"eventsource","type":"STRING","mode":"NULLABLE"},{"name":"eventname","type":"STRING","mode":"NULLABLE"},{"name":"awsregion","type":"STRING","mode":"NULLABLE"},{"name":"sourceipaddress","type":"STRING","mode":"NULLABLE"},{"name":"useragent","type":"STRING","mode":"NULLABLE"},{"name":"errorcode","type":"STRING","mode":"NULLABLE"},{"name":"errormessage","type":"STRING","mode":"NULLABLE"},{"name":"requestparameters","type":"STRING","mode":"NULLABLE"},{"name":"responseelements","type":"STRING","mode":"NULLABLE"},{"name":"additionaleventdata","type":"STRING","mode":"NULLABLE"},{"name":"requestid","type":"STRING","mode":"NULLABLE"},{"name":"eventid","type":"STRING","mode":"NULLABLE"},{"name":"resources","mode":"NULLABLE","type":"RECORD","fields":[{"name":"arn","type":"STRING","mode":"NULLABLE"},{"name":"accountid","type":"STRING","mode":"NULLABLE"},{"name":"type","type":"STRING","mode":"NULLABLE"}]},{"name":"eventtype","type":"STRING","mode":"NULLABLE"},{"name":"apiversion","type":"STRING","mode":"NULLABLE"},{"name":"readonly","type":"STRING","mode":"NULLABLE"},{"name":"recipientaccountid","type":"STRING","mode":"NULLABLE"},{"name":"serviceeventdetails","type":"STRING","mode":"NULLABLE"},{"name":"sharedeventid","type":"STRING","mode":"NULLABLE"},{"name":"vpcendpointid","type":"STRING","mode":"NULLABLE"}]

```