{"id":20113139,"url":"https://github.com/cloudacademy/optimizing-bigquery","last_synced_at":"2026-03-08T14:40:32.930Z","repository":{"id":52642352,"uuid":"92524718","full_name":"cloudacademy/optimizing-bigquery","owner":"cloudacademy","description":null,"archived":false,"fork":false,"pushed_at":"2023-09-05T19:51:14.000Z","size":7604,"stargazers_count":12,"open_issues_count":0,"forks_count":20,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-01-13T06:07:29.562Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudacademy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-26T15:35:55.000Z","updated_at":"2022-12-18T18:43:20.000Z","dependencies_parsed_at":"2024-11-13T18:26:02.330Z","dependency_job_id":"5a205e76-6204-4a13-a7f3-513f0b24accd","html_url":"https://github.com/cloudacademy/optimizing-bigquery","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudacademy%2Foptimizing-bigquery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudacademy%2Foptimizing-bigquery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudacademy%2Foptimizing-bigquery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudacademy%2Foptimizing-bigquery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudacademy","download_url":"https://codeload.github.com/cloudacademy/optimizing-bigquery/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241557116,"owners_count":19981877,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T18:23:24.904Z","updated_at":"2025-11-08T01:02:07.851Z","avatar_url":"https://github.com/cloudacademy.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"## Optimizing Google BigQuery\nThis file contains text you can copy and paste for the examples in Cloud Academy's _Optimizing Google BigQuery_ course.\n\n### Reducing the Amount of Data Processed\n#### Stock exchange data files\nGBPUSD_2014_01.csv and GBPUSD_2014_02.csv are no longer available from Google, but you can download smaller versions of them from this repository.  \n\nThe schema for the above tables is:  \nvenue:STRING,currencies:STRING,time:TIMESTAMP,bid:FLOAT,ask:FLOAT\n\n#### BETWEEN operator\n```sql\nSELECT time, bid\nFROM examples.gbpusd_201401\nWHERE time\n  BETWEEN TIMESTAMP(\"2014-01-01 00:00:00\")\n  AND TIMESTAMP(\"2014-01-01 00:30:00\")\nORDER BY time ASC\n```\n\n#### Wildcard in table reference\n```\nSELECT MIN(time) AS mintime, MAX(time) AS maxtime\nFROM `examples.gbpusd_20140*`\n```\n\n### Partitioned Tables\n#### Create partitioned table\n```\nbq query --use_legacy_sql=false --replace --destination_table 'examples.gbpusd_201401p' --time_partitioning_field time \"SELECT * from examples.gbpusd_201401\"\n```\n\n### Denormalized Data Structures\n#### MusicBrainz data files\n__***Note: The files below are no longer available. We are working on uploading smaller versions of them to this repository.__\n\n| Table         | Data                          | Schema                                                 |\n| ------------------ | ----------------------------- | ------------------------------------------------------ |\n| artist             | gs://solutions-public-assets/bqetl/artist.json | https://storage.googleapis.com/solutions-public-assets/bqetl/artist_schema.json |\n| artist_credit_name | gs://solutions-public-assets/bqetl/artist_credit_name.json | https://storage.googleapis.com/solutions-public-assets/bqetl/artist_credit_name_schema.json|\n| recording            | gs://solutions-public-assets/bqetl/recording.json | https://storage.googleapis.com/solutions-public-assets/bqetl/recording_schema.json |\n\n#### Create denormalized table\n```\nSELECT\n  artist.id, artist.gid AS artist_gid, artist.name AS artist_name, artist.area,\n  recording.name AS recording_name, recording.length, recording.gid, recording.video\nFROM\n  `examples.artist` AS artist\nINNER JOIN\n  `examples.artist_credit_name` AS artist_credit_name\nON\n  artist.id = artist_credit_name.artist\nINNER JOIN\n  `examples.recording` AS recording\nON\n  artist_credit_name.artist_credit = recording.artist_credit\n```\n\n#### Query denormalized table\n```\nSELECT artist_name, recording_name\nFROM `cloud-academy-content-team.examples.recording_by_artist`\nWHERE artist_name LIKE '%Elvis%'\n```\n\n### Nested Repeated Fields\n#### Example data\n| Table         | Data                          | Schema                                                 |\n| ------------------ | ----------------------------- | ------------------------------------------------------ |\n| persons_data       | https://raw.githubusercontent.com/cloudacademy/optimizing-bigquery/master/personsData.json | https://raw.githubusercontent.com/cloudacademy/optimizing-bigquery/master/personsDataSchema.json |\n\n#### Query a nested field\n```\nSELECT fullName, phoneNumber.number\nFROM `cloud-academy-content-team.examples.persons_data`\n```\n\n#### Unnest a field\n```\nSELECT fullName, place\nFROM `cloud-academy-content-team.examples.persons_data`,\nUNNEST(citiesLived)\nWHERE place = \"Austin\"\n```\n\n#### Query a view\n```\nSELECT fullName, place\nFROM `cloud-academy-content-team.examples.cities_by_person`\nWHERE place = 'Stockholm'\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudacademy%2Foptimizing-bigquery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudacademy%2Foptimizing-bigquery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudacademy%2Foptimizing-bigquery/lists"}