{"id":19317853,"url":"https://github.com/queeniecplusplus/bigquery_cloudsql","last_synced_at":"2026-03-04T04:01:57.092Z","repository":{"id":104588129,"uuid":"310500015","full_name":"QueenieCplusplus/BigQuery_CloudSQL","owner":"QueenieCplusplus","description":null,"archived":false,"fork":false,"pushed_at":"2020-11-07T09:52:33.000Z","size":1344,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-24T04:45:03.046Z","etag":null,"topics":["bigdata","buckets","cloudsql","mysql-server","sql"],"latest_commit_sha":null,"homepage":"https://github.com/QueenieCplusplus/QuickGoThru/blob/master/README.md#cloud-sql-big-data","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/QueenieCplusplus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-06T05:23:05.000Z","updated_at":"2020-11-07T09:52:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"3af7c1ae-f2a7-4df0-8008-4b176cb5d17a","html_url":"https://github.com/QueenieCplusplus/BigQuery_CloudSQL","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/QueenieCplusplus/BigQuery_CloudSQL","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QueenieCplusplus%2FBigQuery_CloudSQL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QueenieCplusplus%2FBigQuery_CloudSQL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QueenieCplusplus%2FBigQuery_CloudSQL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QueenieCplusplus%2FBigQuery_CloudSQL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/QueenieCplusplus","download_url":"https://codeload.github.com/QueenieCplusplus/BigQuery_CloudSQL/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QueenieCplusplus%2FBigQuery_CloudSQL/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30071670,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-04T03:25:38.285Z","status":"ssl_error","status_checked_at":"2026-03-04T03:25:05.086Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigdata","buckets","cloudsql","mysql-server","sql"],"created_at":"2024-11-10T01:16:27.181Z","updated_at":"2026-03-04T04:01:57.078Z","avatar_url":"https://github.com/QueenieCplusplus.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# BigQuery_CloudSQL\n\nSQL (Structured Query Language) is a standard language for Data Ops that allows DBA to query from structured datasets. \n\nIt's commonly used in DB mgmt and allows to perform tasks like transaction record writing into relational databases and do DA (petabyte-scale data analysis).\n\nThis is a data science topics. This lab is divided into two parts: \n\nin the first half, you will learn how to export subsets of the London bikeshare dataset into CSV files, which you will then upload to Cloud SQL. From there you will learn how to use Cloud SQL to create and manage databases and tables. Towards the end, you will get hands-on practice with additional SQL keywords that manipulate and edit data.\n\n![Google Cloud SQL](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/google-cloud-sql.jpg)\n\nIn the second half, you will learn fundamental SQL querying keywords, which you will run in the BigQuery console on a public dataset that contains information on London bikeshares.\n\n![Google CBig Query](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/google-bigquery.jpg)\n\n\n# Core Steps:\n\n(1) uoload CSV data to Storage/Bucket.\n\n(2) Create Cloud SQL instance, config AC.\n\n(3) Create Connection between Storage/Bucket to SQL instance.\n\n(4) as same as (1)\n\n(5) util Google Big Data Production Tool called Big Query\n\n--------------\n\n# Storage Bucket\n\nfrom step 1:\n\n\u003e upload Data to Storage Bucket\n\n* 1.1, in cloud console, navigate to Storage \u003e\u003e Browser, click on \"Create Bucket\".\n\n\n    ![storage bucket](https://cdn.qwiklabs.com/MJaLpJcY4bF7yM0I4XC%2BlzCe3F32kXqqayPLGZ5vK4Q%3D)\n    \n* 1.2, upload CSV data to the newly created Cloud Storage Bucket.\n\n    ![upload](https://cdn.qwiklabs.com/O0gGDUAw3%2BKFgvwpeQvYtmRFgfAlChH09mZMXpztL%2FM%3D)\n    \n# Cloud SQL instance\n\n![Google Cloud SQL](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/google-cloud-sql.jpg)\n\nfrom step 2:\n\n\u003e create Cloud SQL instance\n\n* 2.1, navigate to Storage \u003e\u003e SQL, then click on \"create Cloud SQL instance\".\n\n    ![cloud sql instance](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/sql_instance.png)\n\n* 2.2, check the user \u0026 password to do Access Control to DB.\n\n    ![AC](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/AC.png)\n\n* 2.3, Cloud SQL Instance is created then.\n\n    ![](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/cloud_sql_instance_created_1.png)\n\n# Connection using Cloud Shell\n\nfrom step 3:\n\n\u003e activate Cloud Shell and type cmd line. (auth, project resource), then make it to connect with sql instance.\n\n* 3.1, activate Cloud Shell.\n\n![gcloud](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/Cloud%20Shell.png)\n\n* 3.2, type cmd line in shell, to check Project ID and auth Users Account.\n\n         gcloud auth list\n         \n         [output]\n         Credentialed accounts: -\u003cusername\u003e@\u003chostname\u003e.com (active)\n     \n         \n         gcloud config list project\n         \n         [output]\n         [Core]\n         Project=\u003cpoj_id\u003e\n         \n         [example output]\n         Project=kates-gcp-1234567890abcedee\n         // as above blue bar in step 1.5\n\n\n* 3.3, make connection to sql.\n\n         gcloud sql connect [cloud sql instance name] --user=root\n         // the name is as same as pink bar in step 2.3\n\n         [output]\n         \n            Welcome to the MariaDB monitor.  Commands end with ; or \\g.\n            Your MySQL connection id is 494\n            Server version: 5.7.14-google-log (Google)\n\n            Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.\n\n            Type 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.\n\n            MySQL [(none)]\u003e\n            \n * 3.4, execute SQL statement now.\n  \n           CREATE DATABASE poupou;\n           \n            USE poupou;\n            CREATE TABLE taiwanHome (start_station_name VARCHAR(255), num INT);\n            \n            SELECT * FROM taiwanHome;\n            \n            [output]\n            It says \"empty set\" because you haven't loaded in any data yet.\n  \n # Import Data to  Cloud SQL instance\n  \n  from step 4:\n  \n  \u003e upload csv data to table.\n  \n  * 4.1, in the console for Cloud SQL Instance, click on \"Import\".\n  \n  * 4.2, then browse the Storage Bucket File.\n  \n  * 4.3, select the right file, and click on \"Import\".\n  \n  * 4.4, CSV files uploaded to tables called taiwanHome in the poupou database.\n\n        [output UI]\n        \n      ![](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/import.png)\n      \n  * 4.5, do Big Query now!\n  \n       Return to your Cloud Shell session and run the following command at the MySQL server prompt to inspect the contents of table called taiwanHome, run sql cmd at the sql server prompt (we choose hereby the MySQL server. see step 2.1)\n\n        SELECT * FROM taiwanHome;\n\n        SELECT * FROM chinaHome;\n\n\n\n        DELETE FROM taiwanHome WHERE num=0;\n\n\n\n\n        INSERT INTO taiwanHome (start_station_name, num) VALUES (\"test destination\", 1);\n\n\n\n\n        SELECT start_station_name AS top_stations, num FROM london1 WHERE num\u003e100000\n        UNION\n        SELECT end_station_name, num FROM london2 WHERE num\u003e100000\n        ORDER BY top_stations DESC;\n\n\n# Big Query\n\n![Google CBig Query](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/google-bigquery.jpg)\n\nfrom step 5:\n\n\u003e util Big Query\n\n* 5.1, in cloud console, navigate to BigData \u003e BigQuery, and both of the BigQuery Console and the Message Box are opened.\n\n   ![big query](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/Big%20Query%20Console.png)\n\n* 5.2. check project id, then add data, and check DB \u0026 table.\n\n check the project → dataset → table of the console, you can load up some queryable data.\n\n  ![](https://raw.githubusercontent.com/QueenieCplusplus/BigQuery_CloudSQL/main/add%20data%200.png)\n\n* tips \u0026 attentions:\n\nIt's important to note that you are still working out of your \"main project\" in this left-hands tab. \n\nAll you did was pull a publicly accessible project that contains datasets and tables into BigQuery for analysis.It means you didn't switch over to that \"main project\". (see green bar in step 5.2)\n\nAll of your jobs and services are still tied to your account. You can see this for yourself by inspecting the project field near the top of the console. \n\n# Ref Code\n\n2802, 3642(trouble shoot)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqueeniecplusplus%2Fbigquery_cloudsql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqueeniecplusplus%2Fbigquery_cloudsql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqueeniecplusplus%2Fbigquery_cloudsql/lists"}