https://github.com/zkan/data-engineering-on-gcp

Data Engineering on Google Cloud Platform (GCP)
https://github.com/zkan/data-engineering-on-gcp

bigquery data-engineering data-lake data-pipeline data-warehouse gcs google-cloud-platform machine-learning

Last synced: about 1 month ago
JSON representation

Data Engineering on Google Cloud Platform (GCP)

• Host: GitHub
• URL: https://github.com/zkan/data-engineering-on-gcp
• Owner: zkan
• License: cc0-1.0
• Created: 2023-08-29T12:19:01.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-01-27T07:26:37.000Z (about 1 year ago)
• Last Synced: 2024-04-14T13:48:42.783Z (10 months ago)
• Topics: bigquery, data-engineering, data-lake, data-pipeline, data-warehouse, gcs, google-cloud-platform, machine-learning
• Language: Python
• Homepage:
• Size: 4.96 MB
• Stars: 0
• Watchers: 3
• Forks: 31
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Data Engineering on Google Cloud Platform (GCP)

### Table of Contents

* [Prerequisites](#prerequisites)

* [Course Modules](#course-modules)

* [Datasets](#datasets)

## Prerequisites

1. Python

1. SQL

1. Git & GitHub

1. Command-Line Interface (CLI)

1. Docker

1. Google Cloud Platform (GCP)

## Course Modules

1. [Introduction to Data Engineering](01-introduction-to-data-engineering)

1. [Building a Data Warehouse with BigQuery](02-building-a-data-warehouse-with-bigquery)

1. [Setting up Data Lake using Google Cloud Storage](03-setting-up-data-lake-using-google-cloud-storage)

1. [Building Data Pipelines with Apache Airflow (Cloud Composer)](04-building-data-pipelines-with-apache-airflow-cloud-composer)

1. [Data Visualization with Looker Studio](05-data-visualization-with-looker-studio)

1. [Implementing Machine Learning Models in BigQuery ML](06-implementing-machine-learning-models-in-bigquery-ml)

1. [Real-world Capstone Project](07-real-world-capstone-project)

## Datasets

* [Breakfast at the Frat](https://github.com/zkan/open-data/tree/main/breakfast-at-the-frat)

* [Best Buy](https://github.com/zkan/open-data/tree/main/best-buy-apis)