https://github.com/kenthsu/udacity-data-engineering-nanodgree
Udacity Data Engineering Nanodegree Program
https://github.com/kenthsu/udacity-data-engineering-nanodgree
apache-airflow apache-cassandra apache-spark aws-redshift aws-s3 data-engineering data-lake data-pipelines data-quality data-warehouses postgresql
Last synced: about 1 year ago
JSON representation
Udacity Data Engineering Nanodegree Program
- Host: GitHub
- URL: https://github.com/kenthsu/udacity-data-engineering-nanodgree
- Owner: KentHsu
- Created: 2021-01-19T18:01:12.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2021-03-04T07:29:22.000Z (over 5 years ago)
- Last Synced: 2025-03-24T05:51:44.953Z (over 1 year ago)
- Topics: apache-airflow, apache-cassandra, apache-spark, aws-redshift, aws-s3, data-engineering, data-lake, data-pipelines, data-quality, data-warehouses, postgresql
- Language: Jupyter Notebook
- Homepage:
- Size: 2.12 MB
- Stars: 51
- Watchers: 1
- Forks: 58
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Udatcity - Data Engineering Nanodgree Program
Learn to design data models, build data warehouses and data lakes, automate data pipelines, and work with massive datasets.
* Create user-friendly relational and NoSQL data models
* Create scalable and efficient data warehouses
* Work efficiently with massive datasets
* Build and interact with a cloud-based data lake
* Automate and monitor data pipelines
* Develop proficiency in Spark, Airflow, and AWS tools
## Course 1 - Data Modeling
Learn to create relational and NoSQL data models to fit the diverse needs of data consumers. Use ETL to build databases in PostgreSQL and Apache Cassandra.
### Contents
* Introduction to Data Modeling
* Relational Data Models
* NoSQL Data Models
### Projects
* Data Modeling with Postgres
* Data Modeling with Apache Cassandra
## Course 2 - Cloud Data Warehouses
Learn to create cloud-based data warehouses. Sharpen your data warehousing skills, deepen your understanding of data infrastructure, and be introduced to data engineering on the cloud using Amazon Web Services (AWS).
#### Contents
* Introduction to the Data Warehouses
* Introduction to the Cloud with AWS
* Implementing Data Warehouses on AWS
### Project
* Build a Cloud Data Warehouse
## Course 3 - Data Lake with Spark
Learn more about the big data ecosystem and how to use Spark to work with massive datasets. Learn about how to store big data in a data lake and query it with Spark.
#### Contents
* The Power of Spark
* Data Wrangling with Spark
* Debugging and Optimization
* Introduction to Data Lake
### Project
* Build a Data Lake
## Course 4 - Data Pipelines with Airflow
Learn to schedule, automate, and monitor data pipelines using Apache Airflow. Learn to run data quality checks, track data lineage, and work with data pipelines in production.
#### Contents
* Data Pipelines
* Data Quality
* Production Data Pipelines
### Project
* Data Pipelines with Airflow
## Final Project - DEND Capstone Project
Combine all the skills throughout the program to build your own data engineering portfolio project.
### Project
* Data Engineer Capstone