https://github.com/hanan-nawaz/100_days_of_data_engineering

Journey through 100 days of Data Engineering, featuring daily learning, practice, and projects. This repository includes notes, exercises, and code snippets covering essential topics such as GitHub, Python, ETL, data pipelines, and more
https://github.com/hanan-nawaz/100_days_of_data_engineering

data-engineer data-engineering git github python3

Last synced: 3 months ago
JSON representation

Journey through 100 days of Data Engineering, featuring daily learning, practice, and projects. This repository includes notes, exercises, and code snippets covering essential topics such as GitHub, Python, ETL, data pipelines, and more

• Host: GitHub
• URL: https://github.com/hanan-nawaz/100_days_of_data_engineering
• Owner: Hanan-Nawaz
• License: mit
• Created: 2024-08-07T13:05:37.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-08-13T15:48:28.000Z (almost 2 years ago)
• Last Synced: 2025-07-21T02:43:36.012Z (11 months ago)
• Topics: data-engineer, data-engineering, git, github, python3
• Language: Jupyter Notebook
• Homepage: https://hanannawaz.me
• Size: 10.5 MB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# 100 Days of Data Engineering

Welcome to the **100 Days of Data Engineering** challenge! This repository documents my journey of mastering key data engineering concepts, tools, and technologies over the course of 100 days. Each day, I will tackle a specific topic, learn the theory, and practice through hands-on exercises.

## Technologies and Topics Covered

Throughout this challenge, I will focus on the following areas:

- **Python**: Core programming language for data engineering.

- **Pandas**: Data manipulation and analysis library.

- **SQL**: Structured Query Language for relational databases.

- **NoSQL**: Databases like MongoDB for unstructured data.

- **Linux**: Command-line tools and scripting.

- **ETL / Data Pipeline**: Building and managing data workflows.

- **Data Warehousing / Snowflake**: Designing and managing data warehouses using Snowflake.

- **Apache Spark**: Big data processing framework.

- **Apache Kafka**: Distributed event streaming platform.

- **Apache Airflow**: Workflow orchestration platform.

- **Power BI**: Business analytics tool.

- **AWS**: Cloud services for data engineering.

- **Git**: Version control for managing code and projects.

## Daily Progress

| **Day**        | **Date**       | **Tasks Completed**                                                                                                               |

|----------------|----------------|-----------------------------------------------------------------------------------------------------------------------------------|

| **Day 1**      | **7 Aug 2024** | - **GitHub basics**: Learned fundamentals and documented notes.
- **Python Chapters 1-6**: Organized notes and completed exercises. |

| **Day 2**      | **8 Aug 2024** | - **Chapter 7 - Lists**: Studied lists, practiced questions.
- **List comprehension**: Learned and applied it in exercises. |

| **Day 3** | **9 Aug 2024** | - **Chapter 8 - Tuples**: Studied tuples, understood their properties, and practiced exercises.
- **Immutability**: Focused on the concept of immutability and how it applies to tuples. |

| **Day 4** | **10 Aug 2024** | - **Chapter 9 - Dicitionary**: Studied Dicitionary, understood their properties, and practiced exercises. |

| **Day 5** | **11 Aug 2024** | - **Chapter 10 - Strings**: Studied Strings, understood their properties, and practiced exercises. |

| **Day 6** | **12 Aug 2024** | - **Chapter 11 - Functions**: Studied Functions and Lambda Functions , understood their properties, and practiced exercises. |


> **Note:** Python notes were sourced from [Code and Debug](https://www.codeanddebug.in).

## How to Use This Repository

- Each folder is organized with relevant notes, code, and practice exercises.

- Check out the specific technology folders for detailed learning paths and examples.

- The learning schedule and content may vary based on prior experience and individual progress.

> **Disclaimer:** I am not an absolute beginner. With nearly 9 months of field experience as a Data Engineer, this challenge is tailored to build on existing knowledge and may progress at a different pace compared to someone just starting in the field.

Feel free to explore, learn, and contribute!

## License

This project is licensed under the MIT License.