Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/razevedo1994/data_engineer_roadmap

Personal roadmap to guide my studies.
https://github.com/razevedo1994/data_engineer_roadmap
Last synced: 13 days ago
JSON representation
Personal roadmap to guide my studies.
Host: GitHub
URL: https://github.com/razevedo1994/data_engineer_roadmap
Owner: razevedo1994
Created: 2022-04-14T02:54:29.000Z (over 2 years ago)
Default Branch: main
Last Pushed: 2022-05-07T14:35:16.000Z (over 2 years ago)
Last Synced: 2024-08-01T22:54:22.802Z (3 months ago)
Size: 5.86 KB
Stars: 80
Watchers: 3
Forks: 8
Open Issues: 0
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project

README

        [![author](https://img.shields.io/badge/author-razevedo1994-red.svg)](https://www.linkedin.com/in/azevedo94/) [![GPLv3 license](https://img.shields.io/badge/License-GPLv3-blue.svg)](http://perso.crans.org/besson/LICENSE.html) [![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/razevedo1994/data_engineer_roadmap/issues)

# Data Engineer Roadmap

Created by: [Rodrigo Azevedo](https://www.linkedin.com/in/azevedo94/)

- `YouTube` [How Data Engineering Works](https://www.youtube.com/watch?v=qWru-b6m030)

### Code versioning

- [What is version control?](https://www.atlassian.com/git/tutorials/what-is-version-control)

- [Versionamento Semântico](https://semver.org/lang/pt-BR/)

- `YouTube` [Curso completo de GIT](https://www.youtube.com/watch?v=OuOb1_qADBQ)

### Programming Language

- `LinuxTips` [Python Base](https://www.linuxtips.io/products/python-base)

- `YouTube` [Python for Data Engineering](https://www.youtube.com/watch?v=yl3uoL050gw&list=PL50mYnndduIHs5KReaz7F7aSelC-zG2zP)

- `Udemy` [Complete Python Bootcamp From Zero to Hero in Python](https://www.udemy.com/course/complete-python-bootcamp/?ranMID=39197&ranEAID=HTtUFxqit0c&ranSiteID=HTtUFxqit0c-1OxN5idcVF_YM1Xge1wiYg&utm_source=aff-campaign&LSNPUBID=HTtUFxqit0c&utm_medium=udemyads)

### Python Virtual Environments

- `YouTube`[Ambientes virtuais e instalação de bibliotecas](https://www.youtube.com/watch?v=naGF7EIUFp0)

- `Real Python`[Python Virtual Environments: A Primer](https://realpython.com/python-virtual-environments-a-primer/)

### SQL

- `Twitch` [Curso de SQL do Zero - Teo Calvo](https://www.twitch.tv/collections/1GD1BC-Y3BbhgA)

- `Udemy` [The Complete SQL Bootcamp](https://www.udemy.com/course/the-complete-sql-bootcamp/?ranMID=39197&ranEAID=HTtUFxqit0c&ranSiteID=HTtUFxqit0c-yn4syphFt3wsdlQl_2Sl0g&utm_source=aff-campaign&utm_medium=udemyads&LSNPUBID=HTtUFxqit0c)

- `YouTube` [SQL Tutorial - Full Database Course for Beginners](https://www.youtube.com/watch?v=HXV3zeQKqGY)

### Linux

- `YouTube` [Linux Command Line Full course](https://www.youtube.com/watch?v=2PGnYjbYuUo)

- `Udemy` [Linux for Beginners](https://www.udemy.com/course/linux-for-beginners-2021/?ranMID=39197&ranEAID=HTtUFxqit0c&ranSiteID=HTtUFxqit0c-hFZAmF_V4qoQMac49XY_Wg&LSNPUBID=HTtUFxqit0c&utm_source=aff-campaign&utm_medium=udemyads)

- [Shell Script: Introdução e Como Automatizar Tarefas](https://www.alura.com.br/artigos/automatizando-tarefas-com-shell-script)

- [Introduction to automation with Bash scripts](https://opensource.com/article/19/12/automation-bash-scripts)

- [Bash Automation & Scripting Basics](https://www.howtogeek.com/devops/bash-automation-scripting-basics-part-1/)

- [27 Simple Bash Scripting Tips for Beginners](https://betterprogramming.pub/27-simple-bash-scripting-tips-for-beginners-d6764c977546)

- [The Ultimate Programmer’s Guide to Bash Scripting](https://betterprogramming.pub/the-ultimate-programmers-guide-to-bash-scripting-2d11d4e6e978)

- [8 Bash Script Tips for Automation Projects](https://towardsdatascience.com/8-bash-script-tips-for-automation-projects-2c63d8716f9a)

### HTTP/HTTPS

- `YouTube` [HTTP Crash Course & Exploration](https://www.youtube.com/watch?v=iYM2zFP3Zn0)

### API's

- `YouTube` [APIs for Beginners - How to use an API](https://www.youtube.com/watch?v=GZvSYJDk-us&t=6965s)

- `Youtube` [What Is A RESTful API? Explanation of REST & HTTP](https://www.youtube.com/watch?v=Q-BpqyOT3a8)

### Docker

- [Começando com Docker](https://www.alura.com.br/artigos/comecando-com-docker)

- `YouTube` [Docker Tutorial for Beginners](https://www.youtube.com/watch?v=3c-iBn73dDE)

- `YouTube` [Docker Compose Tutorial](https://www.youtube.com/watch?v=HG6yIjZapSA)

- [Dockerfile reference](https://docs.docker.com/engine/reference/builder/)

- [Dockerfile: principais comandos e instruções](https://dev.to/soutoigor/dockerfile-principais-comandos-e-instrucoes-2jpp)

- [Docker: Desvendando o DockerFile](https://www.alura.com.br/artigos/desvendando-o-dockerfile?gclid=Cj0KCQjwsdiTBhD5ARIsAIpW8CIz8NZVdIX37M4d2e0VzHUwx7UC2G9D1Lh_gKzbHtkgRJazXZpnxxwaAvmIEALw_wcB)

- [Dockerfile - Principais comandos para criar a receita da imagem](https://blog.rocketseat.com.br/dockerfile-principais-comandos-para-criar-a-receita-da-imagem/)

- [Dockerfile cheat sheets](https://kapeli.com/cheat_sheets/Dockerfile.docset/Contents/Resources/Documents/index)

| Concepts                                                                                                                                |

|-----------------------------------------------------------------------------------------------------------------------------------------|

| [Big Data: Entenda mais sobre esse conceito](https://www.alura.com.br/artigos/big-data)                                                 |

| [Data Lake vs Data Warehouse](https://www.alura.com.br/artigos/data-lake-vs-data-warehouse)                                             |

| [Data Modeling](https://medium.com/sagar-explains-azure-and-analytics-data-engineerin/introduction-to-data-modelling-c0c44432ec0b)      |

| [Data Warehouse](https://www.analytics8.com/blog/what-is-a-data-warehouse/)                                                             |

| [Data Lake](https://www.youtube.com/watch?v=RQOXkuirWF0)                                                                                |

| [Data Lake](https://faun.pub/an-overview-of-data-lake-concepts-and-architectures-on-aws-and-azure-f485ed5110e2)                         |

| [ETL vs ELT](https://www.guru99.com/etl-vs-elt.html)                                                                                    |

| [Data Mesh](https://medium.com/data-hackers/data-mesh-indo-al%C3%A9m-do-data-lake-e-data-warehouse-465d57539d89)                        |

| [Batch Processing vs Stream Processing](https://gowthamy.medium.com/big-data-battle-batch-processing-vs-stream-processing-5d94600d8103) |

| [Batch Processing Example](https://www.startdataengineering.com/post/update-mysql-in-batch/)                                            |

| [Stream Processing Example](https://www.startdataengineering.com/post/data-engineering-project-for-beginners-stream-edition/)           |

### Big Data Fundamentals

- `YouTube`[Big Data Tutorial For Beginners](https://www.youtube.com/watch?v=16Z2WRtoO8Y)

- `Udemy` [The Hadoop Ecosystem](https://www.udemy.com/course/learn-big-data-the-hadoop-ecosystem-masterclass/?ranMID=39197&ranEAID=HTtUFxqit0c&ranSiteID=HTtUFxqit0c-cjIFaJzEck5Pdp99WCBpew&LSNPUBID=HTtUFxqit0c&utm_source=aff-campaign&utm_medium=udemyads)

### Data Warehouse

- `Udemy` [Data Warehouse Fundamentals for Beginners](https://www.udemy.com/course/data-warehouse-fundamentals-for-beginners/?ranMID=39197&ranEAID=HTtUFxqit0c&ranSiteID=HTtUFxqit0c-5Cssseb5O_hSY2lvbFd3bQ&LSNPUBID=HTtUFxqit0c&utm_source=aff-campaign&utm_medium=udemyads)

### Data Orchestration

- `Udemy` [The Complete Hands-On Introduction to Apache Airflow](https://www.udemy.com/course/the-complete-hands-on-course-to-master-apache-airflow/?ranMID=39197&ranEAID=HTtUFxqit0c&ranSiteID=HTtUFxqit0c-RLnxYMmqet2OlocFyF.SdQ&LSNPUBID=HTtUFxqit0c&utm_source=aff-campaign&utm_medium=udemyads)

- `Udemy` [Apache Airflow: The Hands-On Guide](https://www.udemy.com/course/the-ultimate-hands-on-course-to-master-apache-airflow/)

- `YouTube` [Supletivo DH - Apache Airflow](https://www.youtube.com/watch?v=f_lnDBR3rFU&t=468s)

### Big Data Processing

- `Udemy` [Spark and Python for Big Data with PySpark](https://www.udemy.com/course/spark-and-python-for-big-data-with-pyspark/?ranMID=39197&ranEAID=HTtUFxqit0c&ranSiteID=HTtUFxqit0c-qofM41MCAbjZoB0kUD_6vA&LSNPUBID=HTtUFxqit0c&utm_source=aff-campaign&utm_medium=udemyads)

### Cloud

- `Udemy` [AWS](https://www.udemy.com/course/aws-certified-cloud-practitioner-new/?ranMID=39197&ranEAID=HTtUFxqit0c&ranSiteID=HTtUFxqit0c-B0OpiE6Fs4IfdBOSfTrANg&utm_source=aff-campaign&utm_medium=udemyads&LSNPUBID=HTtUFxqit0c)