Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/jnyambok/olympics-2024-azure-data-engineering-pipeline

A simple data engineering pipeline on Azure that ingests data via Azure Data Factory from a data source, stores it, transforms it on via Azure Data-bricks, stores the transformed data and performs data warehousing via Azure Synapse Analytics.
https://github.com/jnyambok/olympics-2024-azure-data-engineering-pipeline

Last synced: 10 days ago
JSON representation

Host: GitHub
URL: https://github.com/jnyambok/olympics-2024-azure-data-engineering-pipeline
Owner: Jnyambok
Created: 2024-09-08T12:19:44.000Z (4 months ago)
Default Branch: main
Last Pushed: 2024-12-25T15:10:50.000Z (15 days ago)
Last Synced: 2024-12-25T16:19:41.569Z (15 days ago)
Language: Jupyter Notebook
Size: 3.48 MB
Stars: 1
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

# Building a Azure Data Engineering Pipeline: Ingesting and Transforming Olympics data with Azure

![6 Exploring ADF , creating pipeline and deployng copy data activities](https://github.com/user-attachments/assets/cd7f8769-2a78-4d67-b16e-a5cabcfa0254)
This simple data engineering pipeline on Azure ingests data via Azure Data Factory from GitHub, stores it, transforms it on via Azure Data-bricks and stores the transformed data

* Microsoft Azure Cloud Platform
* Running Locally through Flask

I wrote an article about in on Medium. [Check it out](https://medium.com/dev-genius/building-an-azure-data-engineering-pipeline-ingesting-and-transforming-olympics-data-with-azure-02606cf66dcd)

### Built With
[![My Skills](https://skillicons.dev/icons?i=python,azure,github,vscode)](https://skillicons.dev)