https://github.com/vedanthv/data-engineering-portfolio
Cool DE Projects
https://github.com/vedanthv/data-engineering-portfolio
aws-projects data data-engineering data-modelling databricks portfolio-project python sql
Last synced: 6 months ago
JSON representation
Cool DE Projects
- Host: GitHub
- URL: https://github.com/vedanthv/data-engineering-portfolio
- Owner: vedanthv
- Created: 2023-09-27T16:15:06.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2025-03-01T17:50:58.000Z (7 months ago)
- Last Synced: 2025-04-14T20:16:15.716Z (6 months ago)
- Topics: aws-projects, data, data-engineering, data-modelling, databricks, portfolio-project, python, sql
- Language: Jupyter Notebook
- Homepage:
- Size: 89.9 MB
- Stars: 25
- Watchers: 1
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Hello World! I'm Vedanth.
This is a complete portfolio of the projects I have designed with a major focus on implementing various data engineering tech and cloud services across Azure, AWS and GCP.
Feel Free to Connect with me ðŸ¤
**[LinkedIn](https://www.linkedin.com/in/vedanthbaliga/) | [GitHub](https://github.com/vedanthv/)**
## Infrastructure and Tech Stack
## Quick Links
Here is an index of projects with the tech, domain and link.
### Projects
| Project | Domain |
| :---------: | :---------: |
|[Pt 1 : Streaming ETL with Airflow Orchestrator and Postgres DB](https://github.com/vedanthv/data-engineering-portfolio/tree/main/cricket-livescores-ingestion-kafka-airflow)|Near Realtime Streaming,SQL Database, Kafka, Cricket!|
|[Pt 2 : Serving Postgres Data using FastAPI](https://github.com/vedanthv/data-engineering-portfolio/tree/main/cricket-livescores-fastapi)|Backend API Development, Streaming Data|
|[Pt 3 : Frontend Web App Powered by Streaming Data with Streamlit](https://github.com/vedanthv/data-engineering-portfolio/tree/main/cricket-analytics-frontend-app-streamlit)|Frontend Application Development,MVC Architecture|
|[Pt 4 : Realtime Ingestion and Transformation with Apache Druid as OLAP Database](https://github.com/vedanthv/data-engineering-portfolio/tree/main/real-time-analytics-druid) | Data Ingestion and Big Data Processing |
|[InvestIQ Metrics](https://github.com/vedanthv/data-engineering-portfolio/tree/main/investiq-metrics)|AWS EC2,Docker,Airflow,ReapidAPI,AWS Lambda,AWS S3,AWS CloudWatch,AWS Redshift,Prometheus,Grafana,PowerBI|
|[User Mingle : Kafka Driven User Profile Streaming](https://github.com/vedanthv/data-engineering-portfolio/tree/main/user-mingle)|Airflow, Zookeeper, Cassandra, Schema Registry, Spark|
|[Grand Prix Data Odyssey](https://github.com/vedanthv/data-engineering-projects/tree/main/formula-1-analytics-engg)|Azure Databricks,Spark SQL,Postman,Blob Storage,Unity Catalog,ADF,Azure Devops,Synapse Studio, Delta Lake,PowerBI |
|[Medal Metrics: Tokyo Olympics Data Alchemy](https://github.com/vedanthv/data-engineering-projects/tree/main/tokyo-olympics-de)|ADF,Azure Data Lake Gen2,Blob Storage,Databricks,Synapse Analytics,PowerBI|
|[Airflow with Postgres as OLTP Database [Batch Processing]](https://github.com/vedanthv/data-engineering-portfolio/tree/main/airflow-postgres-db)|Beautiful Soup, Apache Airflow, Postgres, Docker|