{"id":20582974,"url":"https://github.com/vedanthv/data-engineering-portfolio","last_synced_at":"2025-04-14T20:16:27.924Z","repository":{"id":196886984,"uuid":"697387369","full_name":"vedanthv/data-engineering-portfolio","owner":"vedanthv","description":"Cool DE Projects","archived":false,"fork":false,"pushed_at":"2025-03-01T17:50:58.000Z","size":94241,"stargazers_count":25,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-14T20:16:15.716Z","etag":null,"topics":["aws-projects","data","data-engineering","data-modelling","databricks","portfolio-project","python","sql"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vedanthv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-27T16:15:06.000Z","updated_at":"2025-03-04T10:24:30.000Z","dependencies_parsed_at":"2023-10-17T02:01:01.233Z","dependency_job_id":"b244e90d-7bb1-4907-bf95-5480937af109","html_url":"https://github.com/vedanthv/data-engineering-portfolio","commit_stats":null,"previous_names":["vedanthv/data-engineering-projects","vedanthv/data-engineering-portfolio"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vedanthv%2Fdata-engineering-portfolio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vedanthv%2Fdata-engineering-portfolio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vedanthv%2Fdata-engineering-portfolio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vedanthv%2Fdata-engineering-portfolio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vedanthv","download_url":"https://codeload.github.com/vedanthv/data-engineering-portfolio/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248952357,"owners_count":21188427,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-projects","data","data-engineering","data-modelling","databricks","portfolio-project","python","sql"],"created_at":"2024-11-16T06:37:37.104Z","updated_at":"2025-04-14T20:16:27.918Z","avatar_url":"https://github.com/vedanthv.png","language":"Jupyter Notebook","readme":" ![image](https://github.com/vedanthv/data-engineering-portfolio/assets/44313631/664c2886-b8d7-41cd-b231-f9f1ca4bbd3e)\n\nHello World! I'm Vedanth. \n\nThis is a complete portfolio of the projects I have designed with a major focus on implementing various data engineering tech and cloud services across Azure, AWS and GCP.\n\nFeel Free to Connect with me 🤠\n\n**[LinkedIn](https://www.linkedin.com/in/vedanthbaliga/) | [GitHub](https://github.com/vedanthv/)**\n\n## Infrastructure and Tech Stack\n\n![DE Portfolio Tool Used](https://github.com/user-attachments/assets/62fc0dd1-b612-439d-947e-96787e711dd1)\n\n## Quick Links\n\nHere is an index of projects with the tech, domain and link.\n\n### Projects\n\n| Project | Domain | \n| :---------: | :---------: |\n|[Pt 1 : Streaming ETL with Airflow Orchestrator and Postgres DB](https://github.com/vedanthv/data-engineering-portfolio/tree/main/cricket-livescores-ingestion-kafka-airflow)|Near Realtime Streaming,SQL Database, Kafka, Cricket!|\n|[Pt 2 : Serving Postgres Data using FastAPI](https://github.com/vedanthv/data-engineering-portfolio/tree/main/cricket-livescores-fastapi)|Backend API Development, Streaming Data|\n|[Pt 3 : Frontend Web App Powered by Streaming Data with Streamlit](https://github.com/vedanthv/data-engineering-portfolio/tree/main/cricket-analytics-frontend-app-streamlit)|Frontend Application Development,MVC Architecture|\n|[Pt 4 : Realtime Ingestion and Transformation with Apache Druid as OLAP Database](https://github.com/vedanthv/data-engineering-portfolio/tree/main/real-time-analytics-druid) | Data Ingestion and Big Data Processing |\n|[InvestIQ Metrics](https://github.com/vedanthv/data-engineering-portfolio/tree/main/investiq-metrics)|AWS EC2,Docker,Airflow,ReapidAPI,AWS Lambda,AWS S3,AWS CloudWatch,AWS Redshift,Prometheus,Grafana,PowerBI|\n|[User Mingle : Kafka Driven User Profile Streaming](https://github.com/vedanthv/data-engineering-portfolio/tree/main/user-mingle)|Airflow, Zookeeper, Cassandra, Schema Registry, Spark|  \n|[Grand Prix Data Odyssey](https://github.com/vedanthv/data-engineering-projects/tree/main/formula-1-analytics-engg)|Azure Databricks,Spark SQL,Postman,Blob Storage,Unity Catalog,ADF,Azure Devops,Synapse Studio, Delta Lake,PowerBI | \n|[Medal Metrics: Tokyo Olympics Data Alchemy](https://github.com/vedanthv/data-engineering-projects/tree/main/tokyo-olympics-de)|ADF,Azure Data Lake Gen2,Blob Storage,Databricks,Synapse Analytics,PowerBI|\n|[Airflow with Postgres as OLTP Database [Batch Processing]](https://github.com/vedanthv/data-engineering-portfolio/tree/main/airflow-postgres-db)|Beautiful Soup, Apache Airflow, Postgres, Docker|\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvedanthv%2Fdata-engineering-portfolio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvedanthv%2Fdata-engineering-portfolio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvedanthv%2Fdata-engineering-portfolio/lists"}