Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
Projects in Awesome Lists by janaom
A curated list of projects in awesome lists by janaom .
https://github.com/janaom/kodekloud-engineer-2.0
Solutions for the KodeKloud Engineer 2.0 tasks.
ansible docker git jenkins kodekloud-solutions kubernetes linux
Last synced: 19 Nov 2024
https://github.com/janaom/gcp-de-project-uber-etl-pipeline
Technologies used: GCS, Compute Engine, Mage, BigQuery, Looker, Python
Last synced: 19 Nov 2024
https://github.com/janaom/gcp-data-engineering-etl-with-composer-dataflow
This project leverages GCS, Composer, Dataflow, BigQuery, and Looker on Google Cloud Platform (GCP) to build a robust data engineering solution for processing, storing, and reporting daily transaction data in the online food delivery industry.
airflow apache-beam cloud-composer cloud-storage data-engineering dataflow de-project gcp gcs looker
Last synced: 19 Nov 2024
https://github.com/janaom/gcp-de-project-connect-four-with-python-dataflow
Connect Four Data Engineering Project: leveraging GCS for scalable and durable storage, Dataflow for data extraction and transformation, BigQuery as the data repository, Slack Integration for real-time sharing, Looker for insightful reports and visualizations, and Email Scheduler for automated report delivery.
apache-beam data-engineering dataflow etl gcp python slack-integration
Last synced: 19 Nov 2024
https://github.com/janaom/gcp-de-project-weather-forecast-sms-with-airflow
This project was born out of the need to know the weather forecast for Paris/Vilnius while attending the KubeCon Europe conference. Fetches next-day forecast for Paris and Vilnius using a weather API, securely stores data in GCS bucket, and sends personalized SMS updates via Twilio. Powered by GCP and automated with Composer/Airflow
airflow composer2 data-engineering gcp twilio-sms twilio-whatsapp
Last synced: 19 Nov 2024
https://github.com/janaom/gcp-de-project-streaming-pubsub-beam-dataflow
This project demonstrates an end-to-end solution for processing and analyzing real-time conversations data from a JSON file using GCP services and infrastructure automation, showcasing data storage, streaming, processing, and analysis at scale.
apache-beam bigquery dataflow de-project gcp pubsub streaming-data
Last synced: 19 Nov 2024
https://github.com/janaom/gcp-notes
A comprehensive RP of notes, tutorials, and practical tasks to facilitate learning and mastery of Google Cloud Platform (GCP)
Last synced: 19 Nov 2024
https://github.com/janaom/azure_cognitive_services_ex
Examples of Azure Cognitive Services
Last synced: 19 Nov 2024
https://github.com/janaom/meta-database-engineer-professional-certificate
Labs from Meta Database Engineer Professional Certificate on Coursera https://www.coursera.org/professional-certificates/meta-database-engineer
Last synced: 19 Nov 2024
https://github.com/janaom/automating-real-world-tasks-with-python
Solutions for the final course "Automating Real-World Problems with Python" from the course Google IT Automation with Python Professional Certificate
Last synced: 19 Nov 2024
https://github.com/janaom/terraform_ansible_aws
terraform | ansible code for aws task
Last synced: 19 Nov 2024
https://github.com/janaom/gcp-bigquery-project-exploring-londons-travel-network
Use BigQuery to build a project. Use SQL to analyze a database containing information about Transport for London journeys over 12 years
Last synced: 19 Nov 2024
https://github.com/janaom/gcp-de-project-data-pipeline-with-cloud-run-functions-airflow-biggueryml
Build a data pipeline on Google Cloud using an event-driven architecture, leveraging GCS, Cloud Run functions, and BigQuery. Explore both VM and Composer options for Airflow management, and utilize Logging & Monitoring for pipeline health. Discover how SQL-based BigQuery ML can be used for initial ML implementation in specific scenarios.
airflow bigquery bigqueryml cloud-functions cloud-run-functions composer data-engineering-project google-cloud-platform
Last synced: 27 Nov 2024
https://github.com/janaom/introduction-to-pyspark
This repository serves as a comprehensive guide to PySpark, featuring theory and exercises sourced from DataCamp. It is designed for beginners looking to understand the fundamentals of PySpark and its applications in big data processing.
Last synced: 19 Nov 2024
https://github.com/janaom/gcp-associate-data-practitioner-exam-prep-guide
This repository contains my personal notes from the "Introduction to Data Engineering on Google Cloud" course, part of the Associate Data Practitioner Learning Path.
associate-data-practitioner google-cloud-exam
Last synced: 25 Dec 2024