Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/alterra-greeve/de-capstone
Capstone Project SIB Batch 6 x Alterra Academy - Data Engineer
https://github.com/alterra-greeve/de-capstone
bigquery cloud-function data-engineering docker googlefirebase looker-studio python
Last synced: 2 months ago
JSON representation
Capstone Project SIB Batch 6 x Alterra Academy - Data Engineer
- Host: GitHub
- URL: https://github.com/alterra-greeve/de-capstone
- Owner: Alterra-Greeve
- Created: 2024-06-07T07:31:48.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2024-06-24T06:39:51.000Z (7 months ago)
- Last Synced: 2024-11-21T04:02:50.444Z (2 months ago)
- Topics: bigquery, cloud-function, data-engineering, docker, googlefirebase, looker-studio, python
- Language: Python
- Homepage: https://greeve.store/
- Size: 1.28 MB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Greeve
Final Project Capstone Alterra Academy
Explore the Greeve Visualization Dashboard
# About Project
Greeve is an innovative app that aims to increase user awareness and participation in environmental conservation and make it easier to purchase eco-friendly goods and equipment. The app not only offers a platform to purchase environmental goods and equipment, but also provides information on the impact of users' activities on the environment as well as how to measure and reduce that impact. The following is greeve documentation from the data engineer side.
# Our Data Engineer Team
| Name |
| --------------------------- |
| Adhira Riyanti Amanda |
| Muhammad Dzikri Rizaldi |
| Zahra Putri Zanuarti |
# High Level Architecture
- **Data Collection**: We gather data from various sources, such as application logs using `Cloud Logging` and `MySQL` databases. This data includes user activities, transactions, challenges, and other relevant information.
- **ETL Process**: Using `Apache Airflow`, we schedule and manage the process of extracting data, transforming it into a usable format with `Python` scripts, and then loading it into `Firebase` for storage. This process ensures the data is clean and structured properly.
- **Automation and Storage**: We automate the transfer of data from `Firebase` to `Google BigQuery` tables using `Google Cloud Functions`.
- **Data Analysis**: In `BigQuery`, we perform detailed analysis to understand patterns and trends in user behavior, such as how users choose eco-friendly products and participate in challenges.
- **Data Visualization**: We use `Looker Studio` to create interactive dashboards. These visualizations help stakeholders easily understand and make decisions based on the data, such as assessing the effectiveness of environmental initiatives.
# Star Schema
We use this star schema design in Big Query tables to simplify queries and improve their performance
# Visualization Dashboard
### [Looker Studio Dashboard](https://intip.in/GreeveDEVisualization)
