https://github.com/jasontanx/ridership-headline-project
This end to end data engineering / data analytics project will be about the Malaysian public transport ridership data.
https://github.com/jasontanx/ridership-headline-project
bigquery data-engineering minio-server public-transport-ridership terraform
Last synced: about 1 month ago
JSON representation
This end to end data engineering / data analytics project will be about the Malaysian public transport ridership data.
- Host: GitHub
- URL: https://github.com/jasontanx/ridership-headline-project
- Owner: jasontanx
- Created: 2023-09-29T09:10:15.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-10-26T09:26:43.000Z (over 2 years ago)
- Last Synced: 2025-02-01T02:19:53.393Z (over 1 year ago)
- Topics: bigquery, data-engineering, minio-server, public-transport-ridership, terraform
- Language: Python
- Homepage:
- Size: 23.4 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# ridership-headline-project
This end to end data engineering / data analytics project will be about the Malaysian public transport ridership data.
## Malaysia Public Transport Ridership Dashboard
This dashboard was designed via Google Looker Studio
## Starting minio
1. Install minio
2. Create and identify the access key and the secret key
3. Open terminal and key in following
```
minio server start
```
3. Copy the link provided and open the webpage
4. Key in the access key and secret key created earlier
## Files saved at MinIO bucket
## BigQuery ingested data
## Running the script
```
python3 extract_load_minio.py
```