https://github.com/jasontanx/ridership-headline-project

This end to end data engineering / data analytics project will be about the Malaysian public transport ridership data.
https://github.com/jasontanx/ridership-headline-project

bigquery data-engineering minio-server public-transport-ridership terraform

Last synced: 29 days ago
JSON representation

This end to end data engineering / data analytics project will be about the Malaysian public transport ridership data.

• Host: GitHub
• URL: https://github.com/jasontanx/ridership-headline-project
• Owner: jasontanx
• Created: 2023-09-29T09:10:15.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-10-26T09:26:43.000Z (about 1 year ago)
• Last Synced: 2024-04-21T13:30:13.924Z (9 months ago)
• Topics: bigquery, data-engineering, minio-server, public-transport-ridership, terraform
• Language: Python
• Homepage:
• Size: 23.4 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# ridership-headline-project

This end to end data engineering / data analytics project will be about the Malaysian public transport ridership data. 

![test drawio](https://github.com/jasontanx/ridership-headline-project/assets/116934441/e2cd3f9f-ec10-4586-9a5d-54d89218d260)

## Malaysia Public Transport Ridership Dashboard

This dashboard was designed via Google Looker Studio



## Starting minio

1. Install minio

2. Create and identify the access key and the secret key

3. Open terminal and key in following

```

minio server start

```

3. Copy the link provided and open the webpage

4. Key in the access key and secret key created earlier

## Files saved at MinIO bucket



## BigQuery ingested data



## Running the script

```

python3 extract_load_minio.py

```