https://github.com/ket0825/v1-gcp-preview

Manage GCP src for preview services / Preview 서비스를 위한 GCP 레포
https://github.com/ket0825/v1-gcp-preview

bigquery cloud-functions cloud-run cloudbuild gcp gcp-batch gcs logging pubsub

Last synced: 5 months ago
JSON representation

Manage GCP src for preview services / Preview 서비스를 위한 GCP 레포

• Host: GitHub
• URL: https://github.com/ket0825/v1-gcp-preview
• Owner: ket0825
• License: bsd-3-clause
• Created: 2024-07-01T08:55:21.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2025-02-04T17:34:55.000Z (over 1 year ago)
• Last Synced: 2025-07-11T06:02:10.599Z (12 months ago)
• Topics: bigquery, cloud-functions, cloud-run, cloudbuild, gcp, gcp-batch, gcs, logging, pubsub
• Language: Python
• Homepage:
• Size: 215 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# v1-gcp-preview

Contains GCP and deploy codes for Preview: Product Review and OCR analyzer for customer.

*Read this in other languages: [한국어](docs/README_ko.md)*

## Whole service architecture

![image](https://github.com/user-attachments/assets/2bdeee34-ff59-4368-a9d6-e4e6935f44d3)

### Slurm Cluster ML pipeline

![ml_pipeline drawio](https://github.com/user-attachments/assets/f9c22f23-c661-4753-9938-761c47ca5cc9)

## Cloud Architecture (final)

~~Cloud Functions:~~

~~Pub/Sub:~~

~~Cloud Run:~~

### GCP Batch:

- Job as Service

- Find regions which remain extra GPU and can be provisioning as SPOT type and serving models with batch(model - https://github.com/9unu/Preview_model).

- Considered costs and efficiency such as Network cost, SPOT provisioning, etc.

- Deploy with python script "deploy.py"

### Cloud Build:

- Deploy whole architecture except VM instances

- IaC(Infrastructure as a Code)

- consists of yaml file and dockerfile