Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sayakpaul/dual-deployments-on-vertex-ai
Project demonstrating dual model deployment scenarios using Vertex AI (GCP).
https://github.com/sayakpaul/dual-deployments-on-vertex-ai
automl gcp keras kubeflow mlops tensorflow tfx vertex-ai
Last synced: 14 days ago
JSON representation
Project demonstrating dual model deployment scenarios using Vertex AI (GCP).
- Host: GitHub
- URL: https://github.com/sayakpaul/dual-deployments-on-vertex-ai
- Owner: sayakpaul
- License: apache-2.0
- Created: 2021-07-29T07:47:51.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2021-12-28T09:04:31.000Z (almost 3 years ago)
- Last Synced: 2024-10-03T12:38:36.958Z (about 1 month ago)
- Topics: automl, gcp, keras, kubeflow, mlops, tensorflow, tfx, vertex-ai
- Language: Jupyter Notebook
- Homepage: https://cloud.google.com/blog/topics/developers-practitioners/dual-deployments-vertex-ai
- Size: 1.84 MB
- Stars: 35
- Watchers: 6
- Forks: 8
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Dual-Deployments-on-Vertex-AI
_By [Chansung Park](https://github.com/deep-diver) and Sayak Paul_
This project demonstrates a workflow to cover dual model deployment scenarios using [Kubeflow](https://www.kubeflow.org/),
[TensorFlow Extended (TFX)](https://www.tensorflow.org/tfx), and [Vertex AI](https://cloud.google.com/vertex-ai). We suggest
reading the accompanying [blog post](https://cloud.google.com/blog/topics/developers-practitioners/dual-deployments-vertex-ai) first
to get an idea and then following along with the code. This project also received the [#TFCommunitySpotlight Award](https://twitter.com/TensorFlow/status/1446611368078086144?s=20).
## Motivation 💻
Let's say you want to allow your users to run an application both in online and offline mode. Your mobile
application would use a TFLite model depending on the network bandwidth/battery etc., and if sufficient
network coverage/internet bandwidth is available your application would instead use the online cloud one. This way
your application stays resilient and can ensure high availability.
Sometimes we also do layered predictions where we first divide a problem into smaller tasks:
1) predict if it's a yes/no,
2) depending on the output of 1) we run the final model.
In these cases, 1) takes place on-device and 2) takes place on the cloud to ensure a smooth UX. Furthermore, it's
a good practice to use a mobile-friendly network architecture (such as MobileNet) when considering
mobile deployments. This leads us to the following question:
_**Can we train two different models within the same deployment pipeline and manage them seamlessly?**_
This project is motivated by this question.
## AutoML, TFX, etc. ðŸ›
Different organizations have people with varied technical backgrounds. We wanted to provide the easiest solution first
and then move on to something that is more customizable. To this end, we leverage [Kubeflow's AutoML SDKs](https://github.com/kubeflow/pipelines/tree/master/components/google-cloud) to build, train, and deploy models with
different production use-cases. With AutoML, the developers can delegate a large part of their workflows to the SDKs
and the codebase also stays comparatively smaller. The figure below depicts a sample system architecture for
this scenario:
**Figure developed by Chansung Park.**
But the story does not end here. What if we wanted to have better control over the models to be built, trained,
and deployed? Enter TFX! TFX provides the flexibility of writing custom components and including them inside a
pipeline. This way Machine Learning Engineers can focus on building and training their favorite models and delegate
a part of the heavy lifting to TFX and Vertex AI. On Vertex AI (acting as an orchestrator) this pipeline will look like
so:
```txt
🔥 In this project we cover both these situations.
```
## Code 🆘
Our code is distributed as Colab Notebooks. But one needs to have a billing-enabled GCP account
(with a few APIs enabled) to successfully run these notebooks. Alternatively one can also use the
notebooks on [Vertex AI Notebooks](https://cloud.google.com/vertex-ai/docs/general/notebooks). Find
all the notebooks and their descriptions here:
[`notebooks`](https://github.com/sayakpaul/Dual-Deployments-on-Vertex-AI/tree/main/notebooks).
Additionally, you can find the custom TFX components separately here - [`custom_components`](https://github.com/sayakpaul/Dual-Deployments-on-Vertex-AI/tree/main/custom_components).
## Acknowledgements
[ML-GDE program](https://developers.google.com/programs/experts/) for providing GCP credits. Thanks to [Karl Weinmeister](https://twitter.com/kweinmeister?lang=hr) and [Robert Crowe](https://twitter.com/robert_crowe?lang=en) for providing review feedback on this project.