{"id":15344007,"url":"https://github.com/tarilabs/demo20231121","last_synced_at":"2025-04-05T22:28:06.061Z","repository":{"id":208662197,"uuid":"721677660","full_name":"tarilabs/demo20231121","owner":"tarilabs","description":null,"archived":false,"fork":false,"pushed_at":"2023-12-05T09:30:35.000Z","size":5594,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-12T04:16:29.755Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tarilabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-11-21T14:50:29.000Z","updated_at":"2023-11-21T14:50:42.000Z","dependencies_parsed_at":"2023-12-02T22:39:33.846Z","dependency_job_id":null,"html_url":"https://github.com/tarilabs/demo20231121","commit_stats":null,"previous_names":["tarilabs/demo20231121"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarilabs%2Fdemo20231121","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarilabs%2Fdemo20231121/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarilabs%2Fdemo20231121/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarilabs%2Fdemo20231121/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tarilabs","download_url":"https://codeload.github.com/tarilabs/demo20231121/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247411203,"owners_count":20934650,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T10:53:35.198Z","updated_at":"2025-04-05T22:28:06.033Z","avatar_url":"https://github.com/tarilabs.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"A very quick technical demo: https://youtu.be/grXnjGtDFXg\n\n```\n0:00 Intro\n0:33 Model Registry Operator\n1:41 Notebook and Model Registry\n6:30 Data Science Pipeline and Model Registry\n10:00 Deploy ML with Model Registry to Model Serving\n14:13 One more thing... (Intelligent App using ML)\n16:40 Conclusions\n```\n\n# Demo\n\nThis is a technical demo from the Model Registry team.\n\nContent:\n- model-registry-operator\n- Notebook → Model Registry\n- DS Pipeline → Model Registry\n- Model Registry → Serving\n- One More Thing...\n- Conclusions\n\nThis post does not go into details of context and background of the Model Registry, for which we invite you to checkout [this document](https://docs.google.com/document/d/1T3KfOqIfJohp0s1koQ2XrJJQQhj7TECO-m2xPsW59_c/edit?usp=sharing) instead.\n\n## model-registry-operator\n\nWe can use the [Model Registry Operator](https://github.com/opendatahub-io/model-registry-operator) to install the Model Registry in our Kubernetes platform.\nInstalling the operator is easy, following a few steps as detailed in the README.\n\nFirst, we install a Custom Resource Definition for our Model Registry:\n\n![](/img/Screenshot%202023-12-02%20at%2020.54.21%20(2).png)\n\nThen, we deploy the `model-registry-operator` which at present sits in its own namespace:\n\n![](/img/Screenshot%202023-12-02%20at%2020.58.10%20(2).png)\n\nFinally, we create a Model Registry instance:\n\n![](/img/Screenshot%202023-12-02%20at%2021.01.57%20(2).png)\n\nWith an instance of Model Registry now available, we can move to perform some ML training using Notebook in order to later index the resulting models on this model registry instance.\n\n## Notebook → Model Registry\n\nFor the scope of this technical demo of Model Registry, we will be training and making inference with the MNIST dataset, serving as an \"hello world\" style of example for neural network.\n\nWe are given some test set to perform training, so given the following image, the training label is '3':\n\n![](img/Screenshot%202023-12-02%20at%2021.14.11%20(2).png)\n\nSo given the following image from the test set, we would like our ML model, once trained, to successfully predict '4':\n\n![](img/Screenshot%202023-12-02%20at%2021.15.26%20(2).png)\n\n### Notebook → Model Registry: First model version\n\nSo we can traing a first version of our neural network:\n\n![](img/Screenshot%202023-12-02%20at%2021.17.03%20(2).png)\n\nAnd after some quick dry-run to check the prediction seems reasonable:\n\n![](img/Screenshot%202023-12-02%20at%2021.17.32%20(2).png)\n\nWe can store the trained ML model in an S3-compatible bucket:\n\n![](img/Screenshot%202023-12-02%20at%2021.19.46%20(2).png)\n\nAnd index the model in our Model Registry:\n\n![](img/Screenshot%202023-12-02%20at%2021.19.53%20(2).png)\n\n### Notebook → Model Registry: Second model version\n\nFor the scope of this technical demo, we can train a second version of our ML model, by defining instead a convolutional neural network:\n\n![](/img/Screenshot%202023-12-02%20at%2021.22.50%20(2).png)\n\nWe can perform again some quick dry-run to check the prediction seems reasonable, store the trained model in the S3 bucket, and index the model in the Model Registry:\n\n![](img/Screenshot%202023-12-02%20at%2021.24.43%20(2).png)\n\nWe can notice in the outputs the entities created on the Model Registry.\n\n### Notebook → Model Registry: REST APIs\n\nWe don't have yet a GUI web application to display the metadata in our Model Registry, but we can perform this exercise by making use of the REST APIs from the command line.\n\nWe can display the known RegisteredModel, in this case we called our RegisteredModel `MNIST` (as we've been using the MNIST dataset):\n\n```\nMR_HOSTNAME=...\n\ncurl --silent -X 'GET' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/registered_models?pageSize=100\u0026orderBy=ID\u0026sortOrder=DESC\u0026nextPageToken=\" \\\n  -H 'accept: application/json' | jq .items\n```\n\n![](img/Screenshot%202023-12-02%20at%2021.29.58%20(2).png)\n\nWe can display the two versions we have just indexed, which are referencing as well the models as Stored in S3:\n\n```\ncurl --silent -X 'GET' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/model_versions?pageSize=100\u0026orderBy=ID\u0026sortOrder=DESC\u0026nextPageToken=\" \\\n  -H 'accept: application/json' | jq .items\n```\n\n![](img/Screenshot%202023-12-02%20at%2021.32.42%20(2).png)\n\nAnd we can display details for the ModelArtifact corresponding to the latter version:\n\n```\ncurl --silent -X 'GET' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/model_versions/4/artifacts\" \\\n  -H 'accept: application/json' | jq .items\n```\n\n![](img/Screenshot%202023-12-02%20at%2021.33.59%20(2).png)\n\nWe can move now to DSP.\n\n## DS Pipeline → Model Registry\n\nWe can create a Data Science Pipeline which performs analogous step in training, validating, storing and indexing the ML model:\n\n![](img/Screenshot%202023-12-02%20at%2021.40.37%20(2).png)\n\nWe can create a DSP Run, possibly using specific parameters which influence the model training and validation; since the trained model matches the requirements defined in the pipeline, this run successfully completed, the model is stored once again on a S3-compatible bucket and the metadata are indexed on the Model Registry. We have now 3 version of our ML model:\n\n![](img/Screenshot%202023-12-02%20at%2021.52.50%20(2).png)\n\nWe have now a bunch of of ModelVersion(s) available on the Model Registry ready to be deployed for inference on Model Serving.\n\n## Model Registry → Serving\n\nWe can use the following REST API to create an entity for InferenceService on the Model Registry:\n\n```\ncurl -X 'POST' \\\n  $MR_HOSTNAME'/api/model_registry/v1alpha1/serving_environments/1/inference_services' \\\n  -H 'accept: application/json' \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n  \"name\": \"mnist-e2e\",\n  \"modelVersionId\": \"4\",\n  \"runtime\": \"mmserver1\",\n  \"registeredModelId\": \"2\",\n  \"servingEnvironmentId\": \"1\",\n  \"state\": \"DEPLOYED\"\n}'\n```\n\n![](img/Screenshot%202023-12-02%20at%2022.05.03%20(2).png)\n\nIt's important to note the Model Registry is *not* an active orchestrator; what actually happens behind the scene is that the K8s Model Controller performs a reconciliation loop and based on the metadata we just created on the Model Registry, create a CR for ModelMesh in order to deploy our ML for Inference:\n\n![](img/Screenshot%202023-12-02%20at%2022.09.01%20(2).png)\n\nWe can as well undeploy the InferenceService by changing flipping the status on the Model Registry; in this case the same reconcilation loop will trigger again, but will proceed accordingly to remove the ModelMesh CR for Serving:\n\n```\ncurl -X 'PATCH' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/inference_services/6\" \\\n  -H 'accept: application/json' \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n  \"modelVersionId\": \"4\",\n  \"runtime\": \"mmserver1\",\n  \"state\": \"UNDEPLOYED\"\n}'\n```\n\n![](img/Screenshot%202023-12-02%20at%2022.13.00%20(2).png)\n\n## One More Thing...\n\nTo wrap-up this technical exercise, we can re-deploy once more our trained ML model and noting the inference service endpoint:\n\n![](img/Screenshot%202023-12-02%20at%2022.16.26%20(2).png)\n\nAs we can use it to deploy some Intelligent Application on Kubernetes:\n\n![](img/Screenshot%202023-12-02%20at%2022.21.41%20(2).png)\n\nWhich in this case is a (very raw cut!) PoC application exercising our ML inference endpoint on ModelMesh for Model Serving; as we draw by hand:\n\n![](img/Screenshot%202023-12-02%20at%2022.23.00%20(2).png)\n\nWe will get back an array of probabilities; since index 2 is the most probable, our intelligent application is predicting we have drawn the `number: 2` thanks to the model we just trained, indexed and deployed, all while making use of Model Registry!\n\n## Conclusions\n\nIn this short technical demo we have explored some of the capabilites of the Model Registry by making an end-to-end demo.\n\nWe have installed the model-registry-operator in order to deploy our instance of Model Registry.\n\nWe have performed a couple of training of ML neural networks, stored them in an S3 bucket and indexed those model in the Model Registry. We have also performed analogously from the perspective of Data Science Pipelines, which may include parametrization, model validation, etc.\n\nWe have seen how Model Registry is *not* an active orchestrator but a metadata repository; following established Kubernetes best-practices a reconciliation loop from the Model Controller make sure to enact the correct CustomResource deployment and update in the cluster. This mechanism is used to govern deployment of ML models which the Model Registry have previously maintained index, per the previous steps.\n\nFinally, we have seen how to leverage the Inference endpoint of the deployed ML model to drive the end-to-end workload of intelligent application on top of Kubernetes!\n\nIf you found this content interesting, don't hesitate to join the conversation:\n- https://github.com/kubeflow/kubeflow/issues/7396 \n- https://www.kubeflow.org/docs/about/community/#kubeflow-community-call\n\n# Notes\n\ndocker buildx build --platform linux/amd64 -t quay.io/mmortari/rdsp:latest --push .\n\n## Noteboook notes\n\nif using 2023.2 TF, need: `!pip install \"onnx==1.14.1\"`\n\n## Running notes\n\nMR_HOSTNAME=http://manual-route-ds-mre2e.apps.mmortari.dev.datahub.redhat.com\n\ncurl --silent -X 'GET' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/registered_models?pageSize=100\u0026orderBy=ID\u0026sortOrder=DESC\u0026nextPageToken=\" \\\n  -H 'accept: application/json' | jq .items\n\ncurl --silent -X 'GET' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/model_versions?pageSize=100\u0026orderBy=ID\u0026sortOrder=DESC\u0026nextPageToken=\" \\\n  -H 'accept: application/json' | jq .items\n\ncurl --silent -X 'GET' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/model_versions/4/artifacts\" \\\n  -H 'accept: application/json' | jq .items\n\ncurl --silent -X 'GET' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/serving_environments?pageSize=100\u0026orderBy=ID\u0026sortOrder=DESC\u0026nextPageToken=\" \\\n  -H 'accept: application/json' | jq .items\n\ncurl -X 'POST' \\\n  $MR_HOSTNAME'/api/model_registry/v1alpha1/serving_environments/1/inference_services' \\\n  -H 'accept: application/json' \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n  \"name\": \"mnist-e2e\",\n  \"modelVersionId\": \"4\",\n  \"runtime\": \"mmserver1\",\n  \"registeredModelId\": \"2\",\n  \"servingEnvironmentId\": \"1\",\n  \"state\": \"DEPLOYED\"\n}'\n\ncurl -X 'PATCH' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/inference_services/6\" \\\n  -H 'accept: application/json' \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n  \"modelVersionId\": \"4\",\n  \"runtime\": \"mmserver1\",\n  \"state\": \"UNDEPLOYED\"\n}'\n\ncurl -X 'PATCH' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/inference_services/6\" \\\n  -H 'accept: application/json' \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n  \"modelVersionId\": \"4\",\n  \"runtime\": \"mmserver1\",\n  \"state\": \"DEPLOYED\"\n}'\n\ncurl --silent -X 'GET' \\\n  \"$MR_HOSTNAME/api/model_registry/v1alpha1/inference_services/5/serves?pageSize=100\u0026orderBy=ID\u0026sortOrder=DESC\u0026nextPageToken=\" \\\n  -H 'accept: application/json' | jq\n\n# See also\n\nhttps://github.com/tarilabs/demo20231121-web\n\n# References\n\n- https://developers.redhat.com/learning/learn:openshift-data-science:how-create-tensorflow-model/resource/resources:build-train-and-run-your-tensorflow-model\n- https://adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks/\n- https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/\n- https://www.youtube.com/watch?v=bNb2fEVKeEo\u0026list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv\u0026index=5\n  - https://cs231n.github.io/convolutional-networks/#overview\n- https://ai.stackexchange.com/questions/5546/what-is-the-difference-between-a-convolutional-neural-network-and-a-regular-neur","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftarilabs%2Fdemo20231121","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftarilabs%2Fdemo20231121","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftarilabs%2Fdemo20231121/lists"}