{"id":15664236,"url":"https://github.com/deep-diver/mlops-hf-tf-vision-models","last_synced_at":"2026-03-07T22:07:57.143Z","repository":{"id":60312974,"uuid":"541207183","full_name":"deep-diver/mlops-hf-tf-vision-models","owner":"deep-diver","description":"MLOps for Vision Models (TensorFlow) from 🤗 Transformers with TensorFlow Extended (TFX)","archived":false,"fork":false,"pushed_at":"2022-10-05T04:17:12.000Z","size":165,"stargazers_count":115,"open_issues_count":0,"forks_count":20,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-08-16T15:36:34.490Z","etag":null,"topics":["computer-vision","huggingface-transformers","mlops","tensorflow","tensorflow-extended"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deep-diver.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-09-25T14:57:13.000Z","updated_at":"2025-05-27T05:12:34.000Z","dependencies_parsed_at":"2022-09-27T23:50:12.271Z","dependency_job_id":null,"html_url":"https://github.com/deep-diver/mlops-hf-tf-vision-models","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/deep-diver/mlops-hf-tf-vision-models","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deep-diver%2Fmlops-hf-tf-vision-models","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deep-diver%2Fmlops-hf-tf-vision-models/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deep-diver%2Fmlops-hf-tf-vision-models/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deep-diver%2Fmlops-hf-tf-vision-models/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deep-diver","download_url":"https://codeload.github.com/deep-diver/mlops-hf-tf-vision-models/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deep-diver%2Fmlops-hf-tf-vision-models/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30233432,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-07T19:01:10.287Z","status":"ssl_error","status_checked_at":"2026-03-07T18:59:58.103Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","huggingface-transformers","mlops","tensorflow","tensorflow-extended"],"created_at":"2024-10-03T13:41:46.391Z","updated_at":"2026-03-07T22:07:57.107Z","avatar_url":"https://github.com/deep-diver.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Python](https://img.shields.io/badge/python-3.9-blue.svg) [![TFX](https://img.shields.io/badge/TFX-1.9.1-orange)](https://www.tensorflow.org/tfx) [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/chansung/vit-e2e-pipeline-hf-integration)\n\n# MLOps for Vision Models (TensorFlow) from 🤗 Transformers with TensorFlow Extended (TFX)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg height=\"600px\" src=\"https://i.ibb.co/Smw1KKy/Screen-Shot-2022-10-05-at-1-15-57-PM.png\"/\u003e\n\u003c/p\u003e\n\nThis repository shows how to build Machine Learning pipeline for a vision model (TensorFlow) from 🤗 Transformers using the TensorFlow Ecosystem. In particular, we use TensorFlow Extended(TFX), and there are TensorFlow Data Validation(TFDV), Transform(TFT), Model Analysis(TFMA), and Serving(TF Serving) besides TensorFlow itself internally involved.\n\nNOTE: This is a follow-up projects of \"[Deploying Vision Models (TensorFlow) from 🤗 Transformers](https://github.com/sayakpaul/deploy-hf-tf-vision-models)\" which shows how to deploy ViT model locally, on kubernetes, and on a fully managed service Vertex AI.\n\nWe will show how to build ML pipeline with TFX in a step-by-step manner:\n- [X] **Basic** ( [![CI / Basic](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/ci-basic.yml/badge.svg)](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/ci-basic.yml) [![CD / Basic](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-basic.yml/badge.svg)](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-basic.yml) )\n  - as the first step, we show how to build ML pipeline with the most basic components, which are `ExampleGen`, `Trainer`, and `Pusher`. These components are responsible for injecting raw dataset into the ML pipeline, training a TensorFlow model, and deploying a trained model.\n\n  \u003cp align=\"center\"\u003e\n    \u003cimg width=\"700px\" src=\"https://i.ibb.co/0qr0fdR/Screen-Shot-2022-10-05-at-12-29-54-PM.png\"/\u003e\n  \u003c/p\u003e\n\n- [X] **Intermediate** ( [![CI / Intermediate](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/ci-intermediate.yml/badge.svg)](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/ci-intermediate.yml) [![CD / Intermediate](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-intermediate.yml/badge.svg)](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-intermediate.yml) )\n  - as the second step, we show how to extend the ML pipeline from the first step by adding more components, which are `SchemaGen`, `StatisticsGen`, and `Transform`. These components are responsible for analyzing the structures of the dataset, analyzing the statistical traits of the features in the dataset, and data pre-processing.\n  \n  \u003cp align=\"center\"\u003e\n    \u003cimg width=\"550px\" src=\"https://i.ibb.co/Wg83mNy/Screen-Shot-2022-10-05-at-12-34-39-PM.png\"/\u003e\n  \u003c/p\u003e\n\n- [X] **Advanced Part 1** ( [![CI / Advanced Part1](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/ci-advanced-1.yml/badge.svg)](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/ci-advanced-1.yml) [![CD / Advanced Part1](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-advanced-1.yml/badge.svg)](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-advanced-1.yml) )\n  - as the third step, we show how to extend the ML pipeline from the second step by adding more components, which are `Resolver` and `Evaluator`. These components are responsible for importing existing Artifacts (such as previously trained model) and comparing the performance between two models (one from the `Resolver` and one from the current pipeline run).\n\n  \u003cp align=\"center\"\u003e\n    \u003cimg height=\"500px\" src=\"https://i.ibb.co/4FQMqXG/Screen-Shot-2022-10-05-at-1-10-51-PM.png\"/\u003e\n  \u003c/p\u003e\n\n- [X] **Advanced Part 2** ( [![CI / Advanced Part2](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/ci-advanced-2.yml/badge.svg)](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/ci-advanced-2.yml) [![CD / Advanced Part2](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-advanced-2.yml/badge.svg)](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-advanced-2.yml) )\n  - as the fourth step, we show how to extend the ML pipeline from the third step by adding one more additional component, `Tuner`. This component is responsible for running a set of experiments with different sets of hyperparameters with fewer epochs, and the found best hyperparameter combination will be passed to the `Trainer`, and `Trainer` will train the model longer time with that hyperparameter combinations as the starting point.\n\n  \u003cp align=\"center\"\u003e\n    \u003cimg height=\"500px\" src=\"https://i.ibb.co/tsss0Ng/Screen-Shot-2022-10-05-at-1-11-27-PM.png\"/\u003e\n  \u003c/p\u003e\n\n- [X] **🤗 Hub Integration** ( [![CD / HF-Integration](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-hf-integration.yml/badge.svg)](https://github.com/deep-diver/mlops-hf-tf-vision-models/actions/workflows/cd-hf-integration.yml) )\n  - in this optional step, we show how to use custom TFX components for 🤗 Hub. In particular, we use `HFModelPusher` to push currently trained model to 🤗 Model Hub and `HFSpacePusher` to automatically deploy Gradio application to 🤗 Space Hub.\n  \n  \u003cp align=\"center\"\u003e\n    \u003cimg height=\"500px\" src=\"https://i.ibb.co/G7VKW9Y/Screen-Shot-2022-10-05-at-1-16-04-PM.png\"/\u003e\n  \u003c/p\u003e \n\n## Acknowledgements\n\nWe are thankful to the ML Developer Programs team at Google that provided GCP support.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeep-diver%2Fmlops-hf-tf-vision-models","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeep-diver%2Fmlops-hf-tf-vision-models","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeep-diver%2Fmlops-hf-tf-vision-models/lists"}