{"id":29922050,"url":"https://github.com/valohai/tensorflow-example","last_synced_at":"2025-10-20T02:34:39.689Z","repository":{"id":21393851,"uuid":"85334635","full_name":"valohai/tensorflow-example","owner":"valohai","description":"TensorFlow examples for Valohai platform","archived":false,"fork":false,"pushed_at":"2024-08-19T07:24:05.000Z","size":99,"stargazers_count":19,"open_issues_count":0,"forks_count":61,"subscribers_count":11,"default_branch":"master","last_synced_at":"2024-08-19T08:45:05.532Z","etag":null,"topics":["machine-learning","tensorflow"],"latest_commit_sha":null,"homepage":"https://valohai.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/valohai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-17T16:50:01.000Z","updated_at":"2024-08-19T07:24:10.000Z","dependencies_parsed_at":"2024-05-22T18:25:31.727Z","dependency_job_id":"adace8b7-5b56-41a4-a5d0-4881d4c3858d","html_url":"https://github.com/valohai/tensorflow-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/valohai/tensorflow-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valohai%2Ftensorflow-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valohai%2Ftensorflow-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valohai%2Ftensorflow-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valohai%2Ftensorflow-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/valohai","download_url":"https://codeload.github.com/valohai/tensorflow-example/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valohai%2Ftensorflow-example/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268356088,"owners_count":24237380,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-02T02:00:12.353Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","tensorflow"],"created_at":"2025-08-02T08:45:00.680Z","updated_at":"2025-10-20T02:34:39.608Z","avatar_url":"https://github.com/valohai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Valohai TensorFlow Examples\n\nThis repository serves as an example for the [Valohai MLOps platform][vh]. It implements handwritten digit detection\nusing\nTensorFlow, based on [TensorFlow's example][ex].\n\nThe project demonstrates a complete machine learning pipeline consisting of five distinct steps:\n\n1. **Preprocess data**: Prepare the input data for training.\n\n2. **Train model**: Train the TensorFlow model using the preprocessed data.\n\n3. **Batch inference**: Perform inference on a batch of data using the trained model.\n\n4. **Compare predictions**: Analyze and compare the predictions generated by the model.\n\n5. **Online inference deployment**: Deploy the trained model for online inference.\n\nWithin the Valohai platform, you can explore and test four different pipelines:\n\n1. **Basic training pipeline**: This pipeline covers the essential steps for training the model.\n2. **Three parallel trainings pipeline**: This pipeline demonstrates parallel training of the model using three\n   different configurations.\n3. **Three parallel trainings with deployment pipeline**, including a special human approval block: This pipeline\n   showcases parallel training with deployment, incorporating a unique human approval block for manual verification.\n4. Additional: **'Broken' pipeline**: This pipeline highlights a distinctive feature of Valohai that allows successful\n   nodes to be reused.\n\nIf you are just starting out, it is recommended to follow [the learning path][lp] in the Valohai documentation. This\nlearning path recreates the Train model step of this example.\n\n[ex]: https://www.tensorflow.org/tutorials/quickstart/beginner\n\n[vh]: https://valohai.com/\n\n[lp]: https://docs.valohai.com/tutorials/learning-paths/fundamentals/valohai-utils/\n\n## Running on Valohai\n\nLogin to the [Valohai app][app] and create a new project.\n\n### Configure the repository:\n\nConfigure this repository as the project's repository, by following these steps:\n\n1. Go to your project's page.\n2. Navigate to the Settings tab.\n3. Under the Repository section, locate the URL field.\n4. Enter the URL of this repository.\n5. Click on the Save button to save the changes.\n\nNow you are ready to run executions, tasks and pipelines.\n\n### **Running Executions:**\n\n1. Go to the Executions tab in your project.\n2. Create a new execution by selecting the predefined steps: batch-inference, compare-predictions, preprocess-dataset,\n   train-model.\n3. Customize the execution parameters if needed.\n4. Start the execution to run the selected step.\n\n### Running Pipelines:\n\n1. Navigate to the Pipelines tab.\n2. Create a new pipeline by selecting one of the four available options.  _Note: To run the pipeline with deployment you\n   should first create deployment with name 'deployment-test'._\n3. Configure the pipeline settings.\n4. Start the pipeline.\n\n[app]: https://app.valohai.com\n\n## Running on Valohai with Terminal\n\n### Configure the repository:\n\nTo run your code on Valohai using the terminal, follow these steps:\n\n1. Install Valohai on your machine by running the following command:\n```bash\npip install valohai-cli valohai-utils\n```\n\n2. Log in to Valohai from the terminal using the command:\n```bash\nvh login\n```\n\n3. Create a project for your Valohai workflow. \n\nStart by creating a directory for your project:\n```bash\nmkdir valohai-tensorflow-example\ncd valohai-tensorflow-example\n```\n\nThen, create the Valohai project:\n```bash\nvh project create\n```\n\n4. Clone the repository to your local machine:\n```bash\ngit clone https://github.com/valohai/tensorflow-example.git .\n```\n\nCongratulations! You have successfully cloned the repository, and you can now modify the code and run it using Valohai.\n\n### Running Executions:\n\nTo run individual steps, execute the following command:\n```bash\nvh execution run \u003cstep-name\u003e --adhoc\n```\n\nFor example, to run the preprocess-dataset step, use the command:\n```bash\nvh execution run preprocess-dataset --adhoc\n```\n\n### Running Pipelines:\n\nTo run pipelines, use the following command:\n```bash\nvh pipeline run \u003cpipeline-name\u003e --adhoc\n```\n\nFor example, to run the three-trainings-pipeline-w-deployment pipeline, use the command:\n```bash\nvh pipeline run three-trainings-pipeline-w-deployment --adhoc\n```\n\nThese commands will execute your code and run it on Valohai's platform.\n## Running Locally\n\nYou can run all the steps of the pipeline locally. This requires Python 3.9 and specific packages, which you can install with:\n\n```bash\npip install -r requirements.txt\n```\n\nThe steps require different inputs to run, so you need to run them in order.\n\nPreprocess data has all the required inputs defined as defaults and can be run with:\n```bash\npython preprocess_dataset.py\n```\n\nTrain model requires the preprocessed dataset, but that is also defined as a default, so you can run:\n```bash\npython train_model.py\n```\n\nBatch inference requires both a model and some new data. The new data has default values, but the model needs to be provided, for example from an earlier train model run:\n```bash\npython batch_inference.py --model .valohai/outputs/{local_run_id}/train-model/model-{suffix}.h5\n```\n\nCompare predictions requires two or more batch inference results and optionally the corresponding models. We can run it for example like this:\n```bash\npython compare_predictions.py --predictions .valohai/outputs/{local_run_id}/batch-inference/predictions-{suffix}.json .valohai/outputs/{local_run_id}/batch-inference/predictions-{suffix}.json\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalohai%2Ftensorflow-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvalohai%2Ftensorflow-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalohai%2Ftensorflow-example/lists"}