{"id":24098487,"url":"https://github.com/matlab-deep-learning/hyperparameter-tuning-in-matlab-using-experiment-manager-and-tensorflow","last_synced_at":"2025-05-07T19:25:09.626Z","repository":{"id":45667141,"uuid":"470722357","full_name":"matlab-deep-learning/Hyperparameter-Tuning-in-MATLAB-using-Experiment-Manager-and-TensorFlow","owner":"matlab-deep-learning","description":"This example shows how to use MATLAB to train a TensorFlow model and tune it's hyperparameters using co-execution with Python.","archived":false,"fork":false,"pushed_at":"2022-09-28T18:19:28.000Z","size":510,"stargazers_count":17,"open_issues_count":0,"forks_count":3,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-14T21:51:24.430Z","etag":null,"topics":["co-execution","deep-learning","example","hyperparameter-tuning","matlab","model-training","python","tensorflow"],"latest_commit_sha":null,"homepage":"https://www.mathworks.com/solutions/deep-learning.html","language":"MATLAB","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matlab-deep-learning.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null}},"created_at":"2022-03-16T19:21:55.000Z","updated_at":"2024-12-29T05:59:55.000Z","dependencies_parsed_at":"2023-01-18T18:17:00.519Z","dependency_job_id":null,"html_url":"https://github.com/matlab-deep-learning/Hyperparameter-Tuning-in-MATLAB-using-Experiment-Manager-and-TensorFlow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2FHyperparameter-Tuning-in-MATLAB-using-Experiment-Manager-and-TensorFlow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2FHyperparameter-Tuning-in-MATLAB-using-Experiment-Manager-and-TensorFlow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2FHyperparameter-Tuning-in-MATLAB-using-Experiment-Manager-and-TensorFlow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2FHyperparameter-Tuning-in-MATLAB-using-Experiment-Manager-and-TensorFlow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matlab-deep-learning","download_url":"https://codeload.github.com/matlab-deep-learning/Hyperparameter-Tuning-in-MATLAB-using-Experiment-Manager-and-TensorFlow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252942354,"owners_count":21829054,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["co-execution","deep-learning","example","hyperparameter-tuning","matlab","model-training","python","tensorflow"],"created_at":"2025-01-10T14:46:13.944Z","updated_at":"2025-05-07T19:25:09.604Z","avatar_url":"https://github.com/matlab-deep-learning.png","language":"MATLAB","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hyperparameter Tuning in MATLAB using Experiment Manager \u0026 TensorFlow\n![](images/ExperimentManagerCoexecution.png)\n\n## This example shows how to develop a speaker identification algorithm with MATLAB and TensorFlow:\n* Preprocess audio data in MATLAB\u0026reg; using the [Audio Toolbox](https://www.mathworks.com/products/audio.html)\n* Train a TensorFlow\u0026trade; model using co-execution with Python\u0026reg;\n* Tune hyperparameters of a model from MATLAB using the [Experiment Manager App](https://www.mathworks.com/help/deeplearning/ref/experimentmanager-app.html)\n\n## Overview\n\nThere are different options for accessing Deep Learning Models within MATLAB, including:\n\n1. Using models created in MATLAB using [Deep Learning Toolbox](https://www.mathworks.com/products/deep-learning.html)\n2. [Converting](https://www.mathworks.com/help/deeplearning/deep-learning-import-and-export.html) models from other frameworks into MATLAB\n3. Co-executing models from other frameworks with MATLAB\n\nThis example provides an an overview of the third approach. For further details on co-execution, see this example: [MATLAB Calling TensorFlow for Image Classification](https://github.com/matlab-deep-learning/Image-Classification-in-MATLAB-Using-TensorFlow).\n\n## Requirements \n\n* [MATLAB R2021b or later](https://www.mathworks.com/products/matlab.html)\n* [Deep Learning Toolbox](https://www.mathworks.com/products/deep-learning.html)\n* [Statistics and Machine Learning Toolbox](https://www.mathworks.com/products/statistics.html) \n* [Audio Toolbox](https://www.mathworks.com/products/audio.html) *Required to run setup script for downloading and preprocessing data\n* [Tensorflow 2-2.6](https://www.tensorflow.org/install)\n* [Python 3.8+](https://www.python.org/downloads/)\n\n## Setup\n\nIn this example, you use a subset of the LibriSpeech Data Set [1], a large set of English language audioclips sampled at 16 kHz.\n\nTo download the data set and preprocess the data, run the script [hyperparameterTuningExample](https://insidelabs-git.mathworks.com/noahsung/experiment-manager-coexecution/-/blob/master/hyperparameterTuningExample.m). \n```Matlab:Code(Display)\n\u003e\u003e hyperparameterTuningExample\n```\nThe script saves the preprocessed data to the files `audioTrainingData.mat` and `smallerAudioTrainingData.mat` inside the `SpeakerIdentificationProject` folder, checks that your Python environment is configured correctly, and opens Experiment Manager.\n\n## Open the experiment\n\nWhen you open Experiment Manager, a dialog box provides links to the getting started tutorials and your recent projects, as well as buttons to create a new project or open an example from the documentation.\n\n![Startup Page](images/startup_page.PNG)\n\n- In the left pane, click **Open**.\n- Navigate into the `SpeakerIdentificationProject` directory and select the `SpeakerIdentificationProject.prj` file.\n\n![Experiment Browser](images/experiment_browser.PNG)\n\n- In the Experiment Browser, double click **Experiment1**.\n\n![Experiment Setup](images/experiment_setup.PNG)\n\nThe **Experiment** pane displays the description, tuning strategy, hyperparameters, and training function that define the experiment. This experiment uses Bayesian Optimization to find the hyperparameter values (number of epochs, minibatch size, and learning rate) that optimize the test accuracy.\n\nThe training function [Experiment1_training1](https://insidelabs-git.mathworks.com/noahsung/experiment-manager-coexecution/-/blob/master/SpeakerIdentificationProject/Experiment1_training1.m) specifies the training procedure used by the experiment. To view the training function in the MATLAB Editor, under **Training Function**, click **Edit**.\n\n```\nfunction output = Experiment1_training1(params,monitor)\n```\nBecause the `reducedDataset` flag is set to true, the training function loads the training and test data from the `smallerAudioTrainingData.mat` file. Using this smaller data set significantly reduces training time at the expense of performance. To use the larger data set, set the `reducedDataset` flag to false.\n```\n    reducedDataset = true;\n    if reducedDataset\n        load('smallerAudioTrainingData.mat','XTrain','YTrain','XTest','YTest');\n    else\n        load('audioTrainingData.mat','XTrain','YTrain','XTest','YTest');\n    end\n\n```\nThe `Experiment1_training1` function converts the training and test data to numpy arrays and passes them, along with the trial hyperparameters and filename for the saved model, as arguments to the Python function [trainer.py](https://insidelabs-git.mathworks.com/noahsung/experiment-manager-coexecution/-/blob/master/SpeakerIdentificationProject/trainer.py).\n```\n    result = py.trainer.train( ...\n        py.numpy.array(XTrain), ...\n        py.numpy.array(YTrain), ...\n        py.numpy.array(XTest), ...\n        py.numpy.array(YTest), ...\n        params, ...\n        modelFile);\n```\n\n## Run the Experiment\n\nTo run the experiment, click the **Run** button. \n\nBy default, the experiment runs a maximum of 30 trials. To limit the duration of the experiment, you can modify the **Bayesian Optimization Options** by reducing the maximum running time or the maximum number of trials. However, note that running fewer than 30 trials can prevent the Bayesian optimization algorithm from converging to an optimal set of hyperparameters.\n\nA table of results displays the hyperparameter and metric function values for each trial. Experiment Manager highlights the trial with the optimal value for the selected metric.\n\n![Experiment Running](images/experiment_running.PNG)\n\nAfter training is complete, Experiment Manager saves each trained model as a TensorFlow SavedModel.\n\n## References\n\n[1] V. Panayotov, G. Chen, D. Povey and S. Khudanpur, \"Librispeech: An ASR corpus based on public domain audio books,\" 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, QLD, 2015, pp. 5206-5210, doi: 10.1109/ICASSP.2015.7178964\n\nCopyright 2022, The MathWorks, Inc.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatlab-deep-learning%2Fhyperparameter-tuning-in-matlab-using-experiment-manager-and-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatlab-deep-learning%2Fhyperparameter-tuning-in-matlab-using-experiment-manager-and-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatlab-deep-learning%2Fhyperparameter-tuning-in-matlab-using-experiment-manager-and-tensorflow/lists"}