{"id":24098449,"url":"https://github.com/matlab-deep-learning/object-detection-and-explainability-for-tensorflow-and-pytorch-models","last_synced_at":"2026-02-13T15:05:20.665Z","repository":{"id":242900302,"uuid":"808300834","full_name":"matlab-deep-learning/object-detection-and-explainability-for-tensorflow-and-pytorch-models","owner":"matlab-deep-learning","description":null,"archived":false,"fork":false,"pushed_at":"2024-09-13T12:53:08.000Z","size":1831,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-08-31T10:37:48.137Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"MATLAB","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matlab-deep-learning.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-05-30T19:31:26.000Z","updated_at":"2025-03-12T19:20:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"12d1b165-5460-483a-b22d-c24cd78ed3c8","html_url":"https://github.com/matlab-deep-learning/object-detection-and-explainability-for-tensorflow-and-pytorch-models","commit_stats":null,"previous_names":["matlab-deep-learning/object-detection-and-explainability-for-tensorflow-and-pytorch-models"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/matlab-deep-learning/object-detection-and-explainability-for-tensorflow-and-pytorch-models","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Fobject-detection-and-explainability-for-tensorflow-and-pytorch-models","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Fobject-detection-and-explainability-for-tensorflow-and-pytorch-models/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Fobject-detection-and-explainability-for-tensorflow-and-pytorch-models/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Fobject-detection-and-explainability-for-tensorflow-and-pytorch-models/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matlab-deep-learning","download_url":"https://codeload.github.com/matlab-deep-learning/object-detection-and-explainability-for-tensorflow-and-pytorch-models/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matlab-deep-learning%2Fobject-detection-and-explainability-for-tensorflow-and-pytorch-models/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29411138,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-13T06:24:03.484Z","status":"ssl_error","status_checked_at":"2026-02-13T06:23:12.830Z","response_time":78,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-10T14:45:54.813Z","updated_at":"2026-02-13T15:05:20.644Z","avatar_url":"https://github.com/matlab-deep-learning.png","language":"MATLAB","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Open in MATLAB Online](https://www.mathworks.com/images/responsive/global/open-in-matlab-online.svg)](https://matlab.mathworks.com/open/github/v1?repo=matlab-deep-learning/object-detection-and-explainability-for-tensorflow-and-pytorch-models)\n\n# Object Detection and Explainability in MATLAB for TensorFlow and PyTorch Models\n\nThis repository contains two examples on how to use TensorFlow™ and PyTorch® models for object detection in MATLAB® and how to explain the models' decisions with D-RISE.\n* Object Detection and Explainability with Imported TensorFlow Model\n* Object Detection and Explainability with PyTorch Model Using Co-Execution\n\n\n# Example 1: Object Detection and Explainability with Imported TensorFlow Model\n\nThis example shows how to import a TensorFlow model for object detection, how to use the imported model in MATLAB and visualize the detections, and how to use D\\-RISE to explain the predictions of the model.\n\n## Requirements\n\nTo run the following code, you need:\n\n-  [MATLAB](https://www.mathworks.com/products/matlab.html) R2024a \n-  [Deep Learning Toolbox](https://www.mathworks.com/products/deep-learning.html)™ \n-  [Deep Learning Toolbox Converter for TensorFlow Models](https://www.mathworks.com/matlabcentral/fileexchange/64649-deep-learning-toolbox-converter-for-tensorflow-models)™\n-  [Computer Vision Toolbox](https://www.mathworks.com/products/computer-vision.html)™\n-  [Deep Learning Toolbox Verification Library](https://www.mathworks.com/products/deep-learning-verification-library.html)™    \n\n## Import and Initialize Network\n\nImport a pretrained TensorFlow model for object detection. The model is in the SavedModel format. You can get the model from the [TensorFlow 2 Detection Model Zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md).\n\n```matlab\nmodelFolder = \"centernet_resnet50_v2_512x517_coco17\";\ndetector = importNetworkFromTensorFlow(modelFolder);\n```\n\n```matlabTextOutput\nImporting the saved model...\nTranslating the model, this may take a few minutes...\nImport finished.\n```\n\nSpecify the input size of the imported network. You can find the expected image size stated in the name of the TensorFlow network. The data format of the `dlarray` object must have the dimensions `\"SSCB\"` (spatial, spatial, channel, batch) to represent a 2\\-D image input. For more information, see [Data Formats for Prediction with dlnetwork](https://www.mathworks.com/help/deeplearning/ug/tips-on-importing-models-from-tensorflow-pytorch-and-onnx.html#mw_b48d26e1-7484-4802-aef3-502f61c36795). Initialize the imported network.\n\n```matlab\ninput_size = [512 512 3];\ndetector = detector.initialize(dlarray(ones(512,512,3,1),\"SSCB\"))\n```\n\n```matlabTextOutput\ndetector = \n  dlnetwork with properties:\n\n         Layers: [1x1 centernet_resnet50_v2_512x517_coco17.kCall11498]\n    Connections: [0x2 table]\n     Learnables: [388x3 table]\n          State: [0x3 table]\n     InputNames: {'kCall11498'}\n    OutputNames: {'kCall11498/detection_boxes'  'kCall11498/detection_classes'  'kCall11498/detection_scores'  'kCall11498/num_detections'}\n    Initialized: 1\n\n  View summary with summary.\n\n```\n\nSave the imported network, so you don't have to re\\-import the TensorFlow model if you want to run the following sections independently.\n\n```matlab\nsave(\"centernet_object_detector\",\"detector\",\"-v7.3\")\n```\n## Detect with Imported Network\n\nOptionally, load the imported network. The network has four outputs: bounding boxes, classes, scores, and number of detections.\n\n```matlab\n%load(\"centernet_object_detector.mat\")\nmlOutputNames = detector.OutputNames'\n```\n\n```matlabTextOutput\nmlOutputNames = 4x1 cell    \n'kCall11498/detection_boxes'  \n'kCall11498/detection_classes'\n'kCall11498/detection_scores' \n'kCall11498/num_detections'   \n\n```\n\nRead the image that you want to use for object detection. Perform object detection on the image.\n\n```matlab\nimg = imread(\"testCar.png\");\n[y1,y2,y3,y4] = detector.predict(dlarray(single(img),\"SSCB\"));\n```\n## Get Detections with Highest Scores\n\nCreate a map of all the network outputs.\n\n```matlab\nmlOutputMap = containers.Map;\nmlOutputs = {y1,y2,y3,y4};\nfor i = 1:numel(mlOutputNames)\n    opNameStrSplit = strsplit(mlOutputNames{i},'/');\n    opName = opNameStrSplit{end};\n    mlOutputMap(opName) = mlOutputs{i};\nend\n```\n\nGet the detections with scores above the threshold `thr`, and the corresponding class labels.\n\n```matlab\nthr = 0.5;\n[bboxes,classes,scores,num_box] = bestDetections(img,mlOutputMap,thr);\nclass_labels = getClassLabels(classes);\n```\n## Visualize Object Detection\n\nCreate the labels associated with each of the detected objects.\n\n```matlab\ncolons = repmat(\": \",[1 num_box]);\npercents = repmat(\"%\",[1 num_box]);\nlabels = strcat(class_labels,colons,string(round(scores*100)),percents);\n```\n\nVisualize the object detection results with annotations.\n\n```matlab\nfigure\noutputImage = insertObjectAnnotation(img,\"rectangle\",bboxes,labels,LineWidth=1,Color=\"green\");\nimshow(outputImage)\n```\n\n![](images/object_detection_for_car_image_TF.png)\n\n## Explainability for Object Detector\n\nExplain the predictions of the object detection network using D\\-RISE. Specify a custom detection function to use D\\-RISE with the imported TensorFlow network.\n\n```matlab\ntargetBox = bboxes(1,:);\ntargetLabel = 1;\nscoreMap = drise(@(img)customDetector(img),img,targetBox,targetLabel);\n```\n\nPlot the results.\n\n```matlab\nfigure\nannotatedImage = insertObjectAnnotation(img,\"rectangle\",targetBox,\"vehicle\");\nimshow(annotatedImage)\nhold on\nimagesc(scoreMap,AlphaData=0.5)\ntitle(\"DRISE Map: Custom Detector\")\nhold off\ncolormap jet\n```\n\n![](./images/drise_for_car_image.png)\n\n\n# Example 2: Object Detection and Explainability with PyTorch Model Using Co\\-Execution\n\nThis example shows how to perform object detection with a PyTorch model using co\\-execution, and how to use D\\-RISE to explain the predictions of the PyTorch model.\n\n## Requirements\n\nTo run the following code, you need:\n\n-  [MATLAB](https://www.mathworks.com/products/matlab.html) R2024a \n-  [Computer Vision Toolbox](https://www.mathworks.com/products/computer-vision.html)\n-  [Deep Learning Toolbox Verification Library](https://www.mathworks.com/products/deep-learning-verification-library.html)   \n-  [Python](https://www.python.org/)® (tested with 3.11.2) \n-  [PyTorch](https://pytorch.org/) (tested with 2.2.0) \n-  [TorchVision](https://pytorch.org/vision/stable/index.html) (tested with 0.17.0) \n-  [NumPy](https://numpy.org/) (tested with 1.26.4) \n\n## Python Environment\n\nSet up the Python environment by first running commands at a command prompt (Windows® machine) and then, set up the Python interpreter in MATLAB.\n\n\nGo to your working folder. Create the Python virtual environment `venv` in a command prompt outside MATLAB. If you have multiple versions of Python installed, you can specify which Python version to use for your virtual environment.\n\n```\npython -m venv env\n```\n\nActivate the Python virtual environment `env` in your working folder.\n\n```\nenv\\Scripts\\activate\n```\n\nInstall the necessary Python libraries for this example. Check the installed versions of the libraries.\n\n```\npip install torchvision\npython -m pip show torchvision\n```\n\nSet up the Python interpreter for MATLAB® by using the [`pyenv`](https://www.mathworks.com/help/matlab/ref/pyenv.html) function. Specify the version of Python to use.\n\n```matlab\npe = pyenv(Version=\".\\env\\Scripts\\python.exe\",ExecutionMode=\"OutOfProcess\")\n```\n\n```matlabTextOutput\npe = \n  PythonEnvironment with properties:\n\n          Version: \"3.11\"\n       Executable: \"C:\\Users\\sparaske\\OneDrive - MathWorks\\Documents\\AI_Customers\\Demos\\PyTorch_object_detection\\code\\env\\Scripts\\python.exe\"\n          Library: \"C:\\Users\\sparaske\\AppData\\Local\\Programs\\Python\\Python311\\python311.dll\"\n             Home: \"C:\\Users\\sparaske\\OneDrive - MathWorks\\Documents\\AI_Customers\\Demos\\PyTorch_object_detection\\code\\env\"\n           Status: NotLoaded\n    ExecutionMode: OutOfProcess\n\n```\n## Object Detection\n\nRead the image that you want to use for object detection.\n\n```matlab\nimg_filename = \"testCar.png\";\nimg = imread(img_filename);\n```\n\nPerform object detection with a PyTorch model using co\\-execution.\n\n```matlab\npyrun(\"from PT_object_detection import loadPTmodel, detectPT\")\n[model,weights] = pyrun(\"[a,b] = loadPTmodel()\",[\"a\" \"b\"]);\npredictions = pyrun(\"a = detectPT(b,c,d)\",\"a\",b=img,c=model,d=weights);\n```\n\nConvert the prediction outputs from Python data types to MATLAB data types.\n\n```matlab\n[bboxes,labels,scores] = convertVariables(predictions,imread(img_filename));\n```\n\nGet the class labels.\n\n```matlab\nclass_labels = getClassLabels(labels);\n```\n## Visualization\n\nCreate the labels associated with each of the detected objects.\n\n```matlab\nnum_box = length(scores);\ncolons = repmat(\": \",[1 num_box]);\npercents = repmat(\"%\",[1 num_box]);\nclass_labels1 = strcat(class_labels,colons,string(round(scores'*100)),percents);\n```\n\nVisualize the object detection results with annotations.\n\n```matlab\nfigure\noutputImage = insertObjectAnnotation(img,\"rectangle\",bboxes,class_labels1,LineWidth=1,Color=\"green\");\nimshow(outputImage)\n```\n\n![](./images/object_detection_for_car_image_PT.png)\n\n## Explainability\n\nExplain the predictions of the PyTorch model using D\\-RISE. Specify a custom detection function to use D\\-RISE.\n\n```matlab\ntargetBbox = bboxes(1,:);\ntargetLabel = 1;\nscoreMap = drise(@(img)customDetector(img),img,targetBbox,targetLabel,...\n    NumSamples=512,MiniBatchSize=8,Verbose=true);\n```\n\n```matlabTextOutput\nExplaining 1 detections.\nNumber of mini-batches to process: 64\n..........   ..........   ..........   ..........   .......... (50 mini-batches)\n..........   ....                                              (64 mini-batches)\nTotal time = 1145.9secs.\n```\n\nCopyright 2024, The MathWorks, Inc.\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatlab-deep-learning%2Fobject-detection-and-explainability-for-tensorflow-and-pytorch-models","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatlab-deep-learning%2Fobject-detection-and-explainability-for-tensorflow-and-pytorch-models","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatlab-deep-learning%2Fobject-detection-and-explainability-for-tensorflow-and-pytorch-models/lists"}