{"id":14959011,"url":"https://github.com/jiehe96/ios_tensorflow_objectdetection_example","last_synced_at":"2025-06-13T14:02:24.688Z","repository":{"id":44894372,"uuid":"96388694","full_name":"JieHe96/iOS_Tensorflow_ObjectDetection_Example","owner":"JieHe96","description":"An iOS application of Tensorflow Object Detection with different models: SSD with Mobilenet, SSD with InceptionV2, Faster-RCNN-resnet101","archived":false,"fork":false,"pushed_at":"2018-01-12T14:59:27.000Z","size":61814,"stargazers_count":128,"open_issues_count":16,"forks_count":27,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-07T02:02:28.608Z","etag":null,"topics":["faster-rcnn","ios","objectdetection","ssd","ssd-inceptionv2","ssd-mobilenet","tensorflow","tensorflow-ios","tensorflow-models"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JieHe96.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-06T04:36:25.000Z","updated_at":"2024-09-30T05:23:31.000Z","dependencies_parsed_at":"2022-08-31T22:00:24.785Z","dependency_job_id":null,"html_url":"https://github.com/JieHe96/iOS_Tensorflow_ObjectDetection_Example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JieHe96%2FiOS_Tensorflow_ObjectDetection_Example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JieHe96%2FiOS_Tensorflow_ObjectDetection_Example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JieHe96%2FiOS_Tensorflow_ObjectDetection_Example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JieHe96%2FiOS_Tensorflow_ObjectDetection_Example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JieHe96","download_url":"https://codeload.github.com/JieHe96/iOS_Tensorflow_ObjectDetection_Example/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252038170,"owners_count":21684635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["faster-rcnn","ios","objectdetection","ssd","ssd-inceptionv2","ssd-mobilenet","tensorflow","tensorflow-ios","tensorflow-models"],"created_at":"2024-09-24T13:18:41.483Z","updated_at":"2025-05-02T12:31:32.641Z","avatar_url":"https://github.com/JieHe96.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tensorflow iOS ObjectDetection Example\n\nThis example gives a demo of loading a Object Detection model to the iOS platform and using it to do the object detection work. The currently supported models are: \u003ca href=\"http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_11_06_2017.tar.gz\"\u003essd_mobilenet_v1_coco\u003c/a\u003e, \u003ca href=\"http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_11_06_2017.tar.gz\"\u003essd_inception_v2_coco\u003c/a\u003e, \u003ca href=\"http://download.tensorflow.org/models/object_detection/faster_rcnn_resnet101_coco_11_06_2017.tar.gz\"\u003efaster_rcnn_resnet101_coco\u003c/a\u003e.\n\n## Quick Start\n  #### 1.Setup Environment Variable in Terminal\n  First open the terminal, type in the following command:\n  ```\n  export TF_ROOT=/your//tensorflow/root/\n  ```\n  Then cd to the example folder and check your tensroflow version and the correctness of your tensorflow root path:\n  ```\n  bash config.sh\n  ```\n  The config.sh file will automatically check your TensorFlow version and copy some files that are necessary for the compile process.\n  After running the config.sh, if the terminal show the following result then you are good for next step:\n  ```\n  ok=\u003e current version: # Release 1.4.0\n  ok=\u003e Ready!\n  ```\n  Otherwise, please go to the TensorFlow official website and download the latest version of TensorFlow.\n  #### 2.Compile dependencies\n  Compile ios dependencies:\n  ```\n  cd $TF_ROOT\n  tensorflow/contrib/makefile/build_all_ios_ssd.sh\n  ```\n  #### 3.Setup project in Xcode\n  Open the project in Xcode\n  Then in the \"tf_root.xcconfig\" replace the TF_ROOT with your tensorflow root's absolute path.\n  Finally, add the \"op_inference_graph.pb\" to your project folder.\n  #### 4.Build \u0026 Run the project\n  Note: If you'd like to run other two models, download it from the above links and add the .ph file to your project.\n  #### 5.Other Model Resource\n  For other model file, please check my another \u003ca href=\"https://github.com/JieHe96/tf_resource\"\u003erepo\u003c/a\u003e.\n  \n  \n## Result running on iOS device\n![alt text](https://github.com/JieHe96/iOS_Tensorflow_ObjectDetection_Example/blob/master/iphone_result.png)\n![alt text](https://github.com/JieHe96/ios_SSDMobilenet_tensorflow_example/blob/master/ios_result.png)\n\n## Update content for TensorFlow 1.4.0\nAfter updating the TensorFlow to version 1.4.0, I did the following change to make sure the example could run successfully:\n1. Follow the steps in the new QuickStart section\n2. If you've tryed the previous version of this example before, then you need to re-compile the libtensorflow.a, otherwise many register will not able to be found:\n```\ncd $TF_ROOT\nrm -r tensorflow/contrib/makefile/gen/lib/ tensorflow/contrib/makefile/gen/obj/\ntensorflow/contrib/makefile/build_tflib_ssd.sh\n```\n3. In the iOS project, add the new header search, and also make sure you use the right $(TF_ROOT) in the header search:\n```\n$(TF_ROOT)/tensorflow/contrib/makefile/downloads/nsync/public/\n```\n4. In the iOS project, add the new lib search:\n```\n$(TF_ROOT)/tensorflow/contrib/makefile/gen/nsync\n```\n5. In the Makefile_ios, comment out the line:\n```\nTF_CC_SRCS += tensorflow/core/platform/default/gpu_tracer.cc\n```\n\n## Below content is the detailed explanation and FAQs of this example\n## Introduciton\n\nRecently Google released the Tensorflow Object Detection API which includes the selection of multiple models. However, the API does not contain a iOS version of implementation. Therefore, in this example, I wrote a IOS implementation of the object detection API, including the SSDMobilenet model. For this example, it maintains the same functionality as the python version of object detection API. Furthermore, the IOS code is derived from Google tensorflow ios_camera_example.\n\n## Prerequisites\n\n\n### Installing\n#### 1.Xcode\nYou’ll need Xcode 7.3 or later.\n#### 2.Tensorflow\nDownload the Google Tensorflow repository to local:\nhttps://github.com/tensorflow/tensorflow\n#### 3.Bazel\nIf you don't have Bazel, please follow the Bazel's official installation process:\nhttps://docs.bazel.build/versions/master/install.html\n#### 4.Repository Download\nDownload this repository to local and put the directory into the tensorflow directory you just downloaded.\n#### 5.Graph Download\nFollow the below instruction to download the model you want:\nhttps://github.com/tensorflow/models/blob/master/object_detection/g3doc/detection_model_zoo.md\nWe only need the graph file, aka .pb file (We chose SSDMobilenet as example):\n```\nfrozen_inference_graph.pb\n```\nThen download the label file for the model you chose:\nhttps://github.com/tensorflow/models/tree/master/object_detection/data\n```\nmscoco_label_map.pbtxt\n```\n\n### Build\n#### 1.Build Bazel\nBefore you could run the project, you need to build some bazel depedicies by following the Google instruction:\nIf this is your first time build Bazel, please follow the below link to configure the installation:\nhttps://www.tensorflow.org/install/install_sources#configure_the_installation\n##### Optional:\nIf you'd like to get the info of graph's input/output name, using the following command:\n```\nbazel build tensorflow/tools/graph_transforms:summarize_graph\nbazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=YOUR_GRAPH_PATH/example_graph.pb\n```\n\n#### 2.Change Makefile\nThe Makefile is under \"tensorflow/contrib/makefile/\".\n  - In the Makefile, first delete the line \"-D__ANDROID_TYPES_SLIM__ \\\" under \"# Settings for iOS.\" for all \"$(IOS_ARCH)\".\n  \n#### 3.Generate ops_to_register.h\nOne of the biggest issues during iOS Tensorflow building is the missing of different OpKernel. One may get similar errors like below:\n```\nInvalid argument: No OpKernel was registered to support Op 'Equal' with these attrs.  Registered devices: [CPU], Registered kernels:\n  \u003cno registered kernels\u003e\n```\nIn order to solve the problems in one time, we use Bazel to generate a ops_to_register.h, which contains all the needed Ops to loading the certain graph into project. An example of command-line usage is:\n```\n  bazel build tensorflow/python/tools:print_selective_registration_header \n  bazel-bin/tensorflow/python/tools/print_selective_registration_header \\\n    --graphs=path/to/graph.pb \u003e ops_to_register.h\n```\nThis will generate a ops_to_register.h file in the current directory. Copy the file to \"tensorflow/core/framework/\". Then when compiling tensorflow, pass -DSELECTIVE_REGISTRATION and -DSUPPORT_SELECTIVE_REGISTRATION \nSee tensorflow/core/framework/selective_registration.h for more details.\n##### Attention:\nFor different models, you also need to provide certain ops_to_register.h file that fits the model. Therefore, if you'd like to contain several models in one project, you need to first generate a ops_to_register.h for each different model, then merge all the ops_to_register.h into one file. By doing the operation, you could use different models in one project without compiling the Tensorflow lib separately.\n\nIn this example, we provided a combined ops_to_register.h file which is compatible with ssd_mobilenet_v1_coco and ssd_inception_v2_coco  and faster_rcnn_resnet101_coco.\n\n#### 4.Build Tensorflow iOS library\nInstead of using build_all_ios for the building process, we divide the process into several steps:\n  - In tensorflow/contrib/makefile/compile_ios_protobuf.sh, add the line\n  ```\n  export MACOSX_DEPLOYMENT_TARGET=\"10.10\"\n  ```\n  after\n  ```\n  set -x\n  set -e\n  ```\n  - Download the dependencies:\n  ```\n  tensorflow/contrib/makefile/download_dependencies.sh\n  ```\n  - Next, you will need to compile protobufs for iOS:\n  ```\n  tensorflow/contrib/makefile/compile_ios_protobuf.sh \n  ```\n  - Then create the libtensorflow-core.a:\n  ```\n  tensorflow/contrib/makefile/compile_ios_tensorflow.sh \"-O3  -DANDROID_TYPES=ANDROID_TYPES_FULL -DSELECTIVE_REGISTRATION -DSUPPORT_SELECTIVE_REGISTRATION\"\n  ```\n  If you'd like to shorten the building time, you could choose to build the \"compile_ios_tensorflow_s.sh\" file provided in the repository. The \"complie_ios_tensorflow_s.sh\" only complie two IOS_ARCH: ARM64 and x86_64, which make the building process much shorter. Make sure to copy the file to the \"tensorflow/contrib/makefile/\" directory before building. Then the build command is changed to:\n  ```\n  tensorflow/contrib/makefile/compile_ios_tensorflow_s.sh \"-O3  -DANDROID_TYPES=ANDROID_TYPES_FULL -DSELECTIVE_REGISTRATION -DSUPPORT_SELECTIVE_REGISTRATION\"\n  ```\n  \n  Make sure the script has generated the following .a files:\n  ```\n  tensorflow/contrib/makefile/gen/lib/libtensorflow-core.a\n  tensorflow/contrib/makefile/gen/protobuf_ios/lib/libprotobuf.a\n  tensorflow/contrib/makefile/gen/protobuf_ios/lib/libprotobuf-lite.a\n  ```\n#### 5.Xocde Configuration\n  - Open xcode example, put the model and label file you just downloaded into \"TF_Graph\" folder in the project\n  - Follow the link for configuration:\n  https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/ios#creating-your-own-app-from-your-source-libraries\n  (ps: you need to contain a absolute path of \"libtensorflow-core.a\" after the \"-force_load\" option)\n  \n### Running\nBefore you run, make sure to recompile the libtensorflow-core.a according to the modified Makefile. Otherwise, following error may be generated during the runtime:\n```\nError adding graph to session:\nNo OpKernel was registered to support Op 'Less' with these attrs.  \nRegistered devices: [CPU],     Registered kernels: device='CPU';\n T in [DT_FLOAT]......\n ```\nOnce you finish the above process, you could run the project by click the build button in the Xcode\n\n### Label Config\nIn order to get the lable name for each detected box, you have to use proto buffer data structure. In the SSDMobilenet model, the label file is stored as a proto buffer structure, so that you need to proto's own function to extract the data. \n\nTo use proto buffer, first install it by \n```\nbrew install protobuf\n```\nThen follow https://developers.google.com/protocol-buffers/docs/cpptutorial to compile the proto buffer.\nAfter the compiling, you'll get a .h and a .cc files which contain the declaration and implementation of your classes.\n```\nexample.pb.h\nexample.pn.cc\n```\nFinally you could use the funcition in the files to extract your label data.\n\n### FAQ\n  1. If you still get errors like after finishing the above instruction:\n  ```\n  Invalid argument: No OpKernel was registered to support Op 'xxx' with these attrs.  Registered devices: [CPU], Registered kernels:\n  \u003cno registered kernels\u003e\n  ```\n  - Solution: First check if you use the certain ops_to_register.h for the model you choose.\n  Then check the file \"tensorflow/contrib/makefile/tf_op_files.txt\" and add the \"tensorflow/core/kernels/cwise_op_xxx.cc\" into the txt file if it is not in there.\n  \n  2. Make sure you've added the  \"-O3  -DANDROID_TYPES=ANDROID_TYPES_FULL -DSELECTIVE_REGISTRATION -DSUPPORT_SELECTIVE_REGISTRATION\" when run the \"compile_ios_tensorflow_s.sh\".\n  \n\n  3. Invalid argument: No OpKernel was registered to support Op 'Conv2D' with these attrs. Registered devices: [CPU], Registered kernels:\n  ```\n   [[Node: FeatureExtractor/InceptionV2/InceptionV2/Conv2d_1a_7x7/separable_conv2d = Conv2D[T=DT_FLOAT, data_format=\"NHWC\", padding=\"VALID\", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](FeatureExtractor/InceptionV2/InceptionV2/Conv2d_1a_7x7/separable_conv2d/depthwise, FeatureExtractor/InceptionV2/Conv2d_1a_7x7/pointwise_weights/read)]]\n  ```\n  - Solution: Because of the special structure of models, they include the GEMM function in the conv layers. However, the default Makefile does not use the GEMM for conv layer, so that you need to munually replace the line in the ops_to_register.h. In your ops_to_register.h replace the line \"Conv2DOp\u003cCPUDevice, float\u003e\" with \"Conv2DUsingGemmOp\u003c float, Im2ColConvFunctor\u003cfloat, float, float, FastGemmFunctor\u003cfloat, float, float\u003e\u003e\u003e\" and the problem will be solved.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjiehe96%2Fios_tensorflow_objectdetection_example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjiehe96%2Fios_tensorflow_objectdetection_example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjiehe96%2Fios_tensorflow_objectdetection_example/lists"}