{"id":16520853,"url":"https://github.com/jhrcook/coremldemoapp","last_synced_at":"2026-04-13T04:03:25.358Z","repository":{"id":115870073,"uuid":"203067026","full_name":"jhrcook/CoreMLDemoApp","owner":"jhrcook","description":"A demonstration of using CoreML","archived":false,"fork":false,"pushed_at":"2019-09-21T00:32:51.000Z","size":36005,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-13T11:50:09.597Z","etag":null,"topics":["core-ml","coreml","ios","ios-app","machine-learning","machinelearning","python3","r","swift","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jhrcook.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-18T23:31:43.000Z","updated_at":"2021-03-17T14:07:21.000Z","dependencies_parsed_at":null,"dependency_job_id":"af8dc8c0-c7bd-4d24-9caa-5f3765a90786","html_url":"https://github.com/jhrcook/CoreMLDemoApp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhrcook%2FCoreMLDemoApp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhrcook%2FCoreMLDemoApp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhrcook%2FCoreMLDemoApp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhrcook%2FCoreMLDemoApp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jhrcook","download_url":"https://codeload.github.com/jhrcook/CoreMLDemoApp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241592151,"owners_count":19987311,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["core-ml","coreml","ios","ios-app","machine-learning","machinelearning","python3","r","swift","tensorflow"],"created_at":"2024-10-11T16:53:25.988Z","updated_at":"2025-12-31T01:03:27.106Z","avatar_url":"https://github.com/jhrcook.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Demo using CoreML\n\n![ios](https://img.shields.io/badge/iOS-Plant_Tracker-999999.svg?style=flat\u0026logo=apple)\n[![jhc github](https://img.shields.io/badge/GitHub-jhrcook-181717.svg?style=flat\u0026logo=github)](https://github.com/jhrcook)\n[![jhc twitter](https://img.shields.io/badge/Twitter-@JoshDoesA-00aced.svg?style=flat\u0026logo=twitter)](https://twitter.com/JoshDoesa)\n[![jhc website](https://img.shields.io/badge/Website-Joshua_Cook-5087B2.svg?style=flat\u0026logo=telegram)](https://joshuacook.netlify.com)\n\n\n\nThis is a demonstration of using CoreML to recognize succulents from images. It is still very much in it's early stages.\n\n**Overview of the process**\n\n1. Create an R script that scrapes the plant names from [World of Succulents](https://worldofsucculents.com/browse-succulents-scientific-name).\n2. Create a shell script that uses ['googliser'](https://github.com/teracow/googliser) to download the images to a directory called \"images/\" and each plant has a subdirectory.\n3. Use TransorFlow to retrain an image classifier with my new data set.\n4. Use the `core-ml` python package to convert the TensorFlow model into one that can be imported into Xcode for CoreML.\n\n**Current Status**\n\n* I have made the web-scraping script and created a list of over 1,500 succulents.\n* I have ['googliser'](https://github.com/teracow/googliser) funcitoning and a job-array submission system to parrallelize the process for each plant.\n* [Here](./practice_plant_recognition.md), I have demonstrated the feasibility of the workflow using a sample of 5 plants.\n\n\n# Work-flow {#workflow}\n\n## Data Acquisition\n\n### Create plant name list\n\nI scraped plant names from [World of Succulents](https://worldofsucculents.com/browse-succulents-scientific-name) using '[rvest](https://cran.r-project.org/web/packages/rvest/index.html)' to retrieve and parse the HTML. The code is in \"make\\_plant\\_list.r\" and outputs a list of names to \"plant_names.txt\"\n\n```bash\nRscript make_plant_list.r\n```\n\n### Download images\n\nI am using the bash tool ['googliser'](https://github.com/teracow/googliser) to download plant images. It currently has a limit of 1,000 images  per query. This should be sufficient for my needs, though.\n\n#### Set up 'googliser'\n\nThe tool can be installed from GitHub using the following command.\n\n```bash\nwget -qN git.io/googliser.sh \u0026\u0026 chmod +x googliser.sh\n```\n\nIt requires `imagemagick`, which is available on O2.\n\n```bash\nmodule load imageMagick/6.9.1.10\n```\n\nBelow is an example command to download 20 images of *Euphorbia obesa*.\n\n```bash\n./googliser.sh \\\n  --phrase \"Euphorbia obesa\" \\\n  --number 20 \\\n  --no-gallery \\\n  --output images/Euphorbia_obesa\n```\n#### Downloading the images in parallel\n\nI downloaded all of the images for every plant by submitting a job-array, where each job downloads *N* images for a single plant. The script \"download_google_images.sh\" takes an integer (the job number) and downloads the images for the plant on that line of \"plant_names.txt\".\n\n```bash\nsbatch \\\n  --array=1-$(wc -l \u003c plant_names.txt) download_google_images.sh plant_names.txt \\\n  --constraint=\"scratch2\"\n```\n\n### Remove corrupted files and wrong formats\n\n**(The following step may no longer be necessary since each image is reportedly a JPEG.)**\n\nSome of the images were corrupted or of WEBP format that the TensorFlow script could not accept. These were filtered using another R script.\n\n```bash\nmodule load imageMagick/6.9.1.10\nRscript filter_bad_images.r\n```\n\n### Ensure all images were properly downloaded.\n\nThe R Markdown file \"check_images_downloaded.Rmd\" checks that each plant has images downloaded. It outputs an HTML file of the results.\n\n```bash\nRscript -e 'rmarkdown::render(\"check_images_downloaded.Rmd\")'\n```\n\nIn addition, if there are plants that do not have all of the images downloaded (or are within 50 images of the expected number), it creates the file \"failed_dwnlds_plant_names.txt\" with a list of plant names to be run, again.\n\n```bash\nsbatch \\\n  --array=1-$(wc -l \u003c failed_dwnlds_plant_names.txt) download_google_images.sh failed_dwnlds_plant_names.txt \\\n  --constraint=\"scratch2\"\n```\n\n\n## ML Model Creation\n\nI began by following the tutorial [How to Retrain an Image Classifier for New Categories](https://www.tensorflow.org/hub/tutorials/image_retraining) to retrain a general image classifier to recognize the images. I can then convert to and export a CoreML object and import that into a simple iOS app that tries to predict the cactus from a new image.\n\n### Install TensorFlow and TensorFlow Hub\n\n[TensorFlow](https://www.tensorflow.org) is an incredibly powerful machine learning framework that is used extensively in education, research and production. (Excitingly, there is [Swift for TensorFlow](https://www.tensorflow.org/swift), though it is still in beta (as of August 18, 2019)).\n\n\"[TensorFlow Hub](https://www.tensorflow.org/hub) is a library for the publication, discovery, and consumption of reusable parts of machine learning models.\"\n\nTo install both, we can use `pip` from within the virtual environment.\n\nCreate virtual environment.\n\n```bash\n# create and activate a virtual environment\nmodule load python/3.6.0\npython3 -m venv image-download\nsource image-download/bin/activate\n\n# install the necessary packages\npip3 install --upgrade pip\npip3 install setuptools\u003e=41.0.0\npip3 install tensorflow tensorflow-hub\npip3 install coremltools==3.0b5 tfcoreml==0.4.0b1  # betas required for CoreML3\n```\n\n### Example retraining: practice with flowers\n\nThere is an example on the tutorial for retraining ImageNet to identify several different plants by their flower. All of this was performed in a subdirectory called \"flowers_example\".\n\n```bash\nmkdir flowers_example\ncd flowers_example\n```\n\nThe images were downloaded and unarchived.\n\n```bash\ncurl -LO http://download.tensorflow.org/example_images/flower_photos.tgz\ntar xzf flower_photos.tgz\nls flower_photos\n#\u003e daisy  dandelion  LICENSE.txt  roses  sunflowers  tulips\n```\n\nThe retraining script was downloaded from GitHub.\n\n```bash\ncurl -LO https://github.com/tensorflow/hub/raw/master/examples/image_retraining/retrain.py\n```\n\nThe script was run on the plant images.\n\n```bash\npython retrain.py --image_dir ./flower_photos\n```\n\n[If the connection to O2 is set up correctly](), the TensorBoard can be run and opened locally.\n\n```bash\ntensorboard --logdir /tmp/retrain_logs\n#\u003e TensorBoard 1.14.0 at http://compute-e-16-229.o2.rc.hms.harvard.edu:6006/ (Press CTRL+C to quit)\n```\n\nFinally, the newe model was used to classify a photo using the \"label_image.py\" script (downloaded from GitHub).\n\n```bash\n# download the script\ncurl -LO https://github.com/tensorflow/tensorflow/raw/master/tensorflow/examples/label_image/label_image.py\n# run it on an image\npython label_image.py \\\n    --graph=/tmp/output_graph.pb \\\n    --labels=/tmp/output_labels.txt \\\n    --input_layer=Placeholder \\\n    --output_layer=final_result \\\n    --image=./flower_photos/daisy/21652746_cc379e0eea_m.jpg\n#\u003e daisy 0.99798715\n#\u003e sunflowers 0.0011478926\n#\u003e dandelion 0.00045892605\n#\u003e tulips 0.0003524925\n#\u003e roses 5.3392014e-05\n```\n\nIt worked!\n\n### Small-scale experiment\n\nYou can see the results from a small-scale experiement [here](./practice_plant_recognition.md). Overall, it went well, but the plants used were obviously different from each other, so it may be worth running a test with more similar types of plants.\n\n### Retraining work-flow\n\nActivate the virtual environment.\n\n```bash\nmodule load python/3.6.0\nsource image-download/bin/activate\n```\n\nRetrain ImageNet.\n\n```bash\npython3 imageClassifierModel/retrain.py \\\n  --image_dir=/n/scratch2/jc604_plantimages \\\n  --output_graph=imageClassifierModel/tf_succulent_classifier.pb \\\n  --output_labels=imageClassifierModel/tf_output_labels.txt \\\n  --summaries_dir=imageClassifierModel/tf_summaries \\\n  --output_layer=plant_classifier \\\n  --random_brightness=5\n```\n\nTest on some images.\n\n```bash\npython label_image.py \\\n    --graph=imageClassifierModel/tf_succulent_classifier.pb \\\n    --labels=imageClassifierModel/tf_output_labels.txt \\\n    --input_layer=Placeholder \\\n    --output_layer=plant_classifier \\\n    --image=imageClassifierModel/my_plant_images/Euphorbia obesa_5.JPG\n```\n\nConvert to CoreML format. (**untested**)\n\n```python\nimport tfcoreml as tf_converter\n\ntf_converter.convert(tf_model_path='my_model.pb',\n                     mlmodel_path='my_model.mlmodel',\n                     output_feature_names=['softmax'],\n                     input_name_shape_dict={'input': [1, 227, 227, 3]},\n                     use_coreml_3=True)\n```\n\n---\n\n## Notes\n\n- [Meghan Kane - Bootstrapping the Machine Learning Training Process](https://www.youtube.com/watch?v=ugiPfm8ICZo)\n- There are models already available from Apple: https://developer.apple.com/machine-learning/models/\n- use \"transfer learning\" to use knowledge learned from source task (eg. MobileNet or SqueezeNet) to train target task\n- \"tensorboard\" to track learning for the TensorFlow training\n\n\n## Image sources\n\n* http://www.cacti.co.nz/library/\n* https://worldofsucculents.com/browse-succulents-scientific-name\n\n\n## Code Sources\n\n* [Google Images Download](https://github.com/hardikvasa/google-images-download) python library (can `pip` install)\n* [googliser](https://github.com/teracow/googliser)\n\n## For CoreML3\n\n* [Integrating a Core ML Model into Your App](https://developer.apple.com/documentation/coreml/integrating_a_core_ml_model_into_your_app)\n* ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhrcook%2Fcoremldemoapp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjhrcook%2Fcoremldemoapp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhrcook%2Fcoremldemoapp/lists"}