{"id":18930157,"url":"https://github.com/nheidloff/watson-deep-learning-javascript","last_synced_at":"2026-05-10T03:44:33.114Z","repository":{"id":88375338,"uuid":"137228640","full_name":"nheidloff/watson-deep-learning-javascript","owner":"nheidloff","description":"Deploying Watson Deep Learning Models to Browsers","archived":false,"fork":false,"pushed_at":"2018-06-18T08:24:34.000Z","size":33462,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-12-31T19:30:11.975Z","etag":null,"topics":["ai","deep-learning","dl","ibm-cloud","javascript","tensorflow","tensorflowjs","watson","web"],"latest_commit_sha":null,"homepage":"https://nh-hunt.mybluemix.net/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nheidloff.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-13T14:40:18.000Z","updated_at":"2022-04-27T09:01:54.000Z","dependencies_parsed_at":"2024-03-30T16:00:32.697Z","dependency_job_id":null,"html_url":"https://github.com/nheidloff/watson-deep-learning-javascript","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nheidloff%2Fwatson-deep-learning-javascript","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nheidloff%2Fwatson-deep-learning-javascript/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nheidloff%2Fwatson-deep-learning-javascript/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nheidloff%2Fwatson-deep-learning-javascript/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nheidloff","download_url":"https://codeload.github.com/nheidloff/watson-deep-learning-javascript/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239927825,"owners_count":19719835,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deep-learning","dl","ibm-cloud","javascript","tensorflow","tensorflowjs","watson","web"],"created_at":"2024-11-08T11:36:46.235Z","updated_at":"2026-03-16T19:30:16.924Z","avatar_url":"https://github.com/nheidloff.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deploying Watson Deep Learning Models to Browsers\n\nThis project includes sample code how to train a model with [TensorFlow](https://www.tensorflow.org/) and the [Deep Learning service](https://www.ibm.com/blogs/watson/2018/03/deep-learning-service-ibm-makes-advanced-ai-accessible-users-everywhere/) within Watson Studio and how to deploy and access the model in a web browser.\n\nThis project extends the open source project [Emoji Scavenger Hunt](https://github.com/google/emoji-scavenger-hunt) which is a web based game that makes use of [TensorFlow.js](https://js.tensorflow.org/) to identify objects seen by your webcam or mobile camera in the browser. Emojis are shown and you have to find those objects in the real world before the timer runs out.\n\nThis is a screenshot from the app running on an iPhone where currently a hat is recognized:\n\n![alt text](documentation/iphone-1-small.JPEG \"Screenshot\")\n\nI've deployed a [live demo](https://nh-hunt.mybluemix.net) but it will only work for you if you have items that look similar.\n\nCheck out the [video](https://youtu.be/4WTpMmqraXI) for a quick demo.\n\nIn order to train the model I've taken pictures from seven items: plug, soccer ball, mouse, hat, truck, banana and headphones. Here is how the emojis map to the real objects. You can find the images in the [data](data/images) directory.\n\n![alt text](documentation/items-annotated-small.JPEG \"Photo\")\n\n\n## Prerequisites \n\nGet a free [IBM Cloud](https://ibm.biz/nheidloff) lite account (no time restriction, no credit card required).\n\nCreate an instance of the [Machine Learning](https://console.bluemix.net/catalog/services/machine-learning) service. From the credentials get the user name, password and the instance id.\n\nInstall the IBM Cloud CLI with the machine learning plugin and set environment variables by following these [instructions](https://datascience.ibm.com/docs/content/analyze-data/ml_dlaas_environment.html).\n\nCreate an instance of the [Cloud Object Storage\n](https://console.bluemix.net/catalog/services/cloud-object-storage) service and create HMAC credentials by following these [instructions](https://datascience.ibm.com/docs/content/analyze-data/ml_dlaas_object_store.html). Make sure to use 'Writer' or 'Manager' access and note the aws_access_key_id and aws_secret_access_key for a later step.\n\nInstall and configure the AWS CLI by following these [instructions](https://console.bluemix.net/docs/services/cloud-object-storage/cli/aws-cli.html#use-the-aws-cli).\n\n\n## Training of the Model\n\nClone this repo:\n\n```bash\n$ git clone https://github.com/nheidloff/watson-deep-learning-javascript\n```\n\nCreate two buckets (use unique names):\n\n```bash\n$ aws --endpoint-url=http://s3-api.dal-us-geo.objectstorage.softlayer.net --profile ibm_cos s3 mb s3://nh-hunt-input\n$ aws --endpoint-url=http://s3-api.dal-us-geo.objectstorage.softlayer.net --profile ibm_cos s3 mb s3://nh-hunt-output\n```\n\nDownload and extract Mobilenet:\n\n```bash\n$ cd watson-deep-learning-javascript/data\n$ wget http://download.tensorflow.org/models/mobilenet_v1_2018_02_22/mobilenet_v1_0.25_224.tgz\n$ tar xvzf mobilenet_v1_0.25_224.tgz \n```\n\nUpload bucket with MobileNet and data (use your unique bucket name):\n\n```bash\n$ cd xxx/watson-deep-learning-javascript/data\n$ aws --endpoint-url=http://s3-api.dal-us-geo.objectstorage.softlayer.net --profile ibm_cos s3 cp . s3://nh-hunt-input/ --recursive \n```\n\nPrepare the training:\n* Define your object storage credentials and your bucket names in [tf-train.yaml](model/tf-train.yaml).\n* Compress [retrain.py](model/retrain.py) into [tf-model.zip](model/tf-model.zip).\n\nInvoke the training and check for status (change the generated training name):\n\n```bash\n$ cd xxx/watson-deep-learning-javascript/model\n$ bx ml train tf-model.zip tf-train.yaml\n$ bx ml list training-runs\n$ bx ml monitor training-runs training-5PQK89IiR\n$ bx ml show training-runs training-5PQK89IiR\n```\n\nDownload the saved model:\n\n```bash\n$ cd xxx/watson-deep-learning-javascript/saved-model\n$ aws --endpoint-url=http://s3-api.dal-us-geo.objectstorage.softlayer.net --profile ibm_cos s3 sync s3://nh-hunt-output .\n```\n\nOptionally evaluate the model via Tensorboard (either from Docker container or Virtualenv):\n\n```bash\n$ cd xxx/watson-deep-learning-javascript/saved-model/training-0xebs3Iig\n$ tensorboard --logdir=xxx/watson-deep-learning-javascript/saved-model/training-0xebs3Iig/retrain_logs\n```\n\n\n## Deployment of the Web Application\n\nConvert the model:\n\n```bash\n$ cd xxx/watson-deep-learning-javascript/convert\n$ docker build -t model-converter .\n$ cp -a xxx/watson-deep-learning-javascript/saved-model/training-qBnjUqImR/model/. xxx/watson-deep-learning-javascript/convert/data/saved_model/ \n$ docker run -v xxx/watson-deep-learning-javascript/convert/data:/data -it model-converter\n```\n\nBuild the web application (more [details](https://github.com/google/emoji-scavenger-hunt)):\n\nChange your emojis in [scavenger_classes.ts](emoji-scavenger-hunt/src/js/scavenger_classes.ts) and [game_levels.ts](emoji-scavenger-hunt/src/js/game_levels.ts).\n\n```bash\n$ cp -a xxx/watson-deep-learning-javascript/convert/data/saved_model_web/. xxx/watson-deep-learning-javascript/emoji-scavenger-hunt/dist/model/\n$ cd xxx/watson-deep-learning-javascript/emoji-scavenger-hunt\n$ yarn prep\n$ yarn build\n```\n\nPush the application to IBM Cloud (change host and name in [manifest.yaml](emoji-scavenger-hunt/manifest.yaml) to something unique):\n\n```bash\n$ cd xxx/watson-deep-learning-javascript/emoji-scavenger-hunt\n$ cf login\n$ cf push\n```\n\nAfter this you can open the application via URLs like [https://nh-hunt.mybluemix.net](https://nh-hunt.mybluemix.net).\n\n\n## Deployment of the Model to Watson Studio\n\nDeploy the model (change training id and model id):\n\n```bash\n$ bx ml store training-runs training-qBnjUqImR\n$ bx ml deploy 0c78b7d6-9d22-4719-90da-ab649c0edc90 \"my-deployment\"\n```\n\nGenerate payloads for predictions:\n\n```bash\n$ cd xxx/watson-deep-learning-javascript/predict\n$ docker build -t generate-payload .\n$ docker run -v xxx/watson-deep-learning-javascript/predict:/data -it -e file_name=ball.JPG generate-payload\n```\n\nCopy model id, deployment id and [raw-payload.json](predict/raw-payload.json) in [payload.json](predict/payload.json).\n\nPredict something for a test image: \n\n```bash\n$ cd xxx/watson-deep-learning-javascript/predict\n$ bx ml score payload.json\n```\n\nTo interpret the result, check out [output_labels.txt](saved-model/training-qBnjUqImR/output_lables.txt) for the labels and the order of labels.\n\nAs alternative to the IBM Cloud CLI you can also use curl. See the [API documentation](https://watson-ml-api.mybluemix.net/#!/Deployments/post_v3_wml_instances_instance_id_published_models_published_model_id_deployments_deployment_id_online) for details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnheidloff%2Fwatson-deep-learning-javascript","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnheidloff%2Fwatson-deep-learning-javascript","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnheidloff%2Fwatson-deep-learning-javascript/lists"}