{"id":13958442,"url":"https://github.com/hugozanini/realtime-semantic-segmentation","last_synced_at":"2025-07-21T00:30:51.043Z","repository":{"id":185315414,"uuid":"268330909","full_name":"hugozanini/realtime-semantic-segmentation","owner":"hugozanini","description":"Implementation of RefineNet to perform real time instance segmentation in the browser using TensorFlow.js","archived":false,"fork":false,"pushed_at":"2022-12-13T03:15:39.000Z","size":22938,"stargazers_count":122,"open_issues_count":7,"forks_count":26,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-11-28T02:34:44.046Z","etag":null,"topics":["computer-vision","real-time-machine-learning","semantic-segmentation","tensorflow","tensorflowjs"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hugozanini.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-05-31T17:37:00.000Z","updated_at":"2024-11-25T13:35:56.000Z","dependencies_parsed_at":"2023-08-01T13:45:37.350Z","dependency_job_id":null,"html_url":"https://github.com/hugozanini/realtime-semantic-segmentation","commit_stats":null,"previous_names":["hugozanini/realtime-semantic-segmentation"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hugozanini/realtime-semantic-segmentation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hugozanini%2Frealtime-semantic-segmentation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hugozanini%2Frealtime-semantic-segmentation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hugozanini%2Frealtime-semantic-segmentation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hugozanini%2Frealtime-semantic-segmentation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hugozanini","download_url":"https://codeload.github.com/hugozanini/realtime-semantic-segmentation/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hugozanini%2Frealtime-semantic-segmentation/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266221247,"owners_count":23894964,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","real-time-machine-learning","semantic-segmentation","tensorflow","tensorflowjs"],"created_at":"2024-08-08T13:01:35.695Z","updated_at":"2025-07-21T00:30:49.725Z","avatar_url":"https://github.com/hugozanini.png","language":"JavaScript","funding_links":[],"categories":["其他_机器视觉","JavaScript"],"sub_categories":["网络服务_其他"],"readme":"\r\n\r\n\r\n# Realtime Semantic Segmentation in the browser\r\n\r\n To a more detailed description about this project, [check my article](https://lnkd.in/dFm95MJ) at Towards Data Science.\r\n\r\nThis project uses TensorFlow.js to perform realtime instance segmentation in the browser using [RefineNet](https://arxiv.org/pdf/1611.06612.pdf). Many thanks to  [Bernardo de Oliveira](https://github.com/bernieOllie), that help me to fix some bugs in the code, and [Vladimir Nekrasov](https://drsleep.github.io/) that provided the *.js weights* of the [Light-Weight RefineNet](http://bmvc2018.org/contents/papers/0494.pdf) - a more compact implementation of RefineNet, suitable for tasks requiring real-time performance.\r\n\r\n  \r\n  ![enter image description here](./git_media/objects_demo.gif)\r\n  \r\n  \r\n\r\n### Getting started\r\nTensorflow.js models have to be served through an URL to be loaded in javascript. The  Refine Net model is based on a main JSON file and shared weights files in a binary format. The idea is to create an HTTP server that will make the model available in a URL allowing requests and be treated as REST API. When loading the model, the TensorFlow.js will do the following requests:\r\n\r\n    GET /model.json\r\n    GET /group1-shard1of4\r\n    GET /group1-shard2of4\r\n    GET /group1-shard3of4\r\n    GET /group1-shard4of4\r\n\r\nThe [http-server](https://www.npmjs.com/package/http-server) can be installed using the following command:\r\n\r\n`npm install http-server -g`\r\n\r\nOnce you have the program installed, go to the weights folder, run the command below and the model you be available at http://127.0.0.1:8080.\r\n\r\n`http-server -c1 --cors .`\r\n\r\nTo install the required packages, it can be used `npm` or `yarn`. \r\n\r\nIf you are using  `yarn`:\r\n\r\n\tyarn\r\n\tyarn start\r\n\r\nIf you are using  `npm`:\r\n\r\n\tnpm install\r\n\tnpm start\r\n\r\nTo test the project just go to  http://localhost:3000/ and the application will be running there.\r\n\r\n### References\r\n[1] Romera-Paredes, Bernardino, and Philip Hilaire Sean Torr. “Recurrent instance segmentation.” _European conference on computer vision_. Springer, Cham, 2016.\r\n\r\n[2] Lin, Guosheng, et al. “Refinenet: Multi-path refinement networks for high-resolution semantic segmentation.” _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2017.\r\n\r\n[3] Nekrasov, Vladimir, Chunhua Shen, and Ian Reid. “Light-weight refinenet for real-time semantic segmentation.” _arXiv preprint arXiv:1810.03272_ (2018).\r\n\r\n[4] Medium. 2020. _Introducing Tensorflow.Js: Machine Learning In Javascript_. [online] Available at: \u003chttps://medium.com/tensorflow/introducing-tensorflow-js-machine-learning-in-javascript-bf3eab376db\u003e [Accessed 3 June 2020].\r\n\r\n[5] [https://www.tensorflow.org/js/tutorials/conversion/import_keras](https://www.tensorflow.org/js/tutorials/conversion/import_keras)\r\n\r\n[6] [https://github.com/DrSleep/light-weight-refinenet](https://github.com/DrSleep/light-weight-refinenet)\r\n\r\n[7] [https://www.tensorflow.org/js/guide/conversion](https://www.tensorflow.org/js/guide/conversion)\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhugozanini%2Frealtime-semantic-segmentation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhugozanini%2Frealtime-semantic-segmentation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhugozanini%2Frealtime-semantic-segmentation/lists"}