{"id":17604273,"url":"https://github.com/marcelo-earth/scanner-cam","last_synced_at":"2025-04-30T10:10:41.011Z","repository":{"id":61253142,"uuid":"538719400","full_name":"marcelo-earth/scanner-cam","owner":"marcelo-earth","description":"📹🔊 Web app recognizer of objects in camera. Also you can hear which objects are detected. And it works offline after first load!","archived":false,"fork":false,"pushed_at":"2023-03-04T17:21:15.000Z","size":5924,"stargazers_count":12,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-17T21:33:52.655Z","etag":null,"topics":["camera","machine-learning","react","react-webcam","single-shot-multibox-detector","ssd","tensorflow-experiments","tensorflow-js"],"latest_commit_sha":null,"homepage":"https://scanner.cam","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcelo-earth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-19T22:29:48.000Z","updated_at":"2024-11-26T05:33:38.000Z","dependencies_parsed_at":"2024-09-22T08:01:27.467Z","dependency_job_id":"24650a05-be6f-4de9-9077-7691bf855fe2","html_url":"https://github.com/marcelo-earth/scanner-cam","commit_stats":{"total_commits":218,"total_committers":2,"mean_commits":109.0,"dds":0.00917431192660545,"last_synced_commit":"f307ebe3c85fab686827619bea0f94166148b187"},"previous_names":["marcelo-earth/scanner-cam"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelo-earth%2Fscanner-cam","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelo-earth%2Fscanner-cam/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelo-earth%2Fscanner-cam/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelo-earth%2Fscanner-cam/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcelo-earth","download_url":"https://codeload.github.com/marcelo-earth/scanner-cam/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235053780,"owners_count":18928442,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["camera","machine-learning","react","react-webcam","single-shot-multibox-detector","ssd","tensorflow-experiments","tensorflow-js"],"created_at":"2024-10-22T14:07:37.319Z","updated_at":"2025-01-22T03:14:07.890Z","avatar_url":"https://github.com/marcelo-earth.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg\n    src=\".github/logo.png\"\n    align=\"center\"\n    width=\"100\"\n    alt=\"Scanner Cam\"\n    title=\"Scanner Cam\"\n  /\u003e\n  \u003ch1 align=\"center\"\u003e📸 Scanner Cam 🔊\u003c/h1\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  📸 Camera object recognizer with object-to-voice integrated. 🤖\u003cbr /\u003e\n  \u003cbr /\u003e\n  ✅ This project has diagrams to help you understand how it works! 📌\u003cbr /\u003e\n\u003c/p\u003e\n\n![Demo of ScannerCam](./.github/demo.gif)\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://scanner.cam\"\u003ewww.scanner.cam\u003c/a\u003e\u003cbr /\u003e\n\u003c/p\u003e\n\n## 🤖 Concept\n\nScannerCam is a web application that uses the [TensorFlow.js](https://www.tensorflow.org/js) computer vision library to detect objects in real time with the camera of a mobile device or a computer. Built with Next.js, React, TypeScript, TailwindCSS, and Playwright.\n\n## 🛠 Core Development\n\n```mermaid\ngraph TD\n    A([User clicks Start app]) --\u003e|Downloading TensorFlow model...| B[Model downloaded]\n    B --\u003e |App can work offline from here...|C[Camera button available]\n    C --\u003e D([User clicks on camera button])\n    D --\u003e E[Is camera access allowed?]\n    E --\u003e |No|C\n    E --\u003e |Yes|G[React-Webcam component rendered]\n    G --\u003e H[Tensorgram mechanism is started...]\n```\n\nThese are the main technologies used to build ScannerCam:\n\n### 🧠 Machine Learning\n\n**ScannerCam** is a web application that contains a camera module (capable of reverting to environment camera and front camera on mobile devices) whose frames are fed in real time to the COCO-SSD model (COCO stands for [Common Objects In Context](https://cocodataset.org/#home)) (SSD stands for [Single Shot MultiBox Detection](https://arxiv.org/abs/1512.02325)) of [TensorFlow.js](https://www.tensorflow.org/js) computer vision to detect up to 80 object classes.\n\n![SSD: Single Shot MultiBox Detector](./.github/ssd-paper.jpg)\n\n\u003e \"We present a method for detecting objects in images using a single deep neural network. Our approach, named SSD, discretizes the output space of bounding boxes into a set of default boxes over different aspect ratios and scales per feature map location.\" From the [SSD: Single Shot MultiBox Detector](https://arxiv.org/abs/1512.02325) paper.\n\nWhen the user clicks the button \"Start app\", the app requests the Machine Learning Model from the TensorFlow.js Hub. The TensorFlow.js Hub then responds by sending the model's architecture and weights to the app.\n\nOnce the model is ready, it is not necessary to download it again. The model is stored in the browser's cache. This way, the app can work offline after the first load.\n\nIf you want to know more about the COCO-SSD model, you can read the [TensorFlow.js documentation](https://www.tensorflow.org/js/models).\n\n```mermaid\ngraph TD\n    A(Tensorgram mechanism) --\u003e B[Get the frames from the HTML video element]\n    B --\u003e |Sent to the TensorFlow.js model|C[(COCO SSD Detection Model)]\n    C --\u003e |Returns|D([Array of detections])\n    D --\u003e E[\\Get classes detected/]\n    E --\u003e |Update useState|G[(Detections Storage)]\n    D --\u003e H[Create Diagrams with createTensorgram]\n    H --\u003e I[Create an overlay element on top of the React-Webcam component]\n    I --\u003e J[Paint boxes with text in the overlay element created]\n```\n\n### 🎙 Speech Synthesis\n\nIt also uses the [Speech Synthesis API](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis) to talk about objects detected on camera while the option is active. This feature is called object-to-voice.\n\nBoth features have language internationalization support in English and Spanish. The language change happens by the user's preferred language in the browser.\n\n### 🪐 User Interface\n\nScannerCam UI was built on [React v18](https://reactjs.org/blog/2022/03/29/react-v18.html) with TypeScript.\n\nIt is responsive. And it's also available with a beautiful light and dark mode based on the user's preferred color scheme in the browser.\n\n### ✅ Testing\n\nScannerCam is continuously tested with [Playwright](https://playwright.dev/). Playwright is a Node.js library to automate Chromium, Firefox, and WebKit with a single API.\n\nMore than 10 assertions are made to ensure that the app is working correctly.\n\nThe tests are located in the [`tests`](./tests) folder.\n\n### 🛩 Deployment\n\nScannerCam is deployed on [Vercel](https://vercel.com). Vercel is a cloud platform for static sites, hybrid apps, and Serverless Functions.\n\n## 🎨 UI Design\n\nScannerCam has a simple and clean design. It is based on the [Material Design](https://material.io/design) guidelines.\n\nThe library used to build the interface is [TailwindCSS](https://tailwindcss.com). It is a utility-first CSS framework for rapidly building custom user interfaces.\n\nThe colors chosen to paint Scanner Cam are shades of red that change depending on the user's preference in dark mode and light mode.\n\n- Red darker: `#6A0012`\n- Red dark: `#A00037`\n- Red candydark: `#D81B60`\n- Red candylight: `#FF5C8D`\n- Red light: `#FF90BD`\n- Red lighter: `#FFC2EF`\n\n## 🤲 Contributing\n\nDo you would like to contribute? Do you want to be the author of a new feature? Awesome! please fork the repository and make changes as you like. [Pull requests](https://github.com/360macky/scanner-cam/pulls) are warmly welcome.\n\n## 📃 License\n\nDistributed under the MIT License.\n\nSee [`LICENSE`](./LICENSE) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcelo-earth%2Fscanner-cam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcelo-earth%2Fscanner-cam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcelo-earth%2Fscanner-cam/lists"}