{"id":20309459,"url":"https://github.com/0xtimsb/poser-tensorflow","last_synced_at":"2025-03-04T07:46:40.681Z","repository":{"id":128579642,"uuid":"354725045","full_name":"0xtimsb/poser-tensorflow","owner":"0xtimsb","description":"Image Classification with Tensorflow using Transfer Learning","archived":false,"fork":false,"pushed_at":"2022-07-14T16:55:17.000Z","size":2994,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-02T04:28:30.559Z","etag":null,"topics":["react","teachable-machine","tensorflow","transfer-learning","webcam"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/0xtimsb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-05T05:07:16.000Z","updated_at":"2022-07-14T16:55:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"d1548508-2fc8-49a6-8903-abd5b82623cc","html_url":"https://github.com/0xtimsb/poser-tensorflow","commit_stats":null,"previous_names":["0xtimsb/poser-tensorflow"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xtimsb%2Fposer-tensorflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xtimsb%2Fposer-tensorflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xtimsb%2Fposer-tensorflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xtimsb%2Fposer-tensorflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/0xtimsb","download_url":"https://codeload.github.com/0xtimsb/poser-tensorflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241809603,"owners_count":20023786,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["react","teachable-machine","tensorflow","transfer-learning","webcam"],"created_at":"2024-11-14T17:27:21.626Z","updated_at":"2025-03-04T07:46:40.659Z","avatar_url":"https://github.com/0xtimsb.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Poser-Tensorflow (Project Exhibition 2021)\n\nIn this project, we have tried to develop a system that would be helpful for the disabled people having communication difficulties by developing a system that would help them in expressing themselves clearly and easily. Our model successfully converts the entire input sentence into a single visual rather than depicting different words through a GIF/ picture giving the model a much realistic and lively appeal. Much more development on this track can be done as the ASL dictionary is still small and needs to grow eventually. \n\n\n### via NPM\n\n[NPM Package](https://www.npmjs.com/package/@teachablemachine/image)\n\n```\nnpm i @tensorflow/tfjs\nnpm i @teachablemachine/image\n```\n\n```js\nimport * as tf from '@tensorflow/tfjs';\nimport * as tmImage from '@teachablemachine/image';\n\n```\n\n### Sample snippet\n\n```js\n\u003cdiv\u003eTeachable Machine Image Model\u003c/div\u003e\n\u003cbutton type='button' onclick='init()'\u003eStart\u003c/button\u003e\n\u003cdiv id='webcam-container'\u003e\u003c/div\u003e\n\u003cdiv id='label-container'\u003e\u003c/div\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@1.3.1/dist/tf.min.js\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@teachablemachine/image@0.8.3/dist/teachablemachine-image.min.js\"\u003e\u003c/script\u003e\n\u003cscript type=\"text/javascript\"\u003e\n    // More API functions here:\n    // https://github.com/googlecreativelab/teachablemachine-community/tree/master/libraries/image\n\n    // the link to your model provided by Teachable Machine export panel\n    const URL = '{{URL}}';\n\n    let model, webcam, labelContainer, maxPredictions;\n\n    // Load the image model and setup the webcam\n    async function init() {\n        const modelURL = URL + 'model.json';\n        const metadataURL = URL + 'metadata.json';\n\n        // load the model and metadata\n        // Refer to tmImage.loadFromFiles() in the API to support files from a file picker\n        // or files from your local hard drive\n        model = await tmImage.load(modelURL, metadataURL);\n        maxPredictions = model.getTotalClasses();\n\n        // Convenience function to setup a webcam\n        const flip = true; // whether to flip the webcam\n        webcam = new tmImage.Webcam(200, 200, flip); // width, height, flip\n        await webcam.setup(); // request access to the webcam\n        webcam.play();\n        window.requestAnimationFrame(loop);\n\n        // append elements to the DOM\n        document.getElementById('webcam-container').appendChild(webcam.canvas);\n        labelContainer = document.getElementById('label-container');\n        for (let i = 0; i \u003c maxPredictions; i++) { // and class labels\n            labelContainer.appendChild(document.createElement('div'));\n        }\n    }\n\n    async function loop() {\n        webcam.update(); // update the webcam frame\n        await predict();\n        window.requestAnimationFrame(loop);\n    }\n\n    // run the webcam image through the image model\n    async function predict() {\n        // predict can take in an image, video or canvas html element\n        const prediction = await model.predict(webcam.canvas);\n        for (let i = 0; i \u003c maxPredictions; i++) {\n            const classPrediction =\n                prediction[i].className + ': ' + prediction[i].probability.toFixed(2);\n            labelContainer.childNodes[i].innerHTML = classPrediction;\n        }\n    }\n\u003c/script\u003e\n```\n\n\n## API\n\n### Loading the model - url checkpoints\n\n`tmImage` is the module name, which is automatically included when you use the `\u003cscript src\u003e` method. It gets added as an object to your window so you can access via `window.tmImage` or simply `tmImage`.\n\n```ts\ntmImage.load(\n\tcheckpoint: string, \n\tmetadata?: string | Metadata\n)\n```\n\nArgs:\n\n* **checkpoint**: a URL to a json file that contains the model topology and a reference to a bin file (model weights)\n* **metadata**: a URL to a json file that contains the text labels of your model and additional information\n\n\nUsage:\n\n```js\nawait tmImage.load(checkpointURL, metadataURL);\n```\n\n\n### Loading the model - browser files\n\nYou can upload your model files from a local hard drive by using a file picker and the File interface. \n\n```ts\ntmImage.loadFromFiles(\n\tmodel: File, \n\tweights: File, \n\tmetadata: File\n) \n```\n\nArgs:\n\n* **model**: a File object that contains the model topology (.json)\n* **weights**: a File object with the model weights (.bin)\n* **metadata**: a File object that contains the text labels of your model and additional information (.json)\n\nUsage:\n\n```js\n// you need to create File objects, like with file input elements (\u003cinput type=\"file\" ...\u003e)\nconst uploadModel = document.getElementById('upload-model');\nconst uploadWeights = document.getElementById('upload-weights');\nconst uploadMetadata = document.getElementById('upload-metadata');\nmodel = await tmImage.loadFromFiles(uploadModel.files[0], uploadWeights.files[0], uploadMetadata.files[0])\n```\n\n### Model - get total classes\n\nOnce you have loaded a model, you can obtain the total number of classes in the model. \n\nThis method exists on the model that is loaded from `tmImage.load`.\n\n```ts\nmodel.getTotalClasses()\n```\n\nReturns a number representing the total number of classes\n\n### Model - get class labels\n\nOnce you have loaded a model, you can obtain the class labels (i.e. the name of each category the model was trained on). \n\nThis method exists on the model that is loaded from `tmImage.getClassLabels`.\n\n```ts\nmodel.getClassLabels()\n```\n\nReturns an array with class names as strings.\n\n\n### Model - predict\n\nOnce you have loaded a model, you can make a classificaiton with a couple of different input options.\n\nThis method exists on the model that is loaded from `tmImage.load`.\n\n```ts\nmodel.predict(\n  image: HTMLImageElement | HTMLCanvasElement | HTMLVideoElement | ImageBitmap,\n  flipped = false\n)\n```\n\nArgs:\n\n* **image**: an image, canvas, or video element to make a classification on\n* **flipped**: a boolean to trigger whether to flip on X or not the image input\n\nUsage:\n\n```js\n// predict can take in an image, video or canvas html element\n// if using the webcam utility, we set flip to true since the webcam was only \n// flipped in CSS\nconst flip = true;\nconst allPredictions = await model.predict(webcamElement, flip);\n```\n\n\n\n\n### Model - predictTopK\n\nThis is an alternative function to `predict()` which returns the probability for all classes. \n\nThis method exists on the model that is loaded from `tmImage.load`.\n\n```ts\nmodel.predictTopK(\n  image: HTMLImageElement | HTMLCanvasElement | HTMLVideoElement | ImageBitmap,\n  maxPredictions = 10,\n  flipped = false\n)\n```\n\nArgs:\n\n* **image**: an image, canvas, or video element to make a classification on\n* **flipped**: a boolean to trigger whether to flip on X or not the image input\n* **maxPredictions**: total number of predictions to return\n\nUsage:\n\n```js\n// predictTopK can take in an image, video or canvas html element\n// if using the webcam utility, we set flip to true since the webcam was only \n// flipped in CSS\nconst flip = true;\nconst maxPredictions = model.getTotalClasses();\nconst prediction = await model.predictTopK(webcamElement, maxPredictions, flip);\n```\n\n### Webcam\n\nYou can optionally use a webcam class that comes with the library, or spin up your own webcam. This class exists on the `tmImage` module.\n\nPlease note that the default webcam used in Teachable Machine was flipped on X - so you should probably set `flip = true` if creating your own webcam unless you flipped it manually in Teachable Machine.\n\n```ts\nnew tmImage.Webcam(\n    width = 400,\n    height = 400,\n    flip = false,\n)\n```\n\nArgs:\n\n* **width**: width of the webcam. It should ideally be square since that's how the model was trained with Teachable Machine.\n* **height**: height of the webcam. It should ideally be square since that's how the model was trained with Teachable Machine.\n* **flip**: boolean to signal whether webcam should be flipped on X. Please note this is only flipping on CSS.\n\nUsage:\n\n```js\n// webcam has a square ratio and is flipped by default to match training\nconst webcam = new tmImage.Webcam(200, 200, true);\nawait webcam.setup();\nwebcam.play();\ndocument.body.appendChild(webcam.canvas);\n```\n\n### Webcam - setup\n\nAfter creating a Webcam object you need to call setup just once to set it up.\n\n```ts\nwebcam.setup(\n\toptions: MediaTrackConstraints = {}\n)\n```\n\nArgs:\n\n* **options**: optional media track contraints for the webcam\n\nUsage:\n\n```js\nawait webcam.setup();\n```\n\n### Webcam - play, pause, stop\n\n```ts\nwebcam.play();\nwebcam.pause();\nwebcam.stop();\n```\n\nWebcam play loads and starts playback of a media resource. Returns a promise.\n\n### Webcam - update\n\nCall on update to update the webcam frame.\n\n```ts\nwebcam.update();\n```\n\n### `yarn start`\n\nRuns the app in the development mode.\\\nOpen [http://localhost:3000](http://localhost:3000) to view it in the browser.\n\n### Created By:\nSmit Barmase\n\u003cbr\u003e\nNeel Modi\n\u003cbr\u003e\nAadarsh S\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xtimsb%2Fposer-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F0xtimsb%2Fposer-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xtimsb%2Fposer-tensorflow/lists"}