{"id":15385724,"url":"https://github.com/jgw96/web-ai-toolkit","last_synced_at":"2025-03-17T16:09:44.738Z","repository":{"id":247296870,"uuid":"823507039","full_name":"jgw96/web-ai-toolkit","owner":"jgw96","description":"The Web AI Toolkit simplifies the integration of AI features, such as OCR, speech-to-text, text summarization and more into your application. It ensures data privacy and offline capability by running all AI workloads locally, leveraging WebNN when available, with a fallback to WebGPU.","archived":false,"fork":false,"pushed_at":"2025-01-13T06:13:51.000Z","size":129,"stargazers_count":39,"open_issues_count":2,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-09T20:46:29.254Z","etag":null,"topics":["ai","image-classification","rag","retrieval-augmented-generation","speech-to-text","transformersjs","webai","webgpu","webnn"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jgw96.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-03T07:02:08.000Z","updated_at":"2025-02-27T02:04:54.000Z","dependencies_parsed_at":"2024-07-07T23:26:38.317Z","dependency_job_id":"997362e5-695f-4ccc-ae56-7f8484404f8e","html_url":"https://github.com/jgw96/web-ai-toolkit","commit_stats":{"total_commits":25,"total_committers":2,"mean_commits":12.5,"dds":"0.040000000000000036","last_synced_commit":"003f33c89cb944b31d21f49eb8dd1f39a84bd8f1"},"previous_names":["jgw96/web-ai-toolkit"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jgw96%2Fweb-ai-toolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jgw96%2Fweb-ai-toolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jgw96%2Fweb-ai-toolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jgw96%2Fweb-ai-toolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jgw96","download_url":"https://codeload.github.com/jgw96/web-ai-toolkit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244066179,"owners_count":20392406,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","image-classification","rag","retrieval-augmented-generation","speech-to-text","transformersjs","webai","webgpu","webnn"],"created_at":"2024-10-01T14:45:26.894Z","updated_at":"2025-03-17T16:09:44.731Z","avatar_url":"https://github.com/jgw96.png","language":"TypeScript","funding_links":[],"categories":["ai","Frameworks"],"sub_categories":["Other Demos"],"readme":"\n# Web AI Toolkit\n\nThe Web AI Toolkit simplifies the integration of AI features, such as OCR, speech-to-text, text summarization and more into your application. It ensures data privacy and offline capability by running all AI workloads locally, leveraging WebNN when available, with a fallback to WebGPU.\n\n## Installation\n\nTo install the Web AI Toolkit, run:\n\n```sh\nnpm install web-ai-toolkit\n```\n\n## Available Functions\n\n*Note: Supported hardware is listed in priority of device selection. For example, for transcribing an audio file,\nthe code will attempt to choose the GPU first and then the CPU otherwise.*\n\n| Function Name         | Parameter      | Type                   | Default Value | Supported Hardware |\n|-----------------------|----------------|------------------------|---------------|--------------------|\n| transcribeAudioFile   | audioFile      | Blob                   | -             | GPU / CPU               |\n|                       | model          | string                 | \"Xenova/whisper-tiny\"|                    |\n|                       | timestamps     | boolean                | false         |                    |\n|                       | language       | string                 | \"en-US\"       |                    |\n| textToSpeech          | text           | string                 | -             | GPU / CPU               |\n|                       | model          | string                 | \"Xenova/mms-tts-eng\"|                    |\n| summarize             | text           | string                 | -             | GPU / CPU               |\n|                       | model          | string                 | \"Xenova/distilbart-cnn-6-6\"|                |\n| ocr                   | image          | Blob                   | -             | GPU / CPU               |\n|                       | model          | string                 | \"Xenova/trocr-small-printed\"|                 |\n| classifyImage         | image          | Blob                   | -             | NPU / GPU / CPU               |\n|                       | model          | string                 | \"Xenova/resnet-50\"|                 |\n| doRAGSearch           | texts          | Array\u003cstring\u003e          | []            | GPU\n|                       | query          | string                 | \"\"            |                      |\n\n## Usage\n\nHere are examples of how to use each function:\n\n### RAG (Retrieval-Augmented Generation)\n\n```javascript\nimport { doRAGSearch } from 'web-ai-toolkit';\n\nwindow.showOpenFilePicker().then(async (file) =\u003e {\n    const fileBlob = await file[0].getFile();\n    const text = await fileBlob.text();\n\n    // text can be derived from anything\n    // this sample is just meant to be extremely simple\n    // for example, your text could be an array of text that you have OCR'ed\n    // from some photos\n\n    const query = \"My Search Query\";\n    const ragQuery = await doRAGSearch([text], query);\n    console.log(ragQuery);\n});\n```\n\n### Transcribe Audio File\n\n```javascript\nimport { transcribeAudioFile } from 'web-ai-toolkit';\n\nconst audioFile = ...; // Your audio file Blob\nconst transcription = await transcribeAudioFile(audioFile, \"Xenova/whisper-tiny\", true, \"en-US\");\nconsole.log(transcription);\n```\n\n### Text to Speech\n\n```javascript\nimport { textToSpeech } from 'web-ai-toolkit';\n\nconst text = \"Hello, world!\";\nconst audio = await textToSpeech(text);\nconsole.log(audio);\n```\n\n### Summarize Text\n\n```javascript\nimport { summarize } from 'web-ai-toolkit';\n\nconst text = \"Long text to be summarized...\";\nconst summary = await summarize(text);\nconsole.log(summary);\n```\n\n### Optical Character Recognition (OCR)\n\n```javascript\nimport { ocr } from 'web-ai-toolkit';\n\nconst image = ...; // Your image Blob\nconst text = await ocr(image);\nconsole.log(text);\n```\n\n### Image Classification\n\n```javascript\nimport { classifyImage } from 'web-ai-toolkit';\n\nconst image = ...; // Your image Blob\nconst text = await classifyImage(image);\nconsole.log(text);\n```\n\n## Technical Details\n\nThe Web AI Toolkit utilizes the [transformers.js project](https://huggingface.co/docs/transformers.js/index) to run AI workloads. All AI processing is performed locally on the device, ensuring data privacy and reducing latency. AI workloads are run using the [WebNN API](https://learn.microsoft.com/en-us/windows/ai/directml/webnn-overview) when available, otherwise falling back to the WebGPU API, or even to the CPU with WebAssembly. Choosing the correct hardware to target is handled by the library.\n\n## Contribution\n\nWe welcome contributions to the Web AI Toolkit. Please fork the repository and submit a pull request with your changes. For major changes, please open an issue first to discuss what you would like to change.\n\n## License\n\nThe Web AI Toolkit is licensed under the MIT License. See the [LICENSE](LICENSE) file for more details.\n\n## Contact\n\nFor questions or support, please open an issue here on GitHub\n\n---\n\nThank you for using the Web AI Toolkit! We hope it makes integrating AI into your applications easier and more efficient.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjgw96%2Fweb-ai-toolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjgw96%2Fweb-ai-toolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjgw96%2Fweb-ai-toolkit/lists"}