{"id":14958705,"url":"https://github.com/tensorflow/tfjs-wechat","last_synced_at":"2025-05-15T12:04:14.414Z","repository":{"id":39543338,"uuid":"185839852","full_name":"tensorflow/tfjs-wechat","owner":"tensorflow","description":"WeChat Mini-program plugin for TensorFlow.js","archived":false,"fork":false,"pushed_at":"2025-03-23T07:36:23.000Z","size":2138,"stargazers_count":566,"open_issues_count":44,"forks_count":105,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-05-15T00:14:22.111Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://tensorflow.org/js","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tensorflow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-09T17:00:58.000Z","updated_at":"2025-05-12T06:26:25.000Z","dependencies_parsed_at":"2024-11-29T20:01:51.806Z","dependency_job_id":"cdcafe8f-9675-4b53-9b65-14f9ea5588e9","html_url":"https://github.com/tensorflow/tfjs-wechat","commit_stats":{"total_commits":82,"total_committers":7,"mean_commits":"11.714285714285714","dds":0.2682926829268293,"last_synced_commit":"61902723d83697eb56e99744bbedb54a6aabe59c"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Ftfjs-wechat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Ftfjs-wechat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Ftfjs-wechat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Ftfjs-wechat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tensorflow","download_url":"https://codeload.github.com/tensorflow/tfjs-wechat/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254337612,"owners_count":22054253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T13:17:53.067Z","updated_at":"2025-05-15T12:04:09.402Z","avatar_url":"https://github.com/tensorflow.png","language":"TypeScript","readme":"# TensorFlow.js 微信小程序插件\n[TensorFlow.js](https://github.com/tensorflow/tfjs)是谷歌开发的机器学习开源项目，致力于为javascript提供具有硬件加速的机器学习模型训练和部署。\nTensorFlow.js 微信小程序插件封装了TensorFlow.js库，用于提供给第三方小程序调用。\n例子可以看TFJS Mobilenet [物体识别小程序](https://github.com/tensorflow/tfjs-wechat/tree/master/demo/mobilenet)\n## 添加插件\n在使用插件前，首先要在小程序管理后台的“设置-第三方服务-插件管理”中添加插件。开发者可登录小程序管理后台，通过 appid [wx6afed118d9e81df9] 查找插件并添加。本插件无需申请，添加后可直接使用。\n\n### 引入插件代码包\n使用插件前，使用者要在 app.json 中声明需要使用的插件，例如：\n\n代码示例：\n```\n{\n  ...\n  \"plugins\": {\n    \"tfjsPlugin\": {\n      \"version\": \"0.0.6\",\n      \"provider\": \"wx6afed118d9e81df9\"\n    }\n  }\n  ...\n}\n```\n### 引入TensorFlow.js npm\nTensorFlow.js 最新版本是以npm包的形式发布，小程序需要使用npm或者yarn来载入TensorFlow.js npm包。也可以手动修改 package.json 文件来加入。\n\nTensorFlow.js v2.0 有一个联合包 - @tensorflow/tfjs，包含了六个分npm包：\n- tfjs-core: 基础包\n- tfjs-converter: GraphModel 导入和执行包\n- tfjs-layers: LayersModel 创建，导入和执行包\n- tfjs-backend-webgl: webgl 后端\n- tfjs-backend-cpu: cpu 后端\n- tfjs-data：数据流\n\n\n对于小程序而言，由于有2M的app大小限制，不建议直接使用联合包，而是按照需求加载分包。\n- 如果小程序只需要导入和运行GraphModel模型的的话，建议至少加入tfjs-core， tfjs-converter， tfjs-backend-webgl 和tfjs-backend-cpu包。这样可以尽量减少导入包的大小。\n- 如果需要创建,导入或训练LayersModel模型，需要再加入 tfjs-layers包。\n\n下面的例子是只用到tfjs-core， tfjs-converter，tfjs-backend-webgl 和tfjs-backend-cpu包。代码示例：\n```\n{\n  \"name\": \"yourProject\",\n  \"version\": \"0.0.1\",\n  \"main\": \"dist/index.js\",\n  \"license\": \"Apache-2.0\",\n  \"dependencies\": {\n    \"@tensorflow/tfjs-core\": \"3.5.0\"，\n    \"@tensorflow/tfjs-converter\": \"3.5.0\"，\n    \"@tensorflow/tfjs-backend-webgl\": \"3.5.0\"\n  }\n}\n```\n\n参考小程序npm工具[文档](https://developers.weixin.qq.com/miniprogram/dev/devtools/npm.html)如何编译npm包到小程序中。\n\n__注意__\n请从微信小程序[开发版Nightly Build更新日志](https://developers.weixin.qq.com/miniprogram/dev/devtools/nightly.html)下载最新的微信开发者工具，保证版本号\u003e=v1.02.1907022.\n\n### Polyfill fetch 函数\n如果需要使用tf.loadGraphModel或tf.loadLayersModel API来载入模型，小程序需要按以下流程填充fetch函数：\n\n1. 如果你使用npm, 你可以载入fetch-wechat npm 包\n\n```\n{\n  \"name\": \"yourProject\",\n  \"version\": \"0.0.1\",\n  \"main\": \"dist/index.js\",\n  \"license\": \"Apache-2.0\",\n  \"dependencies\": {\n    \"@tensorflow/tfjs-core\": \"3.5.0\"，\n    \"@tensorflow/tfjs-converter\": \"3.5.0\"，\n    \"@tensorflow/tfjs-backend-webgl\": \"3.5.0\"\n    \"fetch-wechat\": \"0.0.3\"\n  }\n}\n```\n\n2. 也可以直接拷贝以下文件到你的javascript源目录：\nhttps://cdn.jsdelivr.net/npm/fetch-wechat@0.0.3/dist/fetch_wechat.min.js\n\n\n### 在app.js的onLaunch里调用插件configPlugin函数\n\n```\nvar fetchWechat = require('fetch-wechat');\nvar tf = require('@tensorflow/tfjs-core');\nvar webgl = require('@tensorflow/tfjs-backend-webgl');\nvar plugin = requirePlugin('tfjsPlugin');\n//app.js\nApp({\n  onLaunch: function () {\n    plugin.configPlugin({\n      // polyfill fetch function\n      fetchFunc: fetchWechat.fetchFunc(),\n      // inject tfjs runtime\n      tf,\n      // inject webgl backend\n      webgl,\n      // provide webgl canvas\n      canvas: wx.createOffscreenCanvas()\n    });\n  }\n});\n```\n\n### 支持模型localStorage缓存\n采用localStorage缓存可以减少模型下载耗费带宽和时间。由于微信小程序对于localStorage有10MB的限制，这个方法适用于小于10MB的模型。\n步骤如下：\n1. 在app.js中提供localStorageHandler函数.\n\n```\nvar fetchWechat = require('fetch-wechat');\nvar tf = require('@tensorflow/tfjs-core');\nvar plugin = requirePlugin('tfjsPlugin');\n//app.js\nApp({\n  // expose localStorage handler\n  globalData: {localStorageIO: plugin.localStorageIO},\n  ...\n});\n```\n\n2. 在模型加载时加入localStorageHandler逻辑。\n\n```\nconst LOCAL_STORAGE_KEY = 'mobilenet_model';\nexport class MobileNet {\n  private model: tfc.GraphModel;\n  constructor() { }\n\n  async load() {\n\n    const localStorageHandler = getApp().globalData.localStorageIO(LOCAL_STORAGE_KEY);\n    try {\n      this.model = await tfc.loadGraphModel(localStorageHandler);\n    } catch (e) {\n      this.model =\n        await tfc.loadGraphModel(MODEL_URL);\n      this.model.save(localStorageHandler);\n    }\n  }\n\n```\n\n### 支持模型保存为用户文件\n微信也支持保存模型为文件。同localStorage, 微信小程序对于本地文件也有10MB的限制，这个方法适用于小于10MB的模型。由于最终模型是按 binary 保存，较 localstorage 保存为 base64 string 更为节省空间。\n\n步骤如下：\n1. 在app.js中提供 fileStorageHandler 函数.\n\n```js\nvar fetchWechat = require('fetch-wechat');\nvar tf = require('@tensorflow/tfjs-core');\nvar plugin = requirePlugin('tfjsPlugin');\n//app.js\nApp({\n  // expose fileStorage handler\n  globalData: {fileStorageIO: plugin.fileStorageIO},\n  ...\n});\n```\n\n2. 在模型加载时加入 fileStorageHandler 逻辑。\n\n```js\nconst FILE_STORAGE_PATH = 'mobilenet_model';\nexport class MobileNet {\n  private model: tfc.GraphModel;\n  constructor() { }\n\n  async load() {\n\n    const fileStorageHandler = getApp().globalData.fileStorageIO(\n        FILE_STORAGE_PATH, wx.getFileSystemManager());\n    try {\n      this.model = await tfc.loadGraphModel(fileStorageHandler);\n    } catch (e) {\n      this.model =\n        await tfc.loadGraphModel(MODEL_URL);\n      this.model.save(fileStorageHandler);\n    }\n  }\n}\n```\n\n### 使用 WebAssembly backend\n微信小程序在 Android 手机上提供 WebAssembly的支持。TensorFlow.js的WASM backend非常适合在中低端Android手机上使用。\n中低端手机的GPU往往相对CPU要弱一些，而WASM backend是跑在CPU上的，这就为中低端手机提供了另一个加速平台。而且WASM的能耗一般会更低。\n使用WASM backend需要修改package.json文件：\n\n```\n{\n  \"name\": \"yourProject\",\n  \"version\": \"0.0.1\",\n  \"main\": \"dist/index.js\",\n  \"license\": \"Apache-2.0\",\n  \"dependencies\": {\n    \"@tensorflow/tfjs-core\": \"2.0.0\"，\n    \"@tensorflow/tfjs-converter\": \"2.0.0\"，\n    \"@tensorflow/tfjs-backend-wasm\": \"2.0.0\",\n    ...\n  }\n}\n```\n\n然后在app.js中设置 wasm backend, 你可以自行host wasm file以提高下载速度, 下面例子中的 `wasmUrl`可以替代成你host的URL。\n```\n    const info = wx.getSystemInfoSync();\n    const wasmUrl = 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-wasm@2.0.0/wasm-out/tfjs-backend-wasm.wasm';\n    const usePlatformFetch = true;\n    console.log(info.platform);\n    if (info.platform == 'android') {\n      setWasmPath(wasmUrl, usePlatformFetch);\n      tf.setBackend('wasm').then(() =\u003e console.log('set wasm backend'));\n    }\n```\n\n__注意__\nWASM backend is broken due to bundle imcompatible with WeChat npm loader, will update here when it is fixed.\n\n\n__注意__\n由于最新版本的WeChat的OffscreenCanvas会随页面跳转而失效，在app.js的 onLaunch 函数中设置 tfjs 会导致小程序退出或页面跳转之后操作出错。建议在使用tfjs的page的onLoad中调用 configPlugin 函数。\nWeChat的12月版本会修复这个问题。\n\n```\nvar fetchWechat = require('fetch-wechat');\nvar tf = require('@tensorflow/tfjs-core');\nvar plugin = requirePlugin('tfjsPlugin');\n//index.js\nPage({\n  onLoad: function () {\n    plugin.configPlugin({\n      // polyfill fetch function\n      fetchFunc: fetchWechat.fetchFunc(),\n      // inject tfjs runtime\n      tf,\n      // provide webgl canvas\n      canvas: wx.createOffscreenCanvas(),\n      backendName: 'wechat-webgl-' + Date.now()\n    });\n    ...\n  }\n});\n```\n\n组件设置完毕就可以开始使用 TensorFlow.js库的[API](https://js.tensorflow.org/api/latest/)了。\n\n### 使用 [tfjs-models](https://github.com/tensorflow/tfjs-models) 模型库注意事项\n模型库提供了一系列训练好的模型，方便大家快速的给小程序注入ML功能。模型分类包括\n- 图像识别\n- 语音识别\n- 人体姿态识别\n- 物体识别\n- 文字分类\n\n由于这些API默认模型文件都存储在谷歌云上，直接使用会导致中国用户无法直接读取。在小程序内使用模型API时要提供 modelUrl 的参数，可以指向我们在谷歌中国的镜像服务器。\n谷歌云的base url是 https://storage.googleapis.com， 中国镜像的base url是https://www.gstaticcnapps.cn\n模型的url path是一致的，比如\n- posenet模型的谷歌云地址是：\nhttps://storage.googleapis.com/tfjs-models/savedmodel/posenet/mobilenet/float/050/model-stride16.json\n- 中国镜像的地址为 https://www.gstaticcnapps.cn/tfjs-models/savedmodel/posenet/mobilenet/float/050/model-stride16.json\n\n他们的 URL Path 都是 /tfjs-models/savedmodel/posenet/mobilenet/float/050/model-stride16.json\n\n下面是加载posenet模型的例子：\n\n```\nimport * as posenet from '@tensorflow-models/posenet';\n\nconst POSENET_URL =\n    'https://www.gstaticcnapps.cn/tfjs-models/savedmodel/posenet/mobilenet/float/050/model-stride16.json';\n\nconst model = await posenet.load({\n  architecture: 'MobileNetV1',\n  outputStride: 16,\n  inputResolution: 193,\n  multiplier: 0.5,\n  modelUrl: POSENET_URL\n});\n```\n\n## [tfjs-examples](https://github.com/tensorflow/tfjs-examples) tfjs例子库\ntfjs API 使用实例。\n\n## 版本需求\n- 微信基础库版本 \u003e= 2.7.3\n- 微信开发者工具 \u003e= v1.02.1907022\n- tfjs-core \u003e= 1.5.2\n- tfjs-converter \u003e= 1.5.2 如果使用localStorage模型缓存\n\n__注意__\n在微信开发者工具 v1.02.19070300 中，你需要在通用设置中打开硬件加速，从而在TensorFlow.js中启用WebGL加速。\n![setting](https://raw.githubusercontent.com/tensorflow/tfjs-wechat/master/doc/setting.png)\n## 更新说明\n- 0.0.2 plugin不再映射TensorFlow.js API库，由小程序端提供。\n- 0.0.3 使用offscreen canvas，小程序无需加入plugin component。\n- 0.0.5 修改例子程序使用tfjs分包来降低小程序大小。\n- 0.0.6 支持 tfjs-core版本1.2.7。\n- 0.0.7 允许用户设置webgl backend name, 这可以解决小程序offscreen canvas会失效的问题。\n- 0.0.8 加入localStorage支持，允许小于10M模型在localStorage内缓存。\n- 0.0.9 加入fileSystem支持，允许小于10M模型在local file system内缓存。fixed missing kernel bug.\n- 0.1.0 支持 tfjs版本2.0.x。\n- 0.2.0 支持 tfjs版本3.x。\n","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftensorflow%2Ftfjs-wechat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftensorflow%2Ftfjs-wechat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftensorflow%2Ftfjs-wechat/lists"}