{"id":25287410,"url":"https://github.com/daviddaytw/react-native-transformers","last_synced_at":"2025-04-06T02:10:20.459Z","repository":{"id":250758420,"uuid":"834391792","full_name":"daviddaytw/react-native-transformers","owner":"daviddaytw","description":"Run local LLM from Huggingface in React-Native or Expo using onnxruntime.","archived":false,"fork":false,"pushed_at":"2025-03-11T20:35:34.000Z","size":1776,"stargazers_count":76,"open_issues_count":5,"forks_count":8,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-29T21:04:55.427Z","etag":null,"topics":["expo","huggingface","local-llm","onnx","onnxruntime","react-native","transformers"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/react-native-transformers","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/daviddaytw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["daviddaytw"]}},"created_at":"2024-07-27T06:09:18.000Z","updated_at":"2025-03-28T18:09:44.000Z","dependencies_parsed_at":"2024-07-30T00:30:30.546Z","dependency_job_id":"114fb8a7-53dc-42be-a031-3e02b8ee95ff","html_url":"https://github.com/daviddaytw/react-native-transformers","commit_stats":null,"previous_names":["daviddaytw/react-native-transformers"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daviddaytw%2Freact-native-transformers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daviddaytw%2Freact-native-transformers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daviddaytw%2Freact-native-transformers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daviddaytw%2Freact-native-transformers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/daviddaytw","download_url":"https://codeload.github.com/daviddaytw/react-native-transformers/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247423515,"owners_count":20936626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["expo","huggingface","local-llm","onnx","onnxruntime","react-native","transformers"],"created_at":"2025-02-12T22:24:45.124Z","updated_at":"2025-04-06T02:10:20.439Z","avatar_url":"https://github.com/daviddaytw.png","language":"TypeScript","funding_links":["https://github.com/sponsors/daviddaytw"],"categories":[],"sub_categories":[],"readme":"# react-native-transformers\n\n![NPM Version](https://img.shields.io/npm/v/react-native-transformers)\n[![codecov](https://codecov.io/github/daviddaytw/react-native-transformers/graph/badge.svg?token=G3D0Y33SI4)](https://codecov.io/github/daviddaytw/react-native-transformers)\n[![TypeDoc](https://github.com/daviddaytw/react-native-transformers/actions/workflows/docs.yml/badge.svg)](https://daviddaytw.github.io/react-native-transformers)\n\n**Run Hugging Face transformer models directly on your React Native and Expo applications with on-device inference. No cloud service required!**\n\n## Overview\n\n`react-native-transformers` empowers your mobile applications with AI capabilities by running transformer models directly on the device. This means your app can generate text, answer questions, and process language without sending data to external servers - enhancing privacy, reducing latency, and enabling offline functionality.\n\nBuilt on top of ONNX Runtime, this library provides a streamlined API for integrating state-of-the-art language models into your React Native and Expo applications with minimal configuration.\n\n## Key Features\n\n- **On-device inference**: Run AI models locally without requiring an internet connection\n- **Privacy-focused**: Keep user data on the device without sending it to external servers\n- **Optimized performance**: Leverages ONNX Runtime for efficient model execution on mobile CPUs\n- **Simple API**: Easy-to-use interface for model loading and inference\n- **Expo compatibility**: Works seamlessly with both Expo managed and bare workflows\n\n## Installation\n\n### 1. Install peer dependencies\n\n```sh\nnpm install onnxruntime-react-native\n```\n\n### 2. Install react-native-transformers\n\n```sh\n# React-Native\nnpm install react-native-transformers\n\n# Expo\nnpx expo install react-native-transformers\n```\n\n### 3. Platform Configuration\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003cb\u003eReact Native CLI\u003c/b\u003e\u003c/summary\u003e\n\nLink the `onnxruntime-react-native` library:\n\n```sh\nnpx react-native link onnxruntime-react-native\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003cb\u003eExpo\u003c/b\u003e\u003c/summary\u003e\n\nAdd the Expo plugin configuration in `app.json` or `app.config.js`:\n\n```json\n{\n  \"expo\": {\n    \"plugins\": [\n      \"onnxruntime-react-native\"\n    ]\n  }\n}\n```\n\u003c/details\u003e\n\n### 4. Babel Configuration\n\nAdd the `babel-plugin-transform-import-meta` plugin to your Babel configuration:\n\n```js\n// babel.config.js\nmodule.exports = {\n  // ... your existing config\n  plugins: [\n    // ... your existing plugins\n    \"babel-plugin-transform-import-meta\"\n  ]\n};\n```\n\nYou can follow this [document](https://docs.expo.dev/versions/latest/config/babel/) to create config file, and you need to run `npx expo start --clear` to clear the Metro bundler cache.\n\n### 5. Development Client Setup\n\nFor development and testing, it's required to use a development client instead of Expo Go due to the native code of ONNX Runtime and react-native-transformers.\n\nYou can set up a development client using one of these methods:\n\n- **[EAS Development Build](https://docs.expo.dev/develop/development-builds/introduction/)**: Create a custom development client using EAS Build\n- **[Expo Prebuild](https://docs.expo.dev/workflow/prebuild/)**: Eject to a bare workflow to access native code\n\n\n## Usage\n\n### Text Generation\n\n```javascript\nimport React, { useState, useEffect } from \"react\";\nimport { View, Text, Button } from \"react-native\";\nimport { Pipeline } from \"react-native-transformers\";\n\nexport default function App() {\n  const [output, setOutput] = useState(\"\");\n  const [isLoading, setIsLoading] = useState(false);\n  const [isModelReady, setIsModelReady] = useState(false);\n\n  // Load model on component mount\n  useEffect(() =\u003e {\n    loadModel();\n  }, []);\n\n  const loadModel = async () =\u003e {\n    setIsLoading(true);\n    try {\n      // Load a small Llama model\n      await Pipeline.TextGeneration.init(\n        \"Felladrin/onnx-Llama-160M-Chat-v1\",\n        \"onnx/decoder_model_merged.onnx\",\n        {\n          // The fetch function is required to download model files\n          fetch: async (url) =\u003e {\n            // In a real app, you might want to cache the downloaded files\n            const response = await fetch(url);\n            return response.url;\n          }\n        }\n      );\n      setIsModelReady(true);\n    } catch (error) {\n      console.error(\"Error loading model:\", error);\n      alert(\"Failed to load model: \" + error.message);\n    } finally {\n      setIsLoading(false);\n    }\n  };\n\n  const generateText = () =\u003e {\n    setOutput(\"\");\n    // Generate text from the prompt and update the UI as tokens are generated\n    Pipeline.TextGeneration.generate(\n      \"Write a short poem about programming:\",\n      (text) =\u003e setOutput(text)\n    );\n  };\n\n  return (\n    \u003cView style={{ padding: 20 }}\u003e\n      \u003cButton\n        title={isModelReady ? \"Generate Text\" : \"Load Model\"}\n        onPress={isModelReady ? generateText : loadModel}\n        disabled={isLoading}\n      /\u003e\n      \u003cText style={{ marginTop: 20 }}\u003e\n        {output || \"Generated text will appear here\"}\n      \u003c/Text\u003e\n    \u003c/View\u003e\n  );\n}\n```\n\n### With Custom Model Download\n\nFor Expo applications, use `expo-file-system` to download models with progress tracking:\n\n```javascript\nimport * as FileSystem from \"expo-file-system\";\nimport { Pipeline } from \"react-native-transformers\";\n\n// In your model loading function\nawait Pipeline.TextGeneration.init(\"model-repo\", \"model-file\", {\n  fetch: async (url) =\u003e {\n    const localPath = FileSystem.cacheDirectory + url.split(\"/\").pop();\n\n    // Check if file already exists\n    const fileInfo = await FileSystem.getInfoAsync(localPath);\n    if (fileInfo.exists) {\n      console.log(\"Model already downloaded, using cached version\");\n      return localPath;\n    }\n\n    // Download file with progress tracking\n    const downloadResumable = FileSystem.createDownloadResumable(\n      url,\n      localPath,\n      {},\n      (progress) =\u003e {\n        const percentComplete = progress.totalBytesWritten / progress.totalBytesExpectedToWrite;\n        console.log(`Download progress: ${(percentComplete * 100).toFixed(1)}%`);\n      }\n    );\n\n    const result = await downloadResumable.downloadAsync();\n    return result?.uri;\n  }\n});\n```\n\n## Supported Models\n\n`react-native-transformers` works with ONNX-formatted models from Hugging Face. Here are some recommended models based on size and performance:\n\n| Model | Type | Size | Description |\n|-------|------|------|-------------|\n| [Felladrin/onnx-Llama-160M-Chat-v1](https://huggingface.co/Felladrin/onnx-Llama-160M-Chat-v1) | Text Generation | ~300MB | Small Llama model (160M parameters) |\n| [microsoft/Phi-3-mini-4k-instruct-onnx-web](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx-web) | Text Generation | ~1.5GB | Microsoft's Phi-3-mini model |\n| [Xenova/distilgpt2_onnx-quantized](https://huggingface.co/Xenova/distilgpt2_onnx-quantized) | Text Generation | ~165MB | Quantized DistilGPT-2 |\n| [Xenova/tiny-mamba-onnx](https://huggingface.co/Xenova/tiny-mamba-onnx) | Text Generation | ~85MB | Tiny Mamba model |\n| [Xenova/all-MiniLM-L6-v2-onnx](https://huggingface.co/Xenova/all-MiniLM-L6-v2-onnx) | Text Embedding | ~80MB | Sentence embedding model |\n\n## API Reference\n\nFor detailed API documentation, please visit our [TypeDoc documentation](https://daviddaytw.github.io/react-native-transformers/).\n\n## Contributing\n\nContributions are welcome! See the [contributing guide](CONTRIBUTING.md) to learn how to contribute to the repository and the development workflow.\n\n\u003ca href=\"https://github.com/daviddaytw/react-native-transformers/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=daviddaytw/react-native-transformers\" /\u003e\n\u003c/a\u003e\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n## Acknowledgements\n\n- [ONNX Runtime](https://onnxruntime.ai/) for efficient model execution on mobile devices\n- [@xenova/transformers](https://www.npmjs.com/package/@xenova/transformers) for transformer model implementations\n- [Hugging Face](https://huggingface.co/) for providing pre-trained models and model hosting\n\n## External Links\n\n- [Expo Plugins Documentation](https://docs.expo.dev/guides/config-plugins/)\n- [ONNX Runtime Documentation](https://onnxruntime.ai/)\n- [Hugging Face Model Hub](https://huggingface.co/models)\n- [ONNX Format Documentation](https://onnx.ai/onnx/intro/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaviddaytw%2Freact-native-transformers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdaviddaytw%2Freact-native-transformers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaviddaytw%2Freact-native-transformers/lists"}