{"id":51057467,"url":"https://github.com/rbayuokt/expo-mlkit-ocr","last_synced_at":"2026-06-22T22:32:48.239Z","repository":{"id":355060744,"uuid":"1226433212","full_name":"rbayuokt/expo-mlkit-ocr","owner":"rbayuokt","description":"Production-ready Expo Module for on-device text recognition (OCR) using Google ML Kit Text Recognition v2 + OCRTextOverlay","archived":false,"fork":false,"pushed_at":"2026-05-01T17:04:48.000Z","size":3724,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-01T18:15:37.160Z","etag":null,"topics":["expo","expo-mlkit","expo-mlkit-ocr","expo-ocr","expo-text-recognition","mlkit","ocr","react-native","react-native-ocr","text-recognition"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/expo-mlkit-ocr","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rbayuokt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-01T11:47:20.000Z","updated_at":"2026-05-01T17:49:16.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/rbayuokt/expo-mlkit-ocr","commit_stats":null,"previous_names":["rbayuokt/expo-mlkit-ocr"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/rbayuokt/expo-mlkit-ocr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbayuokt%2Fexpo-mlkit-ocr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbayuokt%2Fexpo-mlkit-ocr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbayuokt%2Fexpo-mlkit-ocr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbayuokt%2Fexpo-mlkit-ocr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rbayuokt","download_url":"https://codeload.github.com/rbayuokt/expo-mlkit-ocr/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbayuokt%2Fexpo-mlkit-ocr/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34668500,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-22T02:00:06.391Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["expo","expo-mlkit","expo-mlkit-ocr","expo-ocr","expo-text-recognition","mlkit","ocr","react-native","react-native-ocr","text-recognition"],"created_at":"2026-06-22T22:32:47.633Z","updated_at":"2026-06-22T22:32:48.233Z","avatar_url":"https://github.com/rbayuokt.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"![expo-mlkit-ocr](docs/expo-mlkit-ocr.png)\n\n# expo-mlkit-ocr\n\nProduction-ready Expo Module for on-device text recognition (OCR) using **Google ML Kit Text Recognition v2** for both iOS and Android.\n\n## Preview\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/android.png\" height=\"500\" alt=\"Android Preview\" /\u003e\n  \u003cimg src=\"docs/ios.png\" height=\"500\" alt=\"iOS Preview\" /\u003e\n\u003c/div\u003e\n\n## Features\n\n- ✅ **On-device OCR** — no network requests required\n- ✅ **ML Kit v2** — official Google ML Kit standalone (not Firebase ML Kit)\n- ✅ **Structured output** — blocks, lines, and elements with bounding boxes\n- ✅ **iOS \u0026 Android** — native implementations using Expo Modules API\n- ✅ **TypeScript support** — fully typed API\n- ✅ **Expo Config Plugin** — automatic native setup\n- ✅ **Device support check** — `isSupported()` to detect device compatibility before OCR\n- ✅ **Interactive overlay** — `\u003cOCRTextOverlay\u003e` component for visualizing \u0026 selecting recognized text\n\n## Installation\n\n```bash\nnpx expo install expo-mlkit-ocr expo-image-picker expo-build-properties\n```\n\n\u003e ⚠️ **`expo-build-properties` is REQUIRED** to set the iOS deployment target to `16.0` (needed by Google ML Kit on iOS).\n\n## Setup\n\n### 1. Configure `app.json` / `app.config.js`\n\nAdd **both plugins** to your config:\n\n```json\n{\n  \"expo\": {\n    \"plugins\": [\n      [\n        \"expo-mlkit-ocr\",\n        {\n          \"iosEngine\": \"auto\"\n        }\n      ],\n      [\n        \"expo-build-properties\",\n        {\n          \"ios\": {\n            \"deploymentTarget\": \"16.0\"\n          }\n        }\n      ]\n    ]\n  }\n}\n```\n\n\u003e ⚠️ **Important:** Without `deploymentTarget: \"16.0\"` and `useFrameworks: \"static\"`, you will get:\n\u003e ```\n\u003e ERROR [runtime not ready]: Error: Cannot find native module 'ExpoMlkitOcr'\n\u003e ```\n\n### 2. Build Your App\n\n**With EAS Build:**\n\n```bash\neas build --platform ios\neas build --platform android\n```\n\n**Or with local prebuild:**\n\n```bash\nnpx expo prebuild --clean\nnpx expo run:ios\n# or\nnpx expo run:android\n```\n\n\u003e ❌ **Will NOT work in Expo Go** — this is a custom native module. You need a development client or EAS Build.\n\n### Troubleshooting\n\n**`Cannot find native module 'ExpoMlkitOcr'`**\n- Did you add `expo-build-properties` plugin with `deploymentTarget: \"16.0\"`? ⚠️ Most common cause\n- Did you run `npx expo prebuild --clean` after installing?\n- Are you running on a development client (not Expo Go)?\n\n**Build fails on iOS Simulator (arm64)**\n- Use `iosEngine: \"auto\"` or `\"vision\"` in plugin config (uses Apple Vision instead of ML Kit)\n\n## Usage\n\n### Basic Example\n\n```typescript\nimport { recognizeText } from 'expo-mlkit-ocr';\nimport * as ImagePicker from 'expo-image-picker';\n\nasync function pickAndRecognize() {\n  const result = await ImagePicker.launchImageLibraryAsync({\n    mediaTypes: ['images'],\n  });\n\n  if (result.canceled || !result.assets[0]) return;\n\n  try {\n    const recognition = await recognizeText(result.assets[0].uri);\n    console.log('Recognized text:', recognition.text);\n    console.log('Blocks:', recognition.blocks);\n  } catch (error) {\n    console.error('Recognition failed:', error);\n  }\n}\n```\n\n### Output Format\n\nThe function returns a `RecognitionResult` object with this structure:\n\n```typescript\nexport type RecognitionResult = {\n  text: string; // Full recognized text\n  blocks: TextBlock[];\n};\n\nexport type TextBlock = {\n  text: string;\n  boundingBox: {\n    x: number;\n    y: number;\n    width: number;\n    height: number;\n  };\n  lines: TextLine[];\n};\n\nexport type TextLine = {\n  text: string;\n  boundingBox: {\n    x: number;\n    y: number;\n    width: number;\n    height: number;\n  };\n  elements: TextElement[];\n};\n\nexport type TextElement = {\n  text: string;\n  boundingBox: {\n    x: number;\n    y: number;\n    width: number;\n    height: number;\n  };\n};\n```\n\n**Bounding box coordinates** are in the image's native coordinate system (top-left origin):\n- `x`, `y` — top-left corner\n- `width`, `height` — dimensions in pixels\n\n### Example Output\n\n```json\n{\n  \"text\": \"Hello World\\nExample Text\",\n  \"blocks\": [\n    {\n      \"text\": \"Hello World\",\n      \"boundingBox\": { \"x\": 100, \"y\": 50, \"width\": 200, \"height\": 50 },\n      \"lines\": [\n        {\n          \"text\": \"Hello World\",\n          \"boundingBox\": { \"x\": 100, \"y\": 50, \"width\": 200, \"height\": 50 },\n          \"elements\": [\n            {\n              \"text\": \"Hello\",\n              \"boundingBox\": { \"x\": 100, \"y\": 50, \"width\": 80, \"height\": 50 }\n            },\n            {\n              \"text\": \"World\",\n              \"boundingBox\": { \"x\": 190, \"y\": 50, \"width\": 110, \"height\": 50 }\n            }\n          ]\n        }\n      ]\n    }\n  ]\n}\n```\n\n## API Reference\n\n### `recognizeText(uri: string): Promise\u003cRecognitionResult\u003e`\n\nRecognizes text from an image at the provided URI.\n\n**Parameters:**\n- `uri` (string) — file URI or content URI to the image (e.g., from `expo-image-picker`)\n\n**Returns:**\n- `Promise\u003cRecognitionResult\u003e` — structured text recognition result\n\n**Errors:**\n- `INVALID_URI` — provided URI is not valid\n- `IMAGE_LOAD_FAILED` — image could not be loaded from the URI\n- `RECOGNITION_FAILED` — text recognition failed (rare)\n\n### `isSupported(): boolean`\n\nChecks if the device supports ML Kit text recognition (OCR).\n\nReturns `true` if the device meets the minimum OS requirements; `false` otherwise. Call this before attempting `recognizeText()` to provide graceful fallback UI for unsupported devices.\n\n**Minimum OS versions:**\n- **iOS**: 16.0+\n- **Android**: API 21+ (Android 5.0)\n\n**Example:**\n\n```typescript\nimport { isSupported, recognizeText } from 'expo-mlkit-ocr';\n\nexport default function App() {\n  if (!isSupported()) {\n    return \u003cText\u003eOCR is not supported on this device.\u003c/Text\u003e;\n  }\n\n  return (\n    \u003cButton\n      title=\"Pick Image \u0026 Recognize\"\n      onPress={async () =\u003e {\n        const result = await recognizeText(imageUri);\n        console.log(result.text);\n      }}\n    /\u003e\n  );\n}\n```\n\n### `\u003cOCRTextOverlay /\u003e` — Interactive Bounding Box Overlay\n\nRenders interactive bounding boxes over an image to visualize OCR results. Tap boxes to select text and trigger a callback (e.g., copy to clipboard).\n\n**Usage:**\n\n```typescript\nimport { recognizeText, OCRTextOverlay } from 'expo-mlkit-ocr';\nimport { Image, Clipboard } from 'react-native';\nimport { useState } from 'react';\n\nexport default function App() {\n  const [result, setResult] = useState(null);\n\n  async function pickAndRecognize() {\n    const picked = await ImagePicker.launchImageLibraryAsync({ mediaTypes: ['images'] });\n    if (!picked.assets[0]) return;\n\n    const asset = picked.assets[0];\n    const ocrResult = await recognizeText(asset.uri);\n    setResult(ocrResult);\n  }\n\n  return (\n    \u003c\u003e\n      \u003cButton title=\"Pick \u0026 Recognize\" onPress={pickAndRecognize} /\u003e\n\n      {result \u0026\u0026 (\n        \u003cOCRTextOverlay\n          result={result}\n          imageWidth={picked.assets[0].width}\n          imageHeight={picked.assets[0].height}\n          highlightLevel=\"line\"\n          onSelect={(item) =\u003e Clipboard.setString(item.text)}\n        \u003e\n          \u003cImage source={{ uri: picked.assets[0].uri }} style={{ width: 300, height: 400 }} /\u003e\n        \u003c/OCRTextOverlay\u003e\n      )}\n    \u003c/\u003e\n  );\n}\n```\n\n**Props:**\n\n| Prop | Type | Default | Description |\n|------|------|---------|-------------|\n| `result` | `RecognitionResult` | — | OCR result from `recognizeText()` |\n| `imageWidth` | `number` | — | Native image width (pixels) |\n| `imageHeight` | `number` | — | Native image height (pixels) |\n| `children` | `ReactNode` | — | Image component to wrap |\n| `highlightLevel` | `'block' \\| 'line' \\| 'element'` | `'line'` | Which level to highlight |\n| `resizeMode` | `'contain' \\| 'cover'` | `'contain'` | Image resize behavior |\n| `boxColor` | `string` | `'#00BFFF'` | Box border \u0026 fill color (hex) |\n| `selectedBoxColor` | `string` | `'#FF6347'` | Color when a box is selected |\n| `boxOpacity` | `number` | `0.25` | Fill opacity (0–1) |\n| `strokeWidth` | `number` | `2` | Border width (pixels) |\n| `cornerRadius` | `number` | `4` | Rounded corner radius (pixels) |\n| `multiSelect` | `boolean` | `true` | Allow selecting multiple boxes |\n| `onSelect` | `(item) =\u003e void` | — | Callback when box(es) are tapped (single item or array) |\n| `style` | `ViewStyle` | — | Optional wrapper style |\n\n**Multi-Select Example:**\n\n```typescript\n// Single selection (one box at a time)\n\u003cOCRTextOverlay\n  result={result}\n  imageWidth={1920}\n  imageHeight={1080}\n  multiSelect={false}  // Only one selection\n  onSelect={(item) =\u003e console.log('Selected:', item.text)}\n\u003e\n  \u003cImage source={{ uri }} /\u003e\n\u003c/OCRTextOverlay\u003e\n\n// Multiple selection (tap multiple boxes)\n\u003cOCRTextOverlay\n  result={result}\n  imageWidth={1920}\n  imageHeight={1080}\n  multiSelect={true}  // Default - allow multiple selections\n  onSelect={(items) =\u003e {\n    if (Array.isArray(items)) {\n      console.log('Selected items:', items.map(i =\u003e i.text).join(', '));\n    } else {\n      console.log('Single item:', items.text);\n    }\n  }}\n\u003e\n  \u003cImage source={{ uri }} /\u003e\n\u003c/OCRTextOverlay\u003e\n```\n\n**Highlights:**\n- ✅ Pure React Native (no external canvas library)\n- ✅ Tap to toggle selection + visual highlight\n- ✅ Multi-select mode: tap multiple boxes, all stay highlighted\n- ✅ Single-select mode: only one box highlighted at a time\n- ✅ Works with any `\u003cImage\u003e` component (react-native, expo-image, etc.)\n- ✅ Automatic coordinate scaling for `contain` and `cover` resize modes\n- ✅ Full TypeScript support\n\n## Common Errors\n\n### `Error: expo-mlkit-ocr is not supported on web.`\n\nThe module only works on iOS and Android. For web support, use a third-party OCR service (e.g., Tesseract.js).\n\n```typescript\nimport { Platform } from 'react-native';\nimport { recognizeText } from 'expo-mlkit-ocr';\n\nif (Platform.OS !== 'web') {\n  const result = await recognizeText(uri);\n} else {\n  // Use a web-based OCR service\n}\n```\n\n### `IMAGE_LOAD_FAILED`\n\n- Ensure the URI is valid and the file exists\n- Use URIs from `expo-image-picker` or `expo-camera` which are guaranteed to work\n- On Android, both `file://` and `content://` URIs are supported\n\n### iOS Simulator (arm64) + iOS 26.x\n\nGoogle ML Kit CocoaPods binaries exclude the `ios-arm64-simulator` slice, so arm64-only simulator runtimes (for example iOS 26.x on Apple Silicon) can’t link ML Kit frameworks and will fail at build/link time.\n\nTo keep development on simulator unblocked, `expo-mlkit-ocr` supports **Apple Vision** as a fallback OCR engine on iOS. The **JavaScript response shape stays the same** (`text`, `blocks`, `lines`, `elements`, `boundingBox`); only the underlying OCR engine changes.\n\n```json\n{\n  \"expo\": {\n    \"plugins\": [\n      [\"expo-mlkit-ocr\", { \"iosEngine\": \"vision\" }]\n    ]\n  }\n}\n```\n\nSupported `iosEngine` values:\n- `\"auto\"` (default): use Apple Vision (simulator-friendly default)\n- `\"mlkit\"`: use Google ML Kit (won’t build on arm64-only iOS Simulator runtimes)\n- `\"vision\"`: always use Apple Vision (disables ML Kit pods)\n\nLegacy option (equivalent to `iosEngine: \"vision\"`):\n- `disableMlkitOnSimulator: true`\n\n## Development\n\n### Project Structure\n\n```\nexpo-mlkit-ocr/\n├── src/                          # TypeScript source\n│   ├── index.ts\n│   ├── ExpoMlkitOcr.types.ts\n│   ├── ExpoMlkitOcrModule.ts\n│   ├── ExpoMlkitOcrModule.web.ts\n│   └── OCRTextOverlay.tsx        # Interactive overlay component\n├── ios/\n│   ├── ExpoMlkitOcrModule.swift   # ML Kit integration\n│   └── ExpoMlkitOcr.podspec\n├── android/\n│   ├── build.gradle\n│   └── src/main/java/.../ExpoMlkitOcrModule.kt\n├── app.plugin.js                 # Expo config plugin entry\n├── plugins/\n│   └── withMlkitSimulatorArm64Fix.js\n├── example/                      # Example app\n│   ├── App.tsx\n│   └── app.json\n└── expo-module.config.json\n```\n\n### Building from Source\n\n```bash\n# Install dependencies\nnpm install\n\n# Build the module (src/ → build/)\nnpm run prepare\n\n# Open the example app (iOS)\nnpm run open:ios\n\n# Or run the example app with Expo CLI\ncd example\nnpx expo start\n\n# Scan QR code with Expo Go or run on device/simulator\n```\n\n### Running the Example App\n\nThe example app at `example/` demonstrates:\n1. Picking an image from the device library\n2. Running OCR with `recognizeText()`\n3. Displaying the results (full text, blocks, lines, elements)\n\n## License\n\nMIT\n\n## Contributing\n\nContributions are welcome! Please ensure:\n- TypeScript code compiles without errors\n- Native code follows platform conventions\n- Example app works on both iOS and Android\n\n## References\n\n- [ML Kit Text Recognition (v2) — iOS](https://developers.google.com/ml-kit/vision/text-recognition/v2/ios)\n- [ML Kit Text Recognition (v2) — Android](https://developers.google.com/ml-kit/vision/text-recognition/v2/android)\n- [Expo Modules API](https://docs.expo.dev/modules/get-started/)\n\nMade with ❤️ by [rbayuokt](https://github.com/rbayuokt)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frbayuokt%2Fexpo-mlkit-ocr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frbayuokt%2Fexpo-mlkit-ocr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frbayuokt%2Fexpo-mlkit-ocr/lists"}