{"id":18331988,"url":"https://github.com/mrousavy/vision-camera-resize-plugin","last_synced_at":"2025-05-16T15:03:29.824Z","repository":{"id":78946274,"uuid":"374646185","full_name":"mrousavy/vision-camera-resize-plugin","owner":"mrousavy","description":"A VisionCamera Frame Processor plugin for fast buffer resizing and colorspace (YUV \u003c\u003e RGBA) conversions","archived":false,"fork":false,"pushed_at":"2024-12-06T13:42:04.000Z","size":5626,"stargazers_count":98,"open_issues_count":25,"forks_count":24,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-03T13:15:43.581Z","etag":null,"topics":["buffer","camera","conversion","convert","react-native","react-native-vision-camera","resize","rgb","tensorflow","tflite","vision","yuv"],"latest_commit_sha":null,"homepage":"https://mrousavy.com","language":"Objective-C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mrousavy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-07T11:47:09.000Z","updated_at":"2025-03-24T10:59:27.000Z","dependencies_parsed_at":"2024-06-08T16:37:43.055Z","dependency_job_id":"077938e8-4c53-41cb-99f1-c9cf6e5e8506","html_url":"https://github.com/mrousavy/vision-camera-resize-plugin","commit_stats":{"total_commits":215,"total_committers":9,"mean_commits":23.88888888888889,"dds":"0.18604651162790697","last_synced_commit":"e2b4d79a27da62f8fe9855bf41b0de0695520612"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrousavy%2Fvision-camera-resize-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrousavy%2Fvision-camera-resize-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrousavy%2Fvision-camera-resize-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrousavy%2Fvision-camera-resize-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mrousavy","download_url":"https://codeload.github.com/mrousavy/vision-camera-resize-plugin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248564862,"owners_count":21125412,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["buffer","camera","conversion","convert","react-native","react-native-vision-camera","resize","rgb","tensorflow","tflite","vision","yuv"],"created_at":"2024-11-05T19:36:44.007Z","updated_at":"2025-04-12T11:49:26.583Z","avatar_url":"https://github.com/mrousavy.png","language":"Objective-C++","funding_links":["https://github.com/sponsors/mrousavy"],"categories":[],"sub_categories":[],"readme":"# vision-camera-resize-plugin\n\nA [VisionCamera](https://github.com/mrousavy/react-native-vision-camera) Frame Processor Plugin for fast and efficient Frame resizing, cropping and pixel-format conversion (YUV -\u003e RGB) using GPU-acceleration, CPU-vector based operations and ARM NEON SIMD acceleration.\n\n## Installation\n\n1. Install [react-native-vision-camera](https://github.com/mrousavy/react-native-vision-camera) (\u003e= 3.8.2) and [react-native-worklets-core](https://github.com/margelo/react-native-worklets-core) (\u003e= 0.2.4) and make sure Frame Processors are enabled.\n2. Install vision-camera-resize-plugin:\n    ```sh\n    yarn add vision-camera-resize-plugin\n    cd ios \u0026\u0026 pod install\n    ```\n\n## Usage\n\nUse the `resize` plugin within a Frame Processor:\n\n```tsx\nconst { resize } = useResizePlugin()\n\nconst frameProcessor = useFrameProcessor((frame) =\u003e {\n  'worklet'\n\n  const resized = resize(frame, {\n    scale: {\n      width: 192,\n      height: 192\n    },\n    pixelFormat: 'rgb',\n    dataType: 'uint8'\n  })\n\n  const firstPixel = {\n    r: resized[0],\n    g: resized[1],\n    b: resized[2]\n  }\n}, [])\n```\n\nOr outside of a function component:\n\n```tsx\nconst { resize } = createResizePlugin()\n\nconst frameProcessor = createFrameProcessor((frame) =\u003e {\n  'worklet'\n\n  const resized = resize(frame, {\n    // ...\n  })\n  // ...\n})\n```\n\n## Pixel Formats\n\nThe resize plugin operates in RGB colorspace.\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003cth\u003eName\u003c/th\u003e\n\u003cth\u003e\u003ccode\u003e0\u003c/code\u003e\u003c/th\u003e\n\u003cth\u003e\u003ccode\u003e1\u003c/code\u003e\u003c/th\u003e\n\u003cth\u003e\u003ccode\u003e2\u003c/code\u003e\u003c/th\u003e\n\u003cth\u003e\u003ccode\u003e3\u003c/code\u003e\u003c/th\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003ergb\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003eR\u003c/td\u003e\n\u003ctd\u003eG\u003c/td\u003e\n\u003ctd\u003eB\u003c/td\u003e\n\u003ctd\u003eR\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003ergba\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003eR\u003c/td\u003e\n\u003ctd\u003eG\u003c/td\u003e\n\u003ctd\u003eB\u003c/td\u003e\n\u003ctd\u003eA\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003eargb\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003eA\u003c/td\u003e\n\u003ctd\u003eR\u003c/td\u003e\n\u003ctd\u003eG\u003c/td\u003e\n\u003ctd\u003eB\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003ebgra\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003eB\u003c/td\u003e\n\u003ctd\u003eG\u003c/td\u003e\n\u003ctd\u003eR\u003c/td\u003e\n\u003ctd\u003eA\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003ebgr\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003eB\u003c/td\u003e\n\u003ctd\u003eG\u003c/td\u003e\n\u003ctd\u003eR\u003c/td\u003e\n\u003ctd\u003eB\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003eabgr\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003eA\u003c/td\u003e\n\u003ctd\u003eB\u003c/td\u003e\n\u003ctd\u003eG\u003c/td\u003e\n\u003ctd\u003eR\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003c/table\u003e\n\n## Data Types\n\nThe resize plugin can either convert to uint8 or float32 values:\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003cth\u003eName\u003c/th\u003e\n\u003cth\u003eJS Type\u003c/th\u003e\n\u003cth\u003eValue Range\u003c/th\u003e\n\u003cth\u003eExample size\u003c/th\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003euint8\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003eUint8Array\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e0...255\u003c/td\u003e\n\u003ctd\u003e1920x1080 RGB Frame = ~6.2 MB\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003efloat32\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003eFloat32Array\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e0.0...1.0\u003c/td\u003e\n\u003ctd\u003e1920x1080 RGB Frame = ~24.8 MB\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003c/table\u003e\n\n## Cropping\n\nWhen scaling to a different size (e.g. 1920x1080 -\u003e 100x100), the Resize Plugin performs a center-crop on the image before scaling it down so the resulting image matches the target aspect ratio instead of being stretched.\n\nYou can customize this by passing a custom `crop` parameter, e.g. instead of center-crop, use the top portion of the screen:\n\n```ts\nconst resized = resize(frame, {\n  scale: {\n    width: 192,\n    height: 192\n  },\n  crop: {\n    y: 0,\n    x: 0,\n    // 1:1 aspect ratio because we scale to 192x192\n    width: frame.width,\n    height: frame.width\n  },\n  pixelFormat: 'rgb',\n  dataType: 'uint8'\n})\n```\n\n### Performance\n\nIf possible, use one of these two formats:\n\n- `argb` in `uint8`: Can be converted the fastest, but has an additional unused alpha channel.\n- `rgb` in `uint8`: Requires one more conversion step from `argb`, but uses 25% less memory due to the removed alpha channel.\n\nAll other formats require additional conversion steps, and `float` models have additional memory overhead (4x as big).\n\nWhen using TensorFlow Lite, try to convert your model to use `argb-uint8` or `rgb-uint8` as it's input type.\n\n## react-native-fast-tflite\n\nThe vision-camera-resize-plugin can be used together with [react-native-fast-tflite](https://github.com/mrousavy/react-native-fast-tflite) to prepare the input tensor data.\n\nFor example, to use the [efficientdet](https://www.kaggle.com/models/tensorflow/efficientdet/frameworks/tfLite) TFLite model to detect objects inside a Frame, simply add the model to your app's bundle, set up VisionCamera and react-native-fast-tflite, and resize your Frames accordingly.\n\nFrom the model's description on the website, we understand that the model expects 320 x 320 x 3 buffers as input, where the format is uint8 rgb.\n\n```ts\nconst objectDetection = useTensorflowModel(require('assets/efficientdet.tflite'))\nconst model = objectDetection.state === \"loaded\" ? objectDetection.model : undefined\n\nconst { resize } = useResizePlugin()\n\nconst frameProcessor = useFrameProcessor((frame) =\u003e {\n  'worklet'\n\n  const data = resize(frame, {\n    scale: {\n      width: 320,\n      height: 320,\n    },\n    pixelFormat: 'rgb',\n    dataType: 'uint8'\n  })\n  const output = model.runSync([data])\n\n  const numDetections = output[0]\n  console.log(`Detected ${numDetections} objects!`)\n}, [model])\n```\n\n## Benchmarks\n\nI benchmarked vision-camera-resize-plugin on an iPhone 15 Pro, using the following code:\n\n```tsx\nconst start = performance.now()\nconst result = resize(frame, {\n  scale: {\n    width: 100,\n    height: 100,\n  },\n  pixelFormat: 'rgb',\n  dataType: 'uint8'\n})\nconst end = performance.now()\n\nconst diff = (end - start).toFixed(2)\nconsole.log(`Resize and conversion took ${diff}ms!`)\n```\n\nAnd when running on 1080x1920 yuv Frames, I got the following results:\n\n```\n LOG  Resize and conversion took 6.48ms\n LOG  Resize and conversion took 6.06ms\n LOG  Resize and conversion took 5.89ms\n LOG  Resize and conversion took 5.97ms\n LOG  Resize and conversion took 6.98ms\n```\n\nThis means the Frame Processor can run at up to ~160 FPS.\n\n## Adopting at scale\n\n\u003ca href=\"https://github.com/sponsors/mrousavy\"\u003e\n  \u003cimg align=\"right\" width=\"160\" alt=\"This library helped you? Consider sponsoring!\" src=\".github/funding-octocat.svg\"\u003e\n\u003c/a\u003e\n\nThis library is provided _as is_, I work on it in my free time.\n\nIf you're integrating vision-camera-resize-plugin in a production app, consider [funding this project](https://github.com/sponsors/mrousavy) and \u003ca href=\"mailto:me@mrousavy.com?subject=Adopting vision-camera-resize-plugin at scale\"\u003econtact me\u003c/a\u003e to receive premium enterprise support, help with issues, prioritize bugfixes, request features, help at integrating vision-camera-resize-plugin and/or VisionCamera Frame Processors, and more.\n\n\n## Contributing\n\nSee the [contributing guide](CONTRIBUTING.md) to learn how to contribute to the repository and the development workflow.\n\n## License\n\nMIT\n\n---\n\nMade with [create-react-native-library](https://github.com/callstack/react-native-builder-bob)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrousavy%2Fvision-camera-resize-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmrousavy%2Fvision-camera-resize-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrousavy%2Fvision-camera-resize-plugin/lists"}