{"id":14968240,"url":"https://github.com/kfrancis/ocr","last_synced_at":"2025-05-16T14:09:17.201Z","repository":{"id":229893460,"uuid":"777517150","full_name":"kfrancis/ocr","owner":"kfrancis","description":"A plugin to provide the ability to do simple text from image OCR using nothing but platform APIs.","archived":false,"fork":false,"pushed_at":"2025-03-25T15:34:50.000Z","size":1853,"stargazers_count":139,"open_issues_count":6,"forks_count":8,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-03T11:13:04.351Z","etag":null,"topics":["dotnet","maui","ocr","xamarin"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kfrancis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"kfrancis","buy_me_a_coffee":"korifrancis"}},"created_at":"2024-03-26T01:55:06.000Z","updated_at":"2025-03-31T06:55:39.000Z","dependencies_parsed_at":"2024-04-14T18:24:22.433Z","dependency_job_id":"4887ed86-48fa-45f6-8a91-27f33643b739","html_url":"https://github.com/kfrancis/ocr","commit_stats":{"total_commits":83,"total_committers":1,"mean_commits":83.0,"dds":0.0,"last_synced_commit":"95aeb727f08887d81169f6dd3aaec5f47a91d2bd"},"previous_names":["kfrancis/ocr"],"tags_count":20,"template":false,"template_full_name":"jfversluis/Plugin.Maui.Feature","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kfrancis%2Focr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kfrancis%2Focr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kfrancis%2Focr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kfrancis%2Focr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kfrancis","download_url":"https://codeload.github.com/kfrancis/ocr/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246997277,"owners_count":20866686,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dotnet","maui","ocr","xamarin"],"created_at":"2024-09-24T13:39:34.767Z","updated_at":"2025-04-03T18:12:08.657Z","avatar_url":"https://github.com/kfrancis.png","language":"C#","funding_links":["https://github.com/sponsors/kfrancis","https://buymeacoffee.com/korifrancis"],"categories":[],"sub_categories":[],"readme":"![](nuget.png)\n# Plugin.Maui.OCR | Plugin.Xamarin.OCR\n\n`Plugin.Maui.OCR` and `Plugin.Xamarin.OCR` provide the ability to do simple text from image OCR using nothing but platform APIs.\n\n## What Works Matrix\n\n| Platform | iOS | Android | Windows | macOS |\n|----------|-----|---------|---------|-------|\n| MAUI     | Yes |   Yes   |   Yes   |  Yes  |\n| Xamarin  | Yes |   Yes   |   N/A   |  N/A  |\n\n[![Build for CI](https://github.com/kfrancis/ocr/actions/workflows/ci.yml/badge.svg)](https://github.com/kfrancis/ocr/actions/workflows/ci.yml)\n\n## Why\n\nWhy am I making this? I'm doing this because I want to make it easier for developers to do OCR in their apps. I want to make it so that you can just use this plugin and not have to worry about the platform specifics.\n\nToo many times I've tried to do OCR and had to wrestle with external dependencies like Tesseract (with its dependencies Leptonica, etc) and these types\nof native dependencies can be a real pain to work with. \n\n### Xamarin??\n\nWell, I still have to maintain a Xamarin app that uses Tesseract and I'm tired of all the problems that come with it. I want to make it easier for myself and others to do OCR in their apps.\n\n### Example App\n\nHere's an example of what the OCR library can do. This is the processing of race results receipt, which is a typical dot-matrix printed document. The OCR is able to read the text from the image.\n\n![Example](example.gif)\n\n## Install Plugin\n\n[![NuGet](https://img.shields.io/nuget/v/Plugin.Maui.OCR.svg?label=MAUI)](https://www.nuget.org/packages/Plugin.Maui.OCR/)\n[![NuGet](https://img.shields.io/nuget/v/Plugin.Xamarin.OCR.svg?label=Xamarin)](https://www.nuget.org/packages/Plugin.Xamarin.OCR/)\n\nAvailable on NuGet for [MAUI](http://www.nuget.org/packages/Plugin.Maui.OCR) and [Xamarin](http://www.nuget.org/packages/Plugin.Xamarin.OCR).\n\nInstall with the dotnet CLI: `dotnet add package Plugin.Maui.OCR` or `dotnet add package Plugin.Xamarin.OCR`, or through the NuGet Package Manager in Visual Studio.\n\n### Supported Platforms\n\n| Platform | Minimum Version Supported |\n|----------|---------------------------|\n| iOS      | 13+                       |\n| macOS    | 10.15+                    |\n| Android  | 5.0 (API 21)              |\n| Windows  | 11 and 10 version 1809+   |\n\n## OCR Engines and Algorithms\n\nThe **Plugin.Maui.OCR** library is designed to provide a cross-platform, easy-to-use wrapper for native OCR implementations without relying on external OCR engines like Tesseract. Below is an overview of the OCR engines and frameworks used on supported platforms:\n\n### iOS\nThe plugin leverages Apple's [Vision Framework](https://developer.apple.com/documentation/vision/vnrecognizetextrequest) for text recognition. This framework is highly optimized for Apple devices and supports various recognition features like text in different languages, accuracy levels, and bounding box detection. Key aspects include:\n\n- **OCR Engine**: Vision Framework\n- **Supported Languages**: Determined dynamically at runtime, based on the available languages supported by Vision's text recognition capabilities.\n- **Recognition Levels**: `Fast` (for quick recognition) and `Accurate` (for higher precision).\n- **Other Features**: Bounding box detection, CPU/GPU processing options, and real-time recognition capabilities.\n\nFor further details, you can review the official [Vision Framework documentation](https://developer.apple.com/documentation/vision).\n\n### Android\nThe plugin utilizes Google's [ML Kit](https://developers.google.com/ml-kit/vision/text-recognition/overview) for on-device and cloud-based text recognition. ML Kit provides efficient OCR capabilities and can operate in both offline and online modes depending on the configuration. Key aspects include:\n\n- **OCR Engine**: [ML Kit Text Recognition](https://developers.google.com/ml-kit/vision/text-recognition)\n- **Supported Languages**:\n  - **On-Device**: Primarily Latin-based scripts such as English, Spanish, French, German, Italian, and Portuguese.\n  - **Cloud-Based**: A broader range of languages, including Arabic, Chinese (Simplified and Traditional), Greek, Russian, Thai, and many more.\n- **Recognition Levels**: The plugin offers two modes:\n  - **Fast** (on-device, for quicker results with limited language support)\n  - **Accurate** (cloud-based, for more accurate results and extended language support)\n- **Other Features**: Confidence scores for recognized text, bounding box detection for each text element, and cloud-based recognition for more extensive language support (requires an internet connection).\n\n### Windows\nThe plugin leverages the native [Windows.Media.Ocr](https://learn.microsoft.com/en-us/uwp/api/windows.media.ocr) API for text recognition. This API enables high-quality OCR capabilities on supported Windows devices without the need for external libraries. Key aspects include:\n\n- **OCR Engine**: [Windows.Media.Ocr](https://learn.microsoft.com/en-us/uwp/api/windows.media.ocr)\n- **Supported Languages**: The OCR engine supports languages installed on the user's system. A dynamic list of supported languages can be retrieved using the `OcrEngine.AvailableRecognizerLanguages` property.\n- **Recognition Levels**: The Windows OCR API operates with a single recognition mode, but its efficiency can be influenced by the language settings and image quality.\n- **Other Features**: Bounding box detection for recognized words, support for different image formats, and multi-language support based on user profile languages.\n\n## Camera Specifications and Quality Considerations\nWhile the plugin handles the OCR process, the following can impact the recognition quality:\n\n- **Camera Resolution**: Higher resolutions generally result in better text recognition.\n- **Distance from Target**: Keeping the camera at an optimal distance for clear, sharp text is recommended.\n- **Lighting Conditions**: Adequate lighting will enhance the OCR accuracy.\n- **Supported Image Formats**: The plugin works with common formats such as JPEG and PNG.\n\n## Pattern Matching\n\nOne of the more common things I do with OCR is recognize a text pattern. For example, I might want to read a date, a phone number or an email address. This is where the `OcrPatternConfig` class comes in.\n\nLet's say you want to recognize an Ontario Health Card Number (HCN) in the text of your image. Numbers of those types have some specific qualities that make it easy to match.\n\n1. An Ontario HCN is 10 digits long.\n1. The number must be [Luhn valid](https://en.wikipedia.org/wiki/Luhn_algorithm) (meaning it has a check digit and it's correct).\n\nTo do this, you can create an `OcrPatternConfig` object like so:\n\n```csharp\nbool IsValidLuhn(string number)\n{\n    // Convert the string to an array of digits\n    int[] digits = number.Select(d =\u003e int.Parse(d.ToString())).ToArray();\n    int checkDigit = 0;\n\n    // Luhn algorithm implementation\n    for (int i = digits.Length - 2; i \u003e= 0; i--)\n    {\n        int currentDigit = digits[i];\n        if ((digits.Length - 2 - i) % 2 == 0) // check if it's an even index from the right\n        {\n            currentDigit *= 2;\n            if (currentDigit \u003e 9)\n            {\n                currentDigit -= 9;\n            }\n        }\n        checkDigit += currentDigit;\n    }\n\n    return (10 - (checkDigit % 10)) % 10 == digits.Last();\n}\n\nvar ohipPattern = new OcrPatternConfig(@\"\\d{10}\", IsLuhnValid);\n\nvar options = new OcrOptions.Builder().SetTryHard(true).SetPatternConfig(ohipPattern).Build();\n\nvar result = await OcrPlugin.Default.RecognizeTextAsync(imageData, options);\n\nvar patientHcn = result.MatchedValues.FirstOrDefault(); // This will be the HCN (and only the HCN) if it's found\n```\n\n## MAUI Setup and Usage\n\nFor MAUI, to initialize make sure you use the MauiAppBuilder extension `UseOcr()` like so:\n\n```csharp\npublic static class MauiProgram\n{\n\tpublic static MauiApp CreateMauiApp()\n\t{\n\t\tvar builder = MauiApp.CreateBuilder();\n\t\tbuilder\n\t\t\t.UseMauiApp\u003cApp\u003e()\n\t\t\t.ConfigureFonts(fonts =\u003e\n\t\t\t{\n\t\t\t\tfonts.AddFont(\"OpenSans-Regular.ttf\", \"OpenSansRegular\");\n\t\t\t\tfonts.AddFont(\"OpenSans-Semibold.ttf\", \"OpenSansSemibold\");\n\t\t\t}).\n\t\t\tUseOcr();  // \u003c-- add this line\n\n\t\treturn builder.Build();\n\t}\n}\n```\n\nAnd then you can just inject `IOcrService` into your classes and use it like so:\n\n```csharp\n/// \u003csummary\u003e\n/// Takes a photo and processes it using the OCR service.\n/// \u003c/summary\u003e\n/// \u003cparam name=\"photo\"\u003eThe photo to process.\u003c/param\u003e\n/// \u003creturns\u003eThe OCR result.\u003c/returns\u003e\nprivate async Task\u003cOcrResult\u003e ProcessPhoto(FileResult photo)\n{\n    // Open a stream to the photo\n    using var sourceStream = await photo.OpenReadAsync();\n\n    // Create a byte array to hold the image data\n    var imageData = new byte[sourceStream.Length];\n\n    // Read the stream into the byte array\n    await sourceStream.ReadAsync(imageData);\n\n    // Process the image data using the OCR service\n    return await _ocr.RecognizeTextAsync(imageData);\n}\n```\n\n## Xamarin Setup and Usage\n\nFor Xamarin, if you have some kind of DI framework in place then you can just register the `OcrPlugin` with it.\n\n```csharp\npublic App()\n{\n    InitializeComponent();\n\n    DependencyService.RegisterSingleton(OcrPlugin.Default);\n\n    MainPage = new MainPage();\n}\n```\n\nIf you don't have a DI framework in place, you can use the `OcrPlugin.Default` property to access the `IOcrService` instance.\n\n```csharp\nprivate readonly IOcrService _ocr;\n\npublic MainPage(IOcrService? ocr)\n{\n    InitializeComponent();\n\n    _ocr = ocr ?? OcrPlugin.Default;\n}\n```\n\n## Details\n\nThe `IOcrService` interface exposes the following methods:\n\n```csharp\npublic interface IOcrService\n{\n    event EventHandler\u003cOcrCompletedEventArgs\u003e RecognitionCompleted;\n    IReadOnlyCollection\u003cstring\u003e SupportedLanguages { get; }\n    Task InitAsync(CancellationToken ct = default);\n    Task\u003cOcrResult\u003e RecognizeTextAsync(byte[] imageData, bool tryHard = false, CancellationToken ct = default);\n    Task\u003cOcrResult\u003e RecognizeTextAsync(byte[] imageData, OcrOptions options, CancellationToken ct = default);\n    Task StartRecognizeTextAsync(byte[] imageData, OcrOptions options, CancellationToken ct = default);\n}\n\npublic class OcrResult\n{\n    public bool Success { get; set; }\n\n    public string AllText { get; set; }\n\n    public IList\u003cOcrElement\u003e Elements { get; set; } = new List\u003cOcrElement\u003e();\n    public IList\u003cstring\u003e Lines { get; set; } = new List\u003cstring\u003e();\n\n    public class OcrElement\n    {\n        public string Text { get; set; }\n        public float Confidence { get; set; }\n\n        // Useful for bounding boxes\n        public int X { get; set; }\n        public int Y { get; set; }\n        public int Height { get; set; }\n        public int Width { get; set; }\n    }\n}\n```\n\n### Permissions\n\nBefore you can start using Feature, you will need to request the proper permissions on each platform.\n\n#### iOS\n\nIf you're handling camera, you'll need the usual permissions for that.\n\n#### Android\n\nIf you're handling camera, you'll need the usual permissions for that. The only extra part you'll want in the AndroidManifest.xml is the following:\n\n```xml\n\u003capplication ..\u003e\n  \u003cmeta-data android:name=\"com.google.mlkit.vision.DEPENDENCIES\" android:value=\"ocr\" /\u003e\n\u003c/application\u003e\n```\n\nThis will cause the model necessary to be installed when the application is installed.\n\n## OcrOptions and Builder\n\nThe `OcrOptions` class provides a flexible way to configure OCR settings. You can use the `OcrOptions.Builder` class to create instances of `OcrOptions` with various configurations.\n\n### OcrOptions Class\n\nThe `OcrOptions` class holds the configuration for OCR operations.\n\n```csharp\npublic class OcrOptions\n{\n    public string? Language { get; }\n    public bool TryHard { get; }\n    public List\u003cOcrPatternConfig\u003e PatternConfigs { get; }\n    public CustomOcrValidationCallback? CustomCallback { get; }\n\n    private OcrOptions(string? language, bool tryHard, List\u003cOcrPatternConfig\u003e patternConfigs, CustomOcrValidationCallback? customCallback)\n    {\n        Language = language;\n        TryHard = tryHard;\n        PatternConfigs = patternConfigs;\n        CustomCallback = customCallback;\n    }\n\n    public class Builder\n    {\n        private string? _language;\n        private bool _tryHard;\n        private List\u003cOcrPatternConfig\u003e _patternConfigs = new List\u003cOcrPatternConfig\u003e();\n        private CustomOcrValidationCallback? _customCallback;\n\n        public Builder SetLanguage(string language)\n        {\n            _language = language;\n            return this;\n        }\n\n        public Builder SetTryHard(bool tryHard)\n        {\n            _tryHard = tryHard;\n            return this;\n        }\n\n        public Builder AddPatternConfig(OcrPatternConfig patternConfig)\n        {\n            _patternConfigs.Add(patternConfig);\n            return this;\n        }\n\n        public Builder SetPatternConfigs(List\u003cOcrPatternConfig\u003e patternConfigs)\n        {\n            _patternConfigs = patternConfigs ?? new List\u003cOcrPatternConfig\u003e();\n            return this;\n        }\n\n        public Builder SetCustomCallback(CustomOcrValidationCallback customCallback)\n        {\n            _customCallback = customCallback;\n            return this;\n        }\n\n        public OcrOptions Build()\n        {\n            return new OcrOptions(_language, _tryHard, _patternConfigs, _customCallback);\n        }\n    }\n}\n```\n\n### Usage Example\n\nUsing the `OcrOptions.Builder` to create an `OcrOptions` instance is straightforward and flexible:\n\n```csharp\nvar options = new OcrOptions.Builder()\n    .SetLanguage(\"en-US\")\n    .SetTryHard(true)\n    .AddPatternConfig(new OcrPatternConfig(@\"\\d{10}\"))\n    .SetCustomCallback(myCustomCallback)\n    .Build();\n```\n\n### Dependency Injection\n\nYou will first need to register the `OcrPlugin` with the `MauiAppBuilder` following the same pattern that the .NET MAUI Essentials libraries follow.\n\n```csharp\nbuilder.Services.AddSingleton(OcrPlugin.Default);\n```\n\nYou can then enable your classes to depend on `IOcrService` as per the following example.\n\n```csharp\npublic class OcrViewModel\n{\n    readonly IOcrService _ocr;\n\n    public OcrViewModel(IOcrService? ocr)\n    {\n        _ocr = ocr ?? OcrPlugin.Default;\n    }\n\n    public void DoSomeOcr()\n    {\n        byte[] imageData = GetImageData();\n\n        var result = await _ocr.RecognizeTextAsync(imageData);\n    }\n}\n```\n\n### Straight usage\n\nAlternatively if you want to skip using the dependency injection approach you can use the `Feature.Default` property.\n\n```csharp\npublic class OcrViewModel\n{\n    public void DoSomeOcr()\n    {\n        byte[] imageData = GetImageData();\n\n        var result = await OcrPlugin.Default.RecognizeTextAsync(imageData);\n    }\n}\n```\n\n### Feature\n\nOnce you have the `OCR` instance, you can interact with it in the following ways:\n\n#### Events\n\n##### `RecognitionCompleted`\n\nThis event is fired when the OCR service has completed recognizing text from an image. The event args contain the `OcrResult` object. Only fires if the `StartRecognizeTextAsync` method is called.\n\n#### Properties\n\n##### `SupportedLanguages`\n\nA list of supported languages for the OCR service. This is populated after calling `InitAsync`. Allows you to know what language codes can be used in OcrOptions.\n\n#### Methods\n\n##### `InitAsync(CancellationToken ct = default)`\n\nInitialize the feature. If supported on the platform (like iOS), SupportedLanguages will be populated with the available languages.\n\n##### `RecognizeTextAsync(byte[] imageData, bool tryHard = false, CancellationToken ct = default)`\n\nRecognize text from an image. Specify \"tryHard\" if you want to tell the platform API to do a better job (fast vs accurate, and use language correction (ios/mac)) though it seems very accurate normally.\n\n##### `RecognizeTextAsync(byte[] imageData, OcrOptions options, CancellationToken ct = default)`\n\nRecognize text from an image. OcrOptions contains options for the OCR service, including the language to use and whether to try hard.\n\n##### `Task StartRecognizeTextAsync(byte[] imageData, OcrOptions options, CancellationToken ct = default)`\n\nStart recognizing text from an image. This is a task that will fire the `RecognitionCompleted` event when it completes with the result.\n\n# Acknowledgements\n\nThanks to the great **Gerald Versluis** for making an amazing video about using this project: https://www.youtube.com/watch?v=alY_6Qn0_60\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkfrancis%2Focr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkfrancis%2Focr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkfrancis%2Focr/lists"}