{"id":50082525,"url":"https://github.com/dmitry-brazhenko/SharpToken","last_synced_at":"2026-06-08T07:01:44.070Z","repository":{"id":148386983,"uuid":"620251190","full_name":"dmitry-brazhenko/SharpToken","owner":"dmitry-brazhenko","description":"SharpToken is a C# library for tokenizing natural language text. It's based on the tiktoken Python library and designed to be fast and accurate. ","archived":false,"fork":false,"pushed_at":"2026-03-25T16:25:16.000Z","size":4268,"stargazers_count":259,"open_issues_count":7,"forks_count":19,"subscribers_count":12,"default_branch":"main","last_synced_at":"2026-05-18T02:24:34.148Z","etag":null,"topics":["cl100kbase","csharp","gpt","gpt-3","gpt-4","openai","tokenizer"],"latest_commit_sha":null,"homepage":"https://www.nuget.org/packages/SharpToken","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dmitry-brazhenko.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-03-28T10:17:52.000Z","updated_at":"2026-05-16T12:37:22.000Z","dependencies_parsed_at":"2025-08-10T07:22:41.991Z","dependency_job_id":"da9cd1c4-272b-4350-9365-e69728f837cb","html_url":"https://github.com/dmitry-brazhenko/SharpToken","commit_stats":null,"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"purl":"pkg:github/dmitry-brazhenko/SharpToken","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmitry-brazhenko%2FSharpToken","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmitry-brazhenko%2FSharpToken/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmitry-brazhenko%2FSharpToken/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmitry-brazhenko%2FSharpToken/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dmitry-brazhenko","download_url":"https://codeload.github.com/dmitry-brazhenko/SharpToken/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmitry-brazhenko%2FSharpToken/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34051772,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-08T02:00:07.615Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cl100kbase","csharp","gpt","gpt-3","gpt-4","openai","tokenizer"],"created_at":"2026-05-22T16:04:04.263Z","updated_at":"2026-06-08T07:01:44.063Z","avatar_url":"https://github.com/dmitry-brazhenko.png","language":"C#","funding_links":[],"categories":["Calculators and Estimators"],"sub_categories":["Tokenizers"],"readme":"# SharpToken\n\n[![NuGet](https://img.shields.io/nuget/v/SharpToken.svg)](https://www.nuget.org/packages/SharpToken)\n[![dotnet](https://github.com/dmitry-brazhenko/SharpToken/actions/workflows/build-test-and-publish.yml/badge.svg?branch=main)](https://github.com/dmitry-brazhenko/SharpToken/actions/workflows/build-test-and-publish.yml)\n[![Last Commit](https://img.shields.io/github/last-commit/dmitry-brazhenko/SharpToken.svg)](https://github.com/dmitry-brazhenko/SharpToken/commits/main)\n[![GitHub Issues](https://img.shields.io/github/issues/dmitry-brazhenko/SharpToken.svg)](https://github.com/dmitry-brazhenko/SharpToken/issues)\n[![Used by](https://img.shields.io/nuget/dt/SharpToken.svg)](https://www.nuget.org/packages/SharpToken)\n[![Contributors](https://img.shields.io/github/contributors/dmitry-brazhenko/SharpToken.svg)](https://github.com/dmitry-brazhenko/SharpToken/graphs/contributors)\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\n\nSharpToken is a C# library that serves as a port of the Python [tiktoken](https://github.com/openai/tiktoken) library.\nIt provides functionality for encoding and decoding tokens using GPT-based encodings. This library is built for .NET 6, .NET 8\nand .NET Standard 2.0, making it compatible with a wide range of frameworks.\n\n\u003e [!Important]\n\u003e The functionality in `SharpToken` has been added to [`Microsoft.ML.Tokenizers`](https://www.nuget.org/packages/Microsoft.ML.Tokenizers). `Microsoft.ML.Tokenizers` is a tokenizer library being developed by the .NET team and going forward, the central place for tokenizer development in .NET. By using `Microsoft.ML.Tokenizers`, you should see improved performance over existing tokenizer library implementations, including `SharpToken`. A stable release of `Microsoft.ML.Tokenizers` is expected alongside the .NET 9.0 release (November 2024). Instructions for migration can be found at https://github.com/dotnet/machinelearning/blob/main/docs/code/microsoft-ml-tokenizers-migration-guide.md.\n\n## Installation\n\nTo install SharpToken, use the NuGet package manager:\n\n```powershell\nInstall-Package SharpToken\n```\n\nOr, if you prefer using the .NET CLI:\n\n```powershell\ndotnet add package SharpToken\n```\n\nFor more information, visit the [NuGet package page](https://www.nuget.org/packages/SharpToken).\n\n## Usage\n\nTo use SharpToken in your project, first import the library:\n\n```csharp\nusing SharpToken;\n```\n\nNext, create an instance of GptEncoding by specifying the desired encoding or model:\n\n```csharp\n// Get encoding by encoding name\nvar encoding = GptEncoding.GetEncoding(\"cl100k_base\");\n\n// Get encoding by model name\nvar encoding = GptEncoding.GetEncodingForModel(\"gpt-4\");\n```\n\nYou can then use the Encode method to encode a string:\n\n```csharp\nvar encoded = encoding.Encode(\"Hello, world!\"); // Output: [9906, 11, 1917, 0]\n```\n\nAnd use the Decode method to decode the encoded tokens:\n\n```csharp\nvar decoded = encoding.Decode(encoded); // Output: \"Hello, world!\"\n```\n\nSharpToken also provides a high performance count method.\nIt is usefull to check prompt size before sending it to a LLM or to use it in a TextSplitter/Chunker for RAG.\n\n```csharp\nvar count = encoding.CountTokens(\"Hello, world!\"); // Output: 4\n```\n\n## Supported Models\n\nSharpToken currently supports the following models:\n\n- `r50k_base`\n- `p50k_base`\n- `p50k_edit`\n- `cl100k_base`\n- `o200k_base`\n- `o200k_harmony`\n- `claude`\n\nYou can use any of these encodings when creating an instance of GptEncoding:\n\n```csharp\nvar r50kBaseEncoding = GptEncoding.GetEncoding(\"r50k_base\");\nvar p50kBaseEncoding = GptEncoding.GetEncoding(\"p50k_base\");\nvar p50kEditEncoding = GptEncoding.GetEncoding(\"p50k_edit\");\nvar cl100kBaseEncoding = GptEncoding.GetEncoding(\"cl100k_base\");\nvar o200kBaseEncoding = GptEncoding.GetEncoding(\"o200k_base\");\nvar o200kHarmonyEncoding = GptEncoding.GetEncoding(\"o200k_harmony\");\nvar claudeEncoding = GptEncoding.GetEncoding(\"claude\");\n```\n\n### Claude Model Support\n\nThe `claude` encoding uses Anthropic's official tokenizer vocabulary with NFKC normalization. It is accurate for pre-Claude 3 models and a rough approximation for Claude 3+.\n\n```csharp\nvar encoding = GptEncoding.GetEncodingForModel(\"claude-3.5-sonnet\");\nvar count = encoding.CountTokens(\"Hello, Claude!\");\n```\n\nAll `claude-*` model names are supported (e.g. `claude-3-opus`, `claude-3.5-sonnet`, `claude-3.7-sonnet`, `claude-4-sonnet`).\n\n### Model Prefix Matching\n\nApart from specifying direct model names, SharpToken also provides functionality to map model names based on specific prefixes. This allows users to retrieve an encoding based on a model's prefix.\n\nHere are the current supported prefixes and their corresponding encodings:\n\n| Model Prefix     | Encoding      |\n| ---------------- | ------------- |\n| `claude-`        | `claude`      |\n| `gpt-5`          | `o200k_base`  |\n| `gpt-4o`         | `o200k_base`  |\n| `gpt-4-`         | `cl100k_base` |\n| `gpt-3.5-turbo-` | `cl100k_base` |\n| `gpt-35-turbo`   | `cl100k_base` |\n\nExamples of model names that fall under these prefixes include:\n\n- For the prefix `claude-`: `claude-3-opus-20240229`, `claude-3.5-sonnet-20241022`, etc.\n- For the prefix `gpt-5`: `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-5-pro`, `gpt-5-thinking`, `gpt-5-2024-08-07`, etc.\n- For the prefix `gpt-4o`: `gpt-4o`, `gpt-4o-2024-05-13`, etc.\n- For the prefix `gpt-4-`: `gpt-4-0314`, `gpt-4-32k`, etc.\n- For the prefix `gpt-3.5-turbo-`: `gpt-3.5-turbo-0301`, `gpt-3.5-turbo-0401`, etc.\n- For the Azure deployment name `gpt-35-turbo`.\n\nTo retrieve the encoding name based on a model name or its prefix, you can use the `GetEncodingNameForModel` method:\n\n```csharp\nstring encodingName = Model.GetEncodingNameForModel(\"claude-3.5-sonnet\"); // Returns \"claude\"\nstring encodingName = Model.GetEncodingNameForModel(\"gpt-4-0314\");        // Returns \"cl100k_base\"\n```\n\nIf the provided model name doesn't match any direct model names or prefixes, an exception is thrown.\n\n## Understanding Encoded Values\n\nWhen you encode a string using the Encode method, the returned value is a list of integers that represent tokens in the\nspecified encoding. These tokens are a compact way of representing the input text and can be processed more efficiently\nby various algorithms.\n\nFor example, encoding the text \"Hello world!\" using the cl100k_base encoding might produce the following list of\nintegers:\n\n```csharp\nvar encoded = cl100kBaseEncoding.Encode(\"Hello world!\"); // Output: [9906, 1917, 0]\n```\n\nYou can then use the `Decode` method to convert these tokenized integer values back into the original text:\n\n```csharp\nvar decoded = cl100kBaseEncoding.Decode(encoded); // Output: \"Hello world!\"\n```\n\nWith SharpToken, you can seamlessly switch between different encodings to find the one that best suits your needs. Just\nremember to use the same encoding for both the `Encode` and `Decode` methods to ensure accurate results.\n\n## Advanced usage\n\n### Custom Allowed Sets\n\nSharpToken allows you to specify custom sets of allowed special tokens when encoding text. To do this, pass a\nHashSet\u003cstring\u003e containing the allowed special tokens as a parameter to the Encode method:\n\n```csharp\nconst string encodingName = \"cl100k_base\";\nconst string inputText = \"Some Text \u003c|endofprompt|\u003e\";\nvar allowedSpecialTokens = new HashSet\u003cstring\u003e { \"\u003c|endofprompt|\u003e\" };\n\nvar encoding = GptEncoding.GetEncoding(encodingName);\nvar encoded = encoding.Encode(inputText, allowedSpecialTokens);\nvar expectedEncoded = new List\u003cint\u003e { 8538, 2991, 220, 100276 };\n\nAssert.Equal(expectedEncoded, encoded);\n```\n\n### Custom Disallowed Sets\n\nSimilarly, you can specify custom sets of disallowed special tokens when encoding text. Pass a `HashSet\u003cstring\u003e`\ncontaining the disallowed special tokens as a parameter to the Encode method:\n\n```csharp\nconst string encodingName = \"cl100k_base\";\nconst string inputText = \"Some Text\";\n\nvar encoding = GptEncoding.GetEncoding(encodingName);\n\nvoid TestAction()\n{\n    encoding.Encode(inputText, disallowedSpecial: new HashSet\u003cstring\u003e { \"Some\" });\n}\n\nAssert.Throws\u003cArgumentException\u003e(TestAction);\n```\n\nIn this example, an `ArgumentException` is thrown because the input text contains a disallowed special token\n\n## Testing and Validation\n\nSharpToken includes a set of test cases in the [TestPlans.txt](SharpToken.Tests/data/TestPlans.txt) file to ensure its\ncompatibility with the Python tiktoken library. These test cases validate the functionality and behavior of SharpToken,\nproviding a reliable reference for developers. Running the unit tests and verifying the test cases helps maintain\nconsistency between the C# SharpToken library and the original Python implementation.\n\n## Performance Compared to TiktokenSharp and TokenizerLib\n\nSharpToken is the fastest library with the lowest allocations!\n\n\u003cdetails\u003e\n\u003csummary\u003eBenchmark Code\u003c/summary\u003e\n\n```csharp\n[SimpleJob(RuntimeMoniker.Net60)]\n[SimpleJob(RuntimeMoniker.Net80)]\n[SimpleJob(RuntimeMoniker.Net471)]\n[RPlotExporter]\n[MemoryDiagnoser]\npublic class CompareBenchmark\n{\n    private GptEncoding _sharpToken;\n    private TikToken _tikToken;\n    private ITokenizer _tokenizer;\n    private Tokenizer _mlTokenizer;\n    private string _kLongText;\n\n    [GlobalSetup]\n    public async Task Setup()\n    {\n        _sharpToken = GptEncoding.GetEncoding(\"cl100k_base\");\n        _tikToken = await TikToken.GetEncodingAsync(\"cl100k_base\").ConfigureAwait(false);\n        _tokenizer = await TokenizerBuilder.CreateByModelNameAsync(\"gpt-4\").ConfigureAwait(false);\n        _kLongText = \"King Lear, one of Shakespeare's darkest and most savage plays, tells the story of the foolish and Job-like Lear, who divides his kingdom, as he does his affections, according to vanity and whim. Lear’s failure as a father engulfs himself and his world in turmoil and tragedy.\";\n    }\n\n    [Benchmark]\n    public int SharpToken()\n    {\n        var sum = 0;\n        for (var i = 0; i \u003c 10000; i++)\n        {\n            var encoded = _sharpToken.Encode(_kLongText);\n            var decoded = _sharpToken.Decode(encoded);\n            sum += decoded.Length;\n        }\n\n        return sum;\n    }\n\n    [Benchmark]\n    public int TiktokenSharp()\n    {\n        var sum = 0;\n        for (var i = 0; i \u003c 10000; i++)\n        {\n            var encoded = _tikToken.Encode(_kLongText);\n            var decoded = _tikToken.Decode(encoded);\n            sum += decoded.Length;\n        }\n\n        return sum;\n    }\n\n    [Benchmark]\n    public int TokenizerLib()\n    {\n        var sum = 0;\n        for (var i = 0; i \u003c 10000; i++)\n        {\n            var encoded = _tokenizer.Encode(_kLongText);\n            var decoded = _tokenizer.Decode(encoded.ToArray());\n            sum += decoded.Length;\n        }\n\n        return sum;\n    }\n\n    [Benchmark]\n    public int MLTokenizers()\n    {\n        var sum = 0;\n        for (var i = 0; i \u003c 10000; i++)\n        {\n            var encoded = _mlTokenizer.EncodeToIds(_kLongText);\n            var decoded = _mlTokenizer.Decode(encoded);\n            sum += decoded.Length;\n        }\n\n        return sum;\n    }\n}\n```\n\n\u003c/details\u003e\n\n```\nBenchmarkDotNet v0.13.9+228a464e8be6c580ad9408e98f18813f6407fb5a, Windows 11 (10.0.22631.3296)\n11th Gen Intel Core i9-11950H 2.60GHz, 1 CPU, 16 logical and 8 physical cores\n.NET SDK 9.0.100-preview.2.24157.14\n  [Host]               : .NET 8.0.3 (8.0.324.11423), X64 RyuJIT AVX2\n  .NET 6.0             : .NET 6.0.28 (6.0.2824.12007), X64 RyuJIT AVX2\n  .NET 8.0             : .NET 8.0.3 (8.0.324.11423), X64 RyuJIT AVX2\n  .NET Framework 4.7.1 : .NET Framework 4.8.1 (4.8.9181.0), X64 RyuJIT VectorSize=256\n```\n\n| Method           | Job                  | Runtime              |      Mean |    Error |    StdDev |    Median |       Gen0 |      Gen1 | Allocated |\n| ---------------- | -------------------- | -------------------- | --------: | -------: | --------: | --------: | ---------: | --------: | --------: |\n| **MLTokenizers** | .NET 8.0             | .NET 8.0             |  60.55 ms | 1.143 ms |  1.123 ms |  60.45 ms |  1000.0000 |         - |  13.12 MB |\n| **MLTokenizers** | .NET 6.0             | .NET 6.0             |  95.75 ms | 1.374 ms |  1.147 ms |  95.54 ms | 10500.0000 |         - | 126.19 MB |\n| **MLTokenizers** | .NET Framework 4.7.1 | .NET Framework 4.7.1 | 291.77 ms | 5.811 ms | 11.195 ms | 291.64 ms | 21000.0000 |         - | 127.33 MB |\n|                  |                      |                      |           |          |           |           |            |           |           |\n| _SharpToken_     | .NET 8.0             | .NET 8.0             |  87.78 ms | 1.700 ms |  1.590 ms |  87.34 ms |  1000.0000 |         - |  22.13 MB |\n| _SharpToken_     | .NET 6.0             | .NET 6.0             | 128.84 ms | 1.718 ms |  1.607 ms | 128.17 ms | 16250.0000 |  500.0000 | 196.31 MB |\n| _SharpToken_     | .NET Framework 4.7.1 | .NET Framework 4.7.1 | 356.21 ms | 6.843 ms | 10.854 ms | 355.09 ms | 34000.0000 | 1000.0000 | 204.39 MB |\n|                  |                      |                      |           |          |           |           |            |           |           |\n| _TokenizerLib_   | .NET 8.0             | .NET 8.0             | 109.26 ms | 2.082 ms |  4.482 ms | 107.90 ms | 18200.0000 |  600.0000 | 217.82 MB |\n| _TokenizerLib_   | .NET 6.0             | .NET 6.0             | 126.16 ms | 2.959 ms |  8.630 ms | 122.34 ms | 18000.0000 |  500.0000 | 217.82 MB |\n| _TokenizerLib_   | .NET Framework 4.7.1 | .NET Framework 4.7.1 | 374.71 ms | 7.374 ms | 16.794 ms | 370.12 ms | 40000.0000 | 1000.0000 | 243.79 MB |\n|                  |                      |                      |           |          |           |           |            |           |           |\n| _TiktokenSharp_  | .NET 8.0             | .NET 8.0             | 177.34 ms | 3.506 ms |  8.797 ms | 174.98 ms | 28000.0000 | 1000.0000 | 338.98 MB |\n| _TiktokenSharp_  | .NET 6.0             | .NET 6.0             | 196.17 ms | 3.912 ms |  8.422 ms | 195.52 ms | 26000.0000 |  666.6667 | 313.26 MB |\n| _TiktokenSharp_  | .NET Framework 4.7.1 | .NET Framework 4.7.1 | 488.22 ms | 9.696 ms | 15.931 ms | 487.17 ms | 63000.0000 | 1000.0000 | 378.31 MB |\n\n## Performance\n\nSharpToken is extreamly performance optimized on net8.0.\nIt uses modern multibyte CPU instructions and almost no heap allocations.\n\nAll core methods have been tested on a large and a small input text.\n\n**Inputs:**\n\n- `SmallText`: 453 B (text/plain)\n- `LargeText`: 51 KB (text/html)\n\n**Methods:**\n\n- `Encode`: text to tokens\n- `Decode`: tokens to text\n- `CountTokens`: high performance API to count tokens of text\n\n```\nBenchmarkDotNet v0.13.12, Windows 11 (10.0.22631.3296/23H2/2023Update/SunValley3)\nAMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores\n.NET SDK 8.0.200\n  [Host]               : .NET 8.0.2 (8.0.224.6711), X64 RyuJIT AVX2\n  .NET 6.0             : .NET 6.0.16 (6.0.1623.17311), X64 RyuJIT AVX2\n  .NET 8.0             : .NET 8.0.2 (8.0.224.6711), X64 RyuJIT AVX2\n  .NET Framework 4.7.1 : .NET Framework 4.8.1 (4.8.9181.0), X64 RyuJIT VectorSize=256\n```\n\n| Method                   |          Mean |       Error |      StdDev | Ratio | RatioSD | Allocated | Alloc Ratio |\n| ------------------------ | ------------: | ----------: | ----------: | ----: | ------: | --------: | ----------: |\n| **.NET 8.0**             |               |             |             |       |         |           |             |\n| Encode_SmallText         |     22.649 us |   0.4244 us |   0.4359 us |  0.28 |    0.01 |     696 B |        0.02 |\n| Encode_LargeText         |  4,542.505 us |  87.7988 us | 104.5182 us |  0.24 |    0.01 |  155547 B |        0.03 |\n|                          |               |             |             |       |         |           |             |\n| Decode_SmallText         |      1.623 us |   0.0324 us |   0.0373 us |  0.44 |    0.02 |    2320 B |        0.98 |\n| Decode_LargeText         |    454.570 us |   6.8980 us |   6.4524 us |  0.80 |    0.02 |  286979 B |        1.00 |\n|                          |               |             |             |       |         |           |             |\n| CountTokens_SmallText    |     22.008 us |   0.1165 us |   0.0909 us |  0.28 |    0.00 |     184 B |       0.005 |\n| CountTokens_LargeText    |  4,231.353 us |  14.5157 us |  11.3329 us |  0.23 |    0.00 |     195 B |       0.000 |\n|                          |               |             |             |       |         |           |             |\n| **.NET 6.0**             |               |             |             |       |         |           |             |\n| Encode_SmallText         |     36.370 us |   0.7178 us |   1.0962 us |  0.45 |    0.02 |   37344 B |        0.91 |\n| Encode_LargeText         | 11,213.070 us | 219.6291 us | 269.7243 us |  0.59 |    0.02 | 5062574 B |        0.91 |\n|                          |               |             |             |       |         |           |             |\n| Decode_SmallText         |      2.588 us |   0.0394 us |   0.0350 us |  0.70 |    0.02 |    2320 B |        0.98 |\n| Decode_LargeText         |    489.467 us |   8.9195 us |   8.3433 us |  0.86 |    0.02 |  286985 B |        1.00 |\n|                          |               |             |             |       |         |           |             |\n| CountTokens_SmallText    |     34.758 us |   0.2027 us |   0.1896 us |  0.45 |    0.01 |   36832 B |       0.907 |\n| CountTokens_LargeText    | 11,252.083 us | 215.8912 us | 212.0340 us |  0.61 |    0.01 | 4907169 B |       0.907 |\n|                          |               |             |             |       |         |           |             |\n| **.NET Framework 4.7.1** |               |             |             |       |         |           |             |\n| Encode_SmallText         |     79.947 us |   1.5621 us |   3.0097 us |  1.00 |    0.00 |   41138 B |        1.00 |\n| Encode_LargeText         | 18,961.252 us | 253.1816 us | 236.8262 us |  1.00 |    0.00 | 5567685 B |        1.00 |\n|                          |               |             |             |       |         |           |             |\n| Decode_SmallText         |      3.723 us |   0.0728 us |   0.0997 us |  1.00 |    0.00 |    2375 B |        1.00 |\n| Decode_LargeText         |    570.787 us |  11.0356 us |  11.8080 us |  1.00 |    0.00 |  287496 B |        1.00 |\n|                          |               |             |             |       |         |           |             |\n| CountTokens_SmallText    |     77.521 us |   1.0802 us |   0.9020 us |  1.00 |    0.00 |   40616 B |       1.000 |\n| CountTokens_LargeText    | 18,485.392 us | 313.5834 us | 277.9836 us |  1.00 |    0.00 | 5413237 B |       1.000 |\n\n## Contributions and Feedback\n\nIf you encounter any issues or have suggestions for improvements, please feel free to open an issue or submit a pull\nrequest on the project's repository.\n\nHope you find SharpToken useful for your projects and welcome any feedback you may have.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmitry-brazhenko%2FSharpToken","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmitry-brazhenko%2FSharpToken","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmitry-brazhenko%2FSharpToken/lists"}