{"id":18417328,"url":"https://github.com/gfoidl/base64","last_synced_at":"2025-04-07T12:32:21.119Z","repository":{"id":41063249,"uuid":"158291268","full_name":"gfoidl/Base64","owner":"gfoidl","description":"Base64 encoding / decoding with SIMD-support, also base64Url","archived":false,"fork":false,"pushed_at":"2023-05-22T15:09:57.000Z","size":1615,"stargazers_count":60,"open_issues_count":1,"forks_count":7,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-22T18:41:26.656Z","etag":null,"topics":["base64","base64-decoding","base64-encoding","base64url","simd"],"latest_commit_sha":null,"homepage":"https://gfoidl.github.io/Base64","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gfoidl.png","metadata":{"files":{"readme":"ReadMe.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null},"funding":{"github":["gfoidl"]}},"created_at":"2018-11-19T21:17:19.000Z","updated_at":"2024-12-18T04:55:25.000Z","dependencies_parsed_at":"2023-01-30T20:45:56.744Z","dependency_job_id":null,"html_url":"https://github.com/gfoidl/Base64","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gfoidl%2FBase64","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gfoidl%2FBase64/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gfoidl%2FBase64/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gfoidl%2FBase64/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gfoidl","download_url":"https://codeload.github.com/gfoidl/Base64/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247653210,"owners_count":20973783,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["base64","base64-decoding","base64-encoding","base64url","simd"],"created_at":"2024-11-06T04:08:58.378Z","updated_at":"2025-04-07T12:32:16.098Z","avatar_url":"https://github.com/gfoidl.png","language":"C#","funding_links":["https://github.com/sponsors/gfoidl"],"categories":[],"sub_categories":[],"readme":"| CI | NuGet |\n| -- | -- |\n| [![Build Status](https://dev.azure.com/gh-gfoidl/github-Projects/_apis/build/status/github-Projects-CI)](https://dev.azure.com/gh-gfoidl/github-Projects/_build/latest?definitionId=5)| [![NuGet](https://img.shields.io/nuget/v/gfoidl.Base64.svg?style=flat-square)](https://www.nuget.org/packages/gfoidl.Base64/) |\n\n# gfoidl.Base64\n\nA .NET library for base64 encoding / decoding, as well as base64Url support.\nEncoding can be done to buffers of type `byte` (for UTF-8) or `char`.\nDecoding can read from buffers of type `byte` (for UTF-8) or `char`.\n\nEncoding / decoding supports buffer-chains, for example for very large data or when the data arrives in chunks.\n\nIn .NET Core 3.0[^net3] onwards encoding / decoding is done with SIMD-support:\n\n| Framework | scalar | SSSE3 | AVX2 | arm64 |\n| -- | -- | -- | -- | -- |\n| .NET 7 | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |\n| .NET Core 3.0 | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |\n| .NET Standard 2.0 / .NET 4.5 | :heavy_check_mark: | :x: | :x: | :x: |\n\nIf available AVX will \"eat\" up as much as possible, then SSE will \"eat\" up as much as possible,\nfinally scalar code processes the rest (including padding).\n\n## Usage\n\nBasically the entry to encoder / decoder is `Base64.Default` for _base64_, and `Base64.Url` for _base64Url_.\n\nSee [demo](./demo/gfoidl.Base64.Demo/Program.cs) for further examples.\n\n### Encoding\n\n```c#\nbyte[] guid = Guid.NewGuid().ToByteArray();\n\nstring guidBase64     = Base64.Default.Encode(guid);\nstring guidBases64Url = Base64.Url.Encode(guid);\n```\n\nor `Span\u003cbyte\u003e` based (for UTF-8 encoded output):\n\n```c#\nint guidBase64EncodedLength = Base64.Default.GetEncodedLength(guid.Length);\nSpan\u003cbyte\u003e guidBase64UTF8   = stackalloc byte[guidBase64EncodedLength];\nOperationStatus status      = Base64.Default.Encode(guid, guidBase64UTF8, out int consumed, out int written);\n\nint guidBase64UrlEncodedLength = Base64.Url.GetEncodedLength(guid.Length);\nSpan\u003cbyte\u003e guidBase64UrlUTF8   = stackalloc byte[guidBase64UrlEncodedLength];\nstatus                         = Base64.Url.Encode(guid, guidBase64UrlUTF8, out consumed, out written);\n```\n\n### Decoding\n\n```c#\nGuid guid = Guid.NewGuid();\n\nstring guidBase64    = Convert.ToBase64String(guid.ToByteArray());\nstring guidBase64Url = guidBase64.Replace('+', '-').Replace('/', '_').TrimEnd('=');\n\nbyte[] guidBase64Decoded    = Base64.Default.Decode(guidBase64);\nbyte[] guidBase64UrlDecoded = Base64.Url.Decode(guidBase64Url);\n```\n\nor `Span\u003cchar\u003e` based:\n\n```c#\nint guidBase64DecodedLen    = Base64.Default.GetDecodedLength(guidBase64);\nint guidBase64UrlDecodedLen = Base64.Url.GetDecodedLength(guidBase64Url);\n\nSpan\u003cbyte\u003e guidBase64DecodedBuffer    = stackalloc byte[guidBase64DecodedLen];\nSpan\u003cbyte\u003e guidBase64UrlDecodedBuffer = stackalloc byte[guidBase64UrlDecodedLen];\n\nOperationStatus status = Base64.Default.Decode(guidBase64, guidBase64DecodedBuffer, out int consumed, out int written);\nstatus                 = Base64.Url.Decode(guidBase64Url, guidBase64UrlDecodedBuffer, out consumed, out written);\n```\n\n### Buffer chains\n\nBuffer chains are handy when for encoding / decoding\n\n* very large data\n* data arrives is chunks, e.g. by reading from a (buffered) stream / pipeline\n* the size of data is initially unknown\n* ...\n\n```c#\nvar rnd         = new Random();\nSpan\u003cbyte\u003e data = new byte[1000];\nrnd.NextBytes(data);\n\n// exact length could be computed by Base64.Default.GetEncodedLength, here for demo exzessive size\nSpan\u003cchar\u003e base64 = new char[5000];\n\nOperationStatus status = Base64.Default.Encode(data.Slice(0, 400), base64, out int consumed, out int written, isFinalBlock: false);\nstatus                 = Base64.Default.Encode(data.Slice(consumed), base64.Slice(written), out consumed, out int written1, isFinalBlock: true);\n\nbase64 = base64.Slice(0, written + written1);\n\nSpan\u003cbyte\u003e decoded = new byte[5000];\nstatus             = Base64.Default.Decode(base64.Slice(0, 100), decoded, out consumed, out written, isFinalBlock: false);\nstatus             = Base64.Default.Decode(base64.Slice(consumed), decoded.Slice(written), out consumed, out written1, isFinalBlock: true);\n\ndecoded = decoded.Slice(0, written + written1);\n```\n\n### ReadOnlySequence / IBufferWriter\n\nEncoding / decoding with `ReadOnlySequence\u003cbyte\u003e` and `IBufferWriter\u003cbyte\u003e` can be used together with `System.IO.Pipelines`.\n\n```c#\nvar pipeOptions = PipeOptions.Default;\nvar pipe        = new Pipe(pipeOptions);\n\nvar rnd  = new Random(42);\nvar data = new byte[4097];\nrnd.NextBytes(data);\n\npipe.Writer.Write(data);\nawait pipe.Writer.CompleteAsync();\n\nReadResult readResult = await pipe.Reader.ReadAsync();\n\nvar resultPipe = new Pipe();\nBase64.Default.Encode(readResult.Buffer, resultPipe.Writer, out long consumed, out long written);\nawait resultPipe.Writer.FlushAsync();\nawait resultPipe.Writer.CompleteAsync();\n\npipe.Reader.AdvanceTo(readResult.Buffer.End);\n```\n\n## (Functional) Comparison to classes in .NET\n\n### General\n\n.NET provides the classes [System.Convert](https://docs.microsoft.com/en-us/dotnet/api/system.convert) and [System.Buffers.Text.Base64](https://docs.microsoft.com/en-us/dotnet/api/system.buffers.text.base64)\nfor base64 operations.\n\nbase64Url isn't supported, so hacky solutions like\n```c#\nstring base64    = Convert.ToBase64String(data);\nstring base64Url = base64.Replace('+', '-').Replace('/', '_').TrimEnd('=');\n```\nare needed. This isn't ideal, as there are avoidable allocations and several iterations over the encoded string (see [here](perf/gfoidl.Base64.Benchmarks/results/netcoreapp3.1/EncodeStringUrlBenchmark-report.md) and [here](perf/gfoidl.Base64.Benchmarks/results/netcoreapp3.1/DecodeStringUrlBenchmark-report.md) for benchmark results).\n\n_gfoidl.Base64_ supports encoding / decoding to / from base64Url in a direct way.\nEncoding `byte[] -\u003e byte[]` for UTF-8 is supported, as well as `byte[] -\u003e char[]`.\nDecoding `byte[] -\u003e byte[]` for UTF-8 is supported, as well as `char[] -\u003e byte[]`.\n\n~Further SIMD isn't utilized in the .NET classes.~ SIMD is also supported in the .NET classes.\n(Note: I've opened an [issue](https://github.com/dotnet/corefx/issues/32365) to add SIMD-support to these classes).\n\n### Convert.ToBase64XYZ / Convert.FromBase64XYZ\n\nThese methods only support `byte[] -\u003e char[]` as types for encoding,\nand `char[] -\u003e byte[]` as types for decoding, where `char[]` can also be `string` or `(ReadOnly)Span\u003cchar\u003e`.\n\nTo support UTF-8 another method call like\n```c#\nbyte[] utf8Encoded = Encoding.ASCII.GetBytes(base64String);\n```\nis needed.\n\nAn potential advantage of this class is that it allows the insertion of line-breaks (cf. [Base64FormattingOptions.InsertLineBreaks](https://docs.microsoft.com/en-us/dotnet/api/system.base64formattingoptions)).\n\n### System.Buffers.Text.Base64\n\nThis class only supports `byte[] -\u003e byte[]` for encoding / decoding. So in order to get a `string`\n`Encoding` has to be used.\n\nAn potential advantage of this class is the support for in-place encoding / decoding (cf.\n[Base64.EncodeToUtf8InPlace](https://docs.microsoft.com/en-us/dotnet/api/system.buffers.text.base64.encodetoutf8inplace),\n[Base64.DecodeFromUtf8InPlace](https://docs.microsoft.com/en-us/dotnet/api/system.buffers.text.base64.decodefromutf8inplace)\n)\n\n## Benchmarks\n\nFor old (.NET Core 3.1) benchmarks see [results](/perf/gfoidl.Base64.Benchmarks/results).\nBenchmarks are run regularly [on CI](https://dev.azure.com/gh-gfoidl/github-Projects/_build?definitionId=47), see there for current run-results (in the artifacts).\n\nPerformance gain depends, among a lot of other things, on the workload size, so here no table with superior results will be shown.\n\n[Direct encoding to a string](perf/gfoidl.Base64.Benchmarks/results/netcoreapp3.1/EncodeStringBenchmark-report.md) is for small inputs slower than `Convert.ToBase64String` (has less overhead, and can write to string-buffer in a direct way).\nBut the larger the workload, the better this library works.\n\n[Direct decoding from a string](perf/gfoidl.Base64.Benchmarks/results/netcoreapp3.1/DecodeStringBenchmark-report.md) is generally (a lot) faster than `Convert.ConvertFromBase64CharArray`, also depending on workload size, but in the benchmark the speedup is from 1.5 to 12x.\n\n**Note:** please measure / profile in your real usecase, as this are just micro-benchmarks.\n\n\n## Acknowledgements\n\nThe scalar version of the base64 encoding / decoding is based on [System.Buffers.Text.Base64](https://github.com/dotnet/corefx/tree/9c68db7fb016c6c9ae4d0f6152798d7ab1e38a37/src/System.Memory/src/System/Buffers/Text).\n\nThe scalar version of the base64Url encoding / decoding is based on https://github.com/aspnet/Extensions/pull/334 and https://github.com/aspnet/Extensions/pull/338.\n\nVectorized versions (SSE, AVX) for base64 encoding / decoding is based on https://github.com/aklomp/base64 (see also _Acknowledgements_ in that repository).\n\nVectorized versions (SSE, AVX) for base64Url encoding / decoding is based on https://github.com/aklomp/base64 (see _Acknowledgements_ in that repository).\nFor decoding (SSE, AVX) code is based on [Vector lookup (pshufb)](http://0x80.pl/notesen/2016-01-17-sse-base64-decoding.html#vector-lookup-pshufb) by Wojciech Mula.\n\n\n## Development channel\n\nTo get packages from the development channel use a `nuget.config` similar to this one:\n```xml\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\n\u003cconfiguration\u003e\n    \u003cpackageSources\u003e\n        \u003cadd key=\"gfoidl-public\" value=\"https://pkgs.dev.azure.com/gh-gfoidl/github-Projects/_packaging/gfoidl-public/nuget/v3/index.json\" /\u003e\n    \u003c/packageSources\u003e\n\u003c/configuration\u003e\n```\n\n[^net3]: For targets pre .NET 7 use the _v1.x_ versions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgfoidl%2Fbase64","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgfoidl%2Fbase64","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgfoidl%2Fbase64/lists"}