{"id":26175658,"url":"https://github.com/itn3000/pooledstream","last_synced_at":"2025-10-09T21:11:17.511Z","repository":{"id":22504558,"uuid":"96448846","full_name":"itn3000/PooledStream","owner":"itn3000","description":"MemoryStream with ArrayPool","archived":false,"fork":false,"pushed_at":"2022-03-03T10:20:59.000Z","size":72,"stargazers_count":12,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-13T19:11:39.952Z","etag":null,"topics":["csharp","stream"],"latest_commit_sha":null,"homepage":null,"language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/itn3000.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-06T16:09:35.000Z","updated_at":"2024-07-07T10:31:54.000Z","dependencies_parsed_at":"2022-07-26T03:02:07.896Z","dependency_job_id":null,"html_url":"https://github.com/itn3000/PooledStream","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itn3000%2FPooledStream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itn3000%2FPooledStream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itn3000%2FPooledStream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itn3000%2FPooledStream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/itn3000","download_url":"https://codeload.github.com/itn3000/PooledStream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248964267,"owners_count":21190498,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csharp","stream"],"created_at":"2025-03-11T20:57:21.749Z","updated_at":"2025-10-09T21:11:12.460Z","avatar_url":"https://github.com/itn3000.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Overview\n\n[![NuGet version](https://badge.fury.io/nu/PooledStream.svg)](https://badge.fury.io/nu/PooledStream)\n\nthis library aims to efficient MemoryStream for large data.\n\n# Usage\n\nYou can add reference as [NuGet package](https://www.nuget.org/packages/PooledStream/).\nOnce you add the reference, you can use PooledStream.PooledMemoryStream.\n\n## Code Examples of PooledMemoryStream\n\n### Basic usage\n\n```csharp\n// you can create stream with no parameter(using default setting).\n// you can specify ArrayPool instance and initial capacity.\nusing(var stm = new PooledStream.PooledMemoryStream())\n{\n  // you can ensure internal buffer length before write to stream.\n  stm.Reserve(1024);\n  var wdata = new byte[128];\n  // write stream\n  stm.Write(wdata, 0, wdata.Length);\n  // you can get written data by ToArray()\n  var data = stm.ToArray();\n  // or you can get ArraySegment without allocate new data.\n  // WARNING: you must not use the returned ArraySegment after dispose Stream \n  var segment = stm.ToUnsafeArraySegment();\n}\n```\n\n### Readonly stream\n\n```csharp\nvar data = new byte[128];\n// if you specify bytearray to constructor parameter,\n// PooledMemoryStream will be created as readonly stream.\nusing(var stm = new PooledStream.PooledMemoryStream(data))\n{\n  var buf = new byte[32];\n  stm.Read(buf, 0, buf.Length);\n  // if you try to write, InvalidOperationException will be thrown\n  stm.Write(buf, 0, buf.Length);\n}\n```\n\n### Reusing Stream(Advanced Usage)\n\nfor avoiding allocation completely, [Microsoft.Extensions.ObjectPool](https://www.nuget.org/packages/Microsoft.Extensions.ObjectPool/) is useful.\ncode example:\n\n```csharp\n// using Microsoft.Extensions.ObjectPool;\n// using PooledStream;\n\n// initializing object pool instance.This should be singleton.\nObjectPool\u003cPooledMemoryStream\u003e pool = ObjectPool.Create\u003cPooledMemoryStream\u003e();\nvar stm = pool.Get();\n// initializing stream(buffer initialization included)\nstm.SetLength(0);\ntry\n{\n  // using stream...\n  ...\n}\nfinally\n{\n  // returning instance\n  pool.Return(stm);\n}\n```\n\n#### WARNING\n\n* **You must call `SetLength(0)` at first, and call `Return(stm)` in the end for preventing memory leak.**\n* **`PooledMemoryStream` is not threadsafe, so you must not share instance across threads**\n\n## Code examples of PooledMemoryBufferWriter\n\n### Basic\n\n```csharp\n// using PooledStream;\n\n// create instance\nusing var bw = new PooledMemoryBufferWriter\u003cbyte\u003e();\nvar sp = bw.GetSpan(8);\nsp.Fill(1);\nbw.Advance(8);\n// get buffer as Span\u003cbyte\u003e, you MUST NOT use it outside of bufferwriter lifetime.\nvar rsp = bw.ToSpanUnsafe();\n/// output \"1,1,1,1,1,1,1,1\"\nConsole.WriteLine(\"{0}\", string.Join(',', rsp.ToArray()));\n// reset buffer status\nbw.Reset();\nrsp = bw.ToSpanUnsafe();\n// output empty line\nConsole.WriteLine(\"{0}\", string.Join(',', rsp.ToArray()));\n```\n\n### Reusing(Advanced usage)\n\n```csharp\n// using Microsoft.Extensions.ObjectPool;\n// using PooledStream;\n\n// initializing object pool instance.This should be singleton.\n// PooledMemoryBufferWriter is not threadsafe.\nObjectPool\u003cPooledMemoryBufferWriter\u003e pool = ObjectPool.Create\u003cPooledMemoryBufferWriter\u003e();\nvar bw = pool.Get();\ntry\n{\n  // initializing buffer\n  bw.Reset();\n  // using buffer writer...\n  ...\n}\nfinally\n{\n  // returning instance\n  pool.Return(bw);\n}\n```\n\n* you MUST initialize buffer by `Reset()` at first\n* you MUST return instance to ObjectPool after using it\n\n# Micro benchmark result(powered by [BenchmarkDotNet](http://benchmarkdotnet.org/))\n\n## Comparison of single thread performance\n\n``` ini\n\nBenchmarkDotNet=v0.13.1, OS=Windows 10.0.19044.1466 (21H2)\nIntel Core i7-9700 CPU 3.00GHz, 1 CPU, 8 logical and 8 physical cores\n.NET SDK=6.0.101\n  [Host]     : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT\n  Job-CJIONZ : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT\n  Job-ERJOQR : .NET Core 3.1.22 (CoreCLR 4.700.21.56803, CoreFX 4.700.21.57101), X64 RyuJIT\n\nIterationCount=3  WarmupCount=3  \n\n```\n|               Method |        Job |     Toolchain | DataSize | MaxLoop |        Mean |        Error |    StdDev | Ratio | RatioSD |      Gen 0 |      Gen 1 |   Gen 2 |  Allocated |\n|--------------------- |----------- |-------------- |--------- |-------- |------------:|-------------:|----------:|------:|--------:|-----------:|-----------:|--------:|-----------:|\n|     **NormalStreamTest** | **Job-CJIONZ** |      **.NET 6.0** |      **100** |   **10000** |    **352.1 μs** |    **260.08 μs** |  **14.26 μs** |  **1.00** |    **0.00** |   **548.3398** |     **0.4883** |       **-** |   **3,360 KB** |\n|    PooledStreamBench | Job-CJIONZ |      .NET 6.0 |      100 |   10000 |    362.2 μs |     61.72 μs |   3.38 μs |  1.03 |    0.03 |    88.8672 |          - |       - |     547 KB |\n| RecyclableStreamTest | Job-CJIONZ |      .NET 6.0 |      100 |   10000 | 12,224.8 μs |    283.90 μs |  15.56 μs | 34.76 |    1.36 |   687.5000 |    46.8750 | 31.2500 |   4,353 KB |\n|       ObjectPoolTest | Job-CJIONZ |      .NET 6.0 |      100 |   10000 |    754.0 μs |    562.71 μs |  30.84 μs |  2.15 |    0.17 |   101.5625 |          - |       - |     625 KB |\n|                      |            |               |          |         |             |              |           |       |         |            |            |         |            |\n|     NormalStreamTest | Job-ERJOQR | .NET Core 3.1 |      100 |   10000 |    388.8 μs |    198.45 μs |  10.88 μs |  1.00 |    0.00 |   561.0352 |     0.9766 |       - |   3,438 KB |\n|    PooledStreamBench | Job-ERJOQR | .NET Core 3.1 |      100 |   10000 |    403.8 μs |     76.37 μs |   4.19 μs |  1.04 |    0.04 |   101.5625 |          - |       - |     625 KB |\n| RecyclableStreamTest | Job-ERJOQR | .NET Core 3.1 |      100 |   10000 | 12,498.6 μs |  1,357.66 μs |  74.42 μs | 32.16 |    0.71 |   703.1250 |    46.8750 | 31.2500 |   4,431 KB |\n|       ObjectPoolTest | Job-ERJOQR | .NET Core 3.1 |      100 |   10000 |    785.7 μs |    179.92 μs |   9.86 μs |  2.02 |    0.08 |   101.5625 |          - |       - |     625 KB |\n|                      |            |               |          |         |             |              |           |       |         |            |            |         |            |\n|     **NormalStreamTest** | **Job-CJIONZ** |      **.NET 6.0** |     **1000** |   **10000** |    **732.8 μs** |    **264.67 μs** |  **14.51 μs** |  **1.00** |    **0.00** |  **1734.3750** |    **10.7422** |       **-** |  **10,626 KB** |\n|    PooledStreamBench | Job-CJIONZ |      .NET 6.0 |     1000 |   10000 |    513.3 μs |    465.38 μs |  25.51 μs |  0.70 |    0.05 |    88.8672 |          - |       - |     548 KB |\n| RecyclableStreamTest | Job-CJIONZ |      .NET 6.0 |     1000 |   10000 | 11,926.3 μs |    816.69 μs |  44.77 μs | 16.28 |    0.33 |   687.5000 |    46.8750 | 31.2500 |   4,354 KB |\n|       ObjectPoolTest | Job-CJIONZ |      .NET 6.0 |     1000 |   10000 |    839.0 μs |    191.49 μs |  10.50 μs |  1.15 |    0.03 |   101.5625 |          - |       - |     626 KB |\n|                      |            |               |          |         |             |              |           |       |         |            |            |         |            |\n|     NormalStreamTest | Job-ERJOQR | .NET Core 3.1 |     1000 |   10000 |    769.3 μs |     81.86 μs |   4.49 μs |  1.00 |    0.00 |  1747.0703 |    10.7422 |       - |  10,704 KB |\n|    PooledStreamBench | Job-ERJOQR | .NET Core 3.1 |     1000 |   10000 |    586.8 μs |    495.79 μs |  27.18 μs |  0.76 |    0.04 |   101.5625 |          - |       - |     626 KB |\n| RecyclableStreamTest | Job-ERJOQR | .NET Core 3.1 |     1000 |   10000 | 12,806.1 μs |  2,418.51 μs | 132.57 μs | 16.65 |    0.26 |   703.1250 |    46.8750 | 31.2500 |   4,432 KB |\n|       ObjectPoolTest | Job-ERJOQR | .NET Core 3.1 |     1000 |   10000 |    914.8 μs |     73.68 μs |   4.04 μs |  1.19 |    0.01 |   101.5625 |          - |       - |     626 KB |\n|                      |            |               |          |         |             |              |           |       |         |            |            |         |            |\n|     **NormalStreamTest** | **Job-CJIONZ** |      **.NET 6.0** |    **50000** |   **10000** | **27,280.8 μs** |  **2,486.59 μs** | **136.30 μs** |  **1.00** |    **0.00** | **79312.5000** | **10312.5000** |       **-** | **489,190 KB** |\n|    PooledStreamBench | Job-CJIONZ |      .NET 6.0 |    50000 |   10000 |  9,201.1 μs |  2,050.28 μs | 112.38 μs |  0.34 |    0.01 |    93.7500 |          - |       - |     596 KB |\n| RecyclableStreamTest | Job-CJIONZ |      .NET 6.0 |    50000 |   10000 | 22,549.4 μs |  7,507.94 μs | 411.54 μs |  0.83 |    0.01 |   687.5000 |   125.0000 | 31.2500 |   4,402 KB |\n|       ObjectPoolTest | Job-CJIONZ |      .NET 6.0 |    50000 |   10000 | 10,864.5 μs |  1,082.43 μs |  59.33 μs |  0.40 |    0.00 |   109.3750 |    15.6250 |       - |     674 KB |\n|                      |            |               |          |         |             |              |           |       |         |            |            |         |            |\n|     NormalStreamTest | Job-ERJOQR | .NET Core 3.1 |    50000 |   10000 | 27,684.2 μs |  2,307.66 μs | 126.49 μs |  1.00 |    0.00 | 79343.7500 | 10437.5000 | 31.2500 | 489,268 KB |\n|    PooledStreamBench | Job-ERJOQR | .NET Core 3.1 |    50000 |   10000 |  9,232.5 μs |  2,070.70 μs | 113.50 μs |  0.33 |    0.01 |   109.3750 |    15.6250 |       - |     674 KB |\n| RecyclableStreamTest | Job-ERJOQR | .NET Core 3.1 |    50000 |   10000 | 23,480.2 μs | 13,114.22 μs | 718.83 μs |  0.85 |    0.03 |   687.5000 |   125.0000 | 31.2500 |   4,480 KB |\n|       ObjectPoolTest | Job-ERJOQR | .NET Core 3.1 |    50000 |   10000 | 10,904.2 μs |  1,730.91 μs |  94.88 μs |  0.39 |    0.00 |   109.3750 |    15.6250 |       - |     674 KB |\n\n## Comparison of multithreaded performance\n\n``` ini\n\nBenchmarkDotNet=v0.13.1, OS=Windows 10.0.19044.1466 (21H2)\nIntel Core i7-9700 CPU 3.00GHz, 1 CPU, 8 logical and 8 physical cores\n.NET SDK=6.0.101\n  [Host]     : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT\n  Job-DVURFQ : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT\n  Job-ZKBPJE : .NET Core 3.1.22 (CoreCLR 4.700.21.56803, CoreFX 4.700.21.57101), X64 RyuJIT\n\nIterationCount=3  WarmupCount=3  \n\n```\n|                   Method |        Job |     Toolchain | ParallelNum | DataSize | MaxLoop |         Mean |       Error |    StdDev |  Ratio | RatioSD |     Gen 0 |   Gen 1 | Allocated |\n|------------------------- |----------- |-------------- |------------ |--------- |-------- |-------------:|------------:|----------:|-------:|--------:|----------:|--------:|----------:|\n|     **NormalStreamParallel** | **Job-DVURFQ** |      **.NET 6.0** |           **5** |     **1000** |   **10000** |     **832.3 μs** |    **91.85 μs** |   **5.03 μs** |   **1.00** |    **0.00** | **1734.3750** | **11.7188** |     **10 MB** |\n|     PooledStreamParallel | Job-DVURFQ |      .NET 6.0 |           5 |     1000 |   10000 |   1,461.6 μs | 1,099.12 μs |  60.25 μs |   1.76 |    0.08 |  410.1563 |  3.9063 |      2 MB |\n|       ObjectPoolParallel | Job-DVURFQ |      .NET 6.0 |           5 |     1000 |   10000 |   4,915.4 μs | 1,044.02 μs |  57.23 μs |   5.91 |    0.10 |  507.8125 |       - |      3 MB |\n| RecyclableStreamParallel | Job-DVURFQ |      .NET 6.0 |           5 |     1000 |   10000 |  59,377.8 μs | 2,245.30 μs | 123.07 μs |  71.35 |    0.39 | 3444.4444 |       - |     21 MB |\n|                          |            |               |             |          |         |              |             |           |        |         |           |         |           |\n|     NormalStreamParallel | Job-ZKBPJE | .NET Core 3.1 |           5 |     1000 |   10000 |     883.1 μs |   308.71 μs |  16.92 μs |   1.00 |    0.00 | 1747.0703 | 11.7188 |     10 MB |\n|     PooledStreamParallel | Job-ZKBPJE | .NET Core 3.1 |           5 |     1000 |   10000 |   2,520.1 μs |   982.67 μs |  53.86 μs |   2.85 |    0.07 |  421.8750 |  3.9063 |      3 MB |\n|       ObjectPoolParallel | Job-ZKBPJE | .NET Core 3.1 |           5 |     1000 |   10000 |   5,142.7 μs |   271.51 μs |  14.88 μs |   5.82 |    0.10 |  507.8125 |       - |      3 MB |\n| RecyclableStreamParallel | Job-ZKBPJE | .NET Core 3.1 |           5 |     1000 |   10000 |  63,350.5 μs | 3,453.86 μs | 189.32 μs |  71.75 |    1.56 | 3500.0000 |       - |     21 MB |\n|                          |            |               |             |          |         |              |             |           |        |         |           |         |           |\n|     **NormalStreamParallel** | **Job-DVURFQ** |      **.NET 6.0** |          **10** |     **1000** |   **10000** |     **831.3 μs** |   **318.09 μs** |  **17.44 μs** |   **1.00** |    **0.00** | **1734.3750** |  **9.7656** |     **10 MB** |\n|     PooledStreamParallel | Job-DVURFQ |      .NET 6.0 |          10 |     1000 |   10000 |   1,528.2 μs | 2,194.38 μs | 120.28 μs |   1.84 |    0.11 |  410.1563 |  5.8594 |      2 MB |\n|       ObjectPoolParallel | Job-DVURFQ |      .NET 6.0 |          10 |     1000 |   10000 |   9,440.5 μs |   984.44 μs |  53.96 μs |  11.36 |    0.30 | 1015.6250 |       - |      6 MB |\n| RecyclableStreamParallel | Job-DVURFQ |      .NET 6.0 |          10 |     1000 |   10000 | 118,121.0 μs | 4,079.13 μs | 223.59 μs | 142.14 |    2.73 | 6800.0000 |       - |     41 MB |\n|                          |            |               |             |          |         |              |             |           |        |         |           |         |           |\n|     NormalStreamParallel | Job-ZKBPJE | .NET Core 3.1 |          10 |     1000 |   10000 |     839.9 μs |    91.10 μs |   4.99 μs |   1.00 |    0.00 | 1747.0703 | 11.7188 |     10 MB |\n|     PooledStreamParallel | Job-ZKBPJE | .NET Core 3.1 |          10 |     1000 |   10000 |   2,078.9 μs |    72.45 μs |   3.97 μs |   2.48 |    0.01 |  421.8750 |  3.9063 |      3 MB |\n|       ObjectPoolParallel | Job-ZKBPJE | .NET Core 3.1 |          10 |     1000 |   10000 |  11,123.8 μs | 5,348.25 μs | 293.16 μs |  13.24 |    0.32 | 1015.6250 |       - |      6 MB |\n| RecyclableStreamParallel | Job-ZKBPJE | .NET Core 3.1 |          10 |     1000 |   10000 | 126,562.0 μs | 6,505.48 μs | 356.59 μs | 150.68 |    0.85 | 7000.0000 |       - |     42 MB |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitn3000%2Fpooledstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fitn3000%2Fpooledstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitn3000%2Fpooledstream/lists"}