{"id":15063886,"url":"https://github.com/bodigrim/linear-builder","last_synced_at":"2025-04-13T10:52:06.322Z","repository":{"id":62436412,"uuid":"454931051","full_name":"Bodigrim/linear-builder","owner":"Bodigrim","description":"Strict Text and ByteString builder, which hides mutable buffer behind linear types and takes amortized linear time.","archived":false,"fork":false,"pushed_at":"2024-08-18T23:17:50.000Z","size":173,"stargazers_count":91,"open_issues_count":1,"forks_count":4,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-17T13:11:38.898Z","etag":null,"topics":["buffer","builder","bytestring","concatenation","haskell","linear-types","strict-types","string-buffer","string-builder","string-manipulation","stringbuffer","stringbuilder","text","unicode"],"latest_commit_sha":null,"homepage":"https://hackage.haskell.org/package/text-builder-linear","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Bodigrim.png","metadata":{"files":{"readme":"README.md","changelog":"changelog.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-02T20:50:14.000Z","updated_at":"2025-02-26T20:28:24.000Z","dependencies_parsed_at":"2023-02-16T01:16:05.741Z","dependency_job_id":"4b6ea538-a3fe-46f6-aacb-28b899b22632","html_url":"https://github.com/Bodigrim/linear-builder","commit_stats":{"total_commits":107,"total_committers":1,"mean_commits":107.0,"dds":0.0,"last_synced_commit":"8f4dab0ddc89655b44c98767cf66e6d868651a9d"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bodigrim%2Flinear-builder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bodigrim%2Flinear-builder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bodigrim%2Flinear-builder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bodigrim%2Flinear-builder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Bodigrim","download_url":"https://codeload.github.com/Bodigrim/linear-builder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245767351,"owners_count":20668826,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["buffer","builder","bytestring","concatenation","haskell","linear-types","strict-types","string-buffer","string-builder","string-manipulation","stringbuffer","stringbuilder","text","unicode"],"created_at":"2024-09-25T00:08:28.432Z","updated_at":"2025-04-13T10:52:06.316Z","avatar_url":"https://github.com/Bodigrim.png","language":"Haskell","readme":"# text-builder-linear [![Hackage](http://img.shields.io/hackage/v/text-builder-linear.svg)](https://hackage.haskell.org/package/text-builder-linear) [![Stackage LTS](http://stackage.org/package/text-builder-linear/badge/lts)](http://stackage.org/lts/package/text-builder-linear) [![Stackage Nightly](http://stackage.org/package/text-builder-linear/badge/nightly)](http://stackage.org/nightly/package/text-builder-linear)\n\n_Linear types for linear times!_\n\nBuilder for strict `Text` and `ByteString`, based on linear types. It consistently\noutperforms lazy `Builder` from `text` as well as a strict builder from `text-builder`,\nand scales better.\n\n## Example\n\n```haskell\n\u003e :set -XOverloadedStrings\n\u003e import Data.Text.Builder.Linear\n\u003e fromText \"foo\" \u003c\u003e fromChar '_' \u003c\u003e fromDec (42 :: Int)\n\"foo_42\"\n```\n\n## Design\n\nString builders in Haskell serve the same purpose as `StringBuilder` in Java to prevent\nquadratic slow down in concatenation.\n\nClassic builders such as `Data.Text.Lazy.Builder` are lazy and fundamentally are\n[`dlist`](https://hackage.haskell.org/package/dlist) with bells and whistles:\ninstead of actually concatenating substrings we compose actions, which implement\nconcatenation, building a tree of thunks. The tree can be forced partially, left-to-right,\nproducing chunks of strict `Text`, combined into a lazy one. Neither input, nor output need to be materialized in full, which potentially allows for fusion. Such builders allow\nlinear time complexity, but constant factors are relatively high, because thunks are\nexpensive. To a certain degree this is mitigated by inlining, which massively reduces\nnumber of nodes.\n\nStrict builders such as [`text-builder`](https://hackage.haskell.org/package/text-builder)\noffer another design: they first inspect their input in full to determine output length,\nthen allocate a buffer of required size and fill it in one go. If everything inlines nicely,\nthe length may be known in compile time, which gives blazingly fast runtime. In more\ncomplex cases it still builds a tree of thunks and forces all inputs to be materialized.\n\nThis package offers two interfaces. One is a mutable `Buffer` with linear API,\nwhich operates very similar to `StringBuilder` in Java. It allocates a buffer\nwith extra space at the ends to append new strings. If there is not enough free space\nto insert new data, it allocates a twice larger buffer and copies itself there.\nThe dispatch happens in runtime, so we do not need to inspect and materialize all inputs\nbeforehand; and inlining is mostly irrelevant.\nExponential growth provides for amortized linear time.\nSuch structure can be implemented without linear types, but that would\ngreatly affect user experience by polluting everything with `ST` monad.\nUsers are encouraged to use `Buffer` API, and built-in benchmarks refer to it.\n\nThe second interface is more traditional `newtype Builder = Builder (Buffer ⊸ Buffer)`\nwith `Monoid` instance. This type provides easy migration from other builders,\nbut may suffer from insufficient inlining, allocating a tree of thunks. It is still\nsignificantly faster than `Data.Text.Lazy.Builder`, as witnessed by benchmarks\nfor `blaze-builder` below.\n\n## Case study\n\nLet's benchmark builders, which concatenate all `Char` from `minBound` to `maxBound`, producing a large `Text`:\n\n```haskell\n#!/usr/bin/env cabal\n{- cabal:\nbuild-depends: base, tasty-bench, text, text-builder \u003e= 1.0, text-builder-linear\nghc-options: -O2\n-}\n\nimport qualified Data.Text as T\nimport qualified Data.Text.Lazy as TL\nimport qualified Data.Text.Lazy.Builder as TLB\nimport qualified TextBuilder as TB\nimport qualified Data.Text.Builder.Linear as TBL\nimport System.Environment (getArgs)\nimport Test.Tasty.Bench\n\nmkBench :: Monoid a =\u003e String -\u003e (Char -\u003e a) -\u003e (a -\u003e Int) -\u003e Benchmark\nmkBench s f g = bench s $ nf (g . foldMap f . enumFromTo minBound) maxBound\n{-# INLINE mkBench #-}\n\nmain :: IO ()\nmain = defaultMain\n  [ mkBench \"text, lazy\" TLB.singleton (fromIntegral . TL.length . TLB.toLazyText)\n  , mkBench \"text, strict\" TLB.singleton (T.length . TL.toStrict . TLB.toLazyText)\n  , mkBench \"text-builder\" TB.char (T.length . TB.toText)\n  , mkBench \"text-builder-linear\" TBL.fromChar (T.length . TBL.runBuilder)\n  ]\n```\n\nRunning this program with `cabal run Main.hs -- +RTS -T` yields following results:\n\n```\ntext, lazy:\n  4.25 ms ± 107 μs,  11 MB allocated, 912 B  copied\ntext, strict:\n  7.18 ms ± 235 μs,  24 MB allocated,  10 MB copied\ntext-builder:\n  80.1 ms ± 3.0 ms, 218 MB allocated, 107 MB copied\ntext-builder-linear:\n  5.37 ms ± 146 μs,  44 MB allocated,  78 KB copied\n```\n\nThe first result seems the best both in time and memory and corresponds to the\nusual `Text` builder, where we do not materialize the entire result at all.\nIt builds chunks of lazy `Text` lazily and consumes them at once by\n`TL.length`. Thus there are 11 MB of allocations in nursery, none of which\nsurvive generation 0 garbage collector, so nothing is copied.\n\nThe second result is again the usual `Text` builder, but emulates a strict\nconsumer: we materialize a strict `Text` before computing length. Allocation\nare doubled, and half of them (corresponding to the strict `Text`) survive to\nthe heap. Time is also almost twice longer, but still quite good.\n\nThe third result is for `text-builder` and demonstrates how bad things could\ngo with strict builders, aiming to precompute the precise length of the\nbuffer: allocating a thunk per char is tremendously slow and expensive.\n\nThe last result corresponds to the current package. We generate a strict\n`Text` by growing and reallocating the buffer, thus allocations are quite\nhigh. Nevertheless, it is already faster than the usual `Text` builder with\nstrict consumer and does not strain the garbage collector.\n\nAs of GHC 9.10, things get very different if we remove `{-# INLINE mkBench #-}`:\n\n```\ntext, lazy:\n  36.9 ms ± 599 μs, 275 MB allocated,  30 KB copied\ntext, strict:\n  44.7 ms ± 1.3 ms, 287 MB allocated,  25 MB copied\ntext-builder:\n  77.6 ms ± 2.2 ms, 218 MB allocated, 107 MB copied\ntext-builder-linear:\n  5.35 ms ± 212 μs,  44 MB allocated,  79 KB copied\n```\n\nBuilders from `text` package degrade rapidly, 6-8x slower and 10-20x more\nallocations. That's because their constant factors rely crucially on\neverything getting inlined, which makes their performance fragile and\nunreliable in large-scale applications. On the bright side of things, our\nbuilder remains as fast as before and now is a clear champion.\n\n## Benchmarks for `Text`\n\nMeasured with GHC 9.6 on aarch64:\n\n|Group / size|`text`|`text-builder`|  |This package|  |\n|------------|-----:|-------------:|-:|-----------:|-:|\n| **Text** ||||||\n|1|47.4 ns|24.2 ns|0.51x|35.2 ns|0.74x|\n|10|509 ns|195 ns|0.38x|197 ns|0.39x|\n|100|4.94 μs|1.74 μs|0.35x|1.66 μs|0.34x|\n|1000|52.6 μs|17.0 μs|0.32x|15.0 μs|0.28x|\n|10000|646 μs|206 μs|0.32x|155 μs|0.24x|\n|100000|12.2 ms|3.34 ms|0.27x|2.60 ms|0.21x|\n|1000000|159 ms|55.3 ms|0.35x|16.1 ms|0.10x|\n| **Char** ||||||\n|1|46.9 ns|21.1 ns|0.45x|22.3 ns|0.48x|\n|10|229 ns|152 ns|0.66x|79.9 ns|0.35x|\n|100|2.00 μs|1.23 μs|0.61x|618 ns|0.31x|\n|1000|21.9 μs|10.3 μs|0.47x|6.28 μs|0.29x|\n|10000|285 μs|153 μs|0.54x|68.5 μs|0.24x|\n|100000|7.70 ms|4.08 ms|0.53x|992 μs|0.13x|\n|1000000|110 ms|106 ms|0.96x|9.19 ms|0.08x|\n| **Decimal** ||||||\n|1|97.7 ns|872 ns|8.92x|80.2 ns|0.82x|\n|10|864 ns|8.72 μs|10.09x|684 ns|0.79x|\n|100|9.07 μs|93.5 μs|10.32x|7.25 μs|0.80x|\n|1000|92.4 μs|1.06 ms|11.44x|67.5 μs|0.73x|\n|10000|1.13 ms|13.4 ms|11.88x|667 μs|0.59x|\n|100000|18.7 ms|141 ms|7.57x|7.57 ms|0.41x|\n|1000000|229 ms|1.487 s|6.48x|67.8 ms|0.30x|\n| **Hexadecimal** ||||||\n|1|403 ns|749 ns|1.86x|43.9 ns|0.11x|\n|10|3.94 μs|7.66 μs|1.94x|308 ns|0.08x|\n|100|42.8 μs|89.0 μs|2.08x|2.88 μs|0.07x|\n|1000|486 μs|986 μs|2.03x|27.7 μs|0.06x|\n|10000|7.10 ms|12.6 ms|1.77x|283 μs|0.04x|\n|100000|80.1 ms|133 ms|1.65x|3.53 ms|0.04x|\n|1000000|867 ms|1.340 s|1.55x|28.9 ms|0.03x|\n| **Double** ||||||\n|1|7.56 μs|18.3 μs|2.42x|414 ns|0.05x|\n|10|76.5 μs|188 μs|2.46x|4.23 μs|0.06x|\n|100|754 μs|2.35 ms|3.11x|44.4 μs|0.06x|\n|1000|7.94 ms|25.8 ms|3.25x|436 μs|0.05x|\n|10000|79.1 ms|285 ms|3.60x|4.90 ms|0.06x|\n|100000|796 ms|2.938 s|3.69x|45.1 ms|0.06x|\n|1000000|8.003 s|32.411 s|4.05x|436 ms|0.05x|\n\nIf you are not convinced by synthetic data,\nhere are benchmarks for\n[`blaze-markup` after migration to `Data.Text.Builder.Linear`](https://github.com/Bodigrim/blaze-markup):\n\n```\nbigTable\n  992  μs ±  80 μs, 49% less than baseline\nbasic\n  4.35 μs ± 376 ns, 47% less than baseline\nwideTree\n  1.26 ms ±  85 μs, 53% less than baseline\nwideTreeEscaping\n  217  μs ± 7.8 μs, 58% less than baseline\ndeepTree\n  242  μs ±  23 μs, 48% less than baseline\nmanyAttributes\n  811  μs ±  79 μs, 58% less than baseline\ncustomAttribute\n  1.68 ms ± 135 μs, 56% less than baseline\n```\n\n## Benchmarks for `ByteString`\n\nSomewhat surprisingly, `text-builder-linear` now offers rendering to strict `ByteString`\nas well. It is consistently faster than `bytestring` when a string gets over 32k\n(which is `defaultChunkSize` for `bytestring` builder). For mid-sized strings\n`bytestring` is slightly faster in certain disciplines, mostly by virtue of using\n`cbits` via FFI, while this package remains 100% native Haskell.\n\nBenchmarks below were measured with GHC 9.6 on aarch64 and include comparison\nto [`bytestring-strict-builder`](https://hackage.haskell.org/package/bytestring-strict-builder):\n\n|Group / size|`bytestring`|`…-strict-builder`|  |This package|  |\n|------------|-----------:|-----------------:|-:|-----------:|-:|\n| **Text** ||||||\n|1|106 ns|33.5 ns|0.32x|35.2 ns|0.33x|\n|10|322 ns|217 ns|0.68x|197 ns|0.61x|\n|100|2.49 μs|1.89 μs|0.76x|1.66 μs|0.67x|\n|1000|21.8 μs|18.5 μs|0.85x|15.0 μs|0.69x|\n|10000|231 μs|212 μs|0.92x|155 μs|0.67x|\n|100000|3.97 ms|3.54 ms|0.89x|2.60 ms|0.66x|\n|1000000|81.2 ms|51.5 ms|0.63x|16.1 ms|0.20x|\n| **Char** ||||||\n|1|99.0 ns|19.4 ns|0.20x|22.3 ns|0.23x|\n|10|270 ns|82.9 ns|0.31x|79.9 ns|0.30x|\n|100|1.77 μs|723 ns|0.41x|618 ns|0.35x|\n|1000|20.4 μs|8.37 μs|0.41x|6.28 μs|0.31x|\n|10000|322 μs|129 μs|0.40x|68.5 μs|0.21x|\n|100000|10.4 ms|2.50 ms|0.24x|992 μs|0.10x|\n|1000000|143 ms|67.4 ms|0.47x|9.19 ms|0.06x|\n| **Decimal** ||||||\n|1|152 ns|174 ns|1.14x|80.2 ns|0.53x|\n|10|685 ns|1.55 μs|2.26x|684 ns|1.00x|\n|100|5.88 μs|17.2 μs|2.93x|7.25 μs|1.23x|\n|1000|60.3 μs|196 μs|3.25x|67.5 μs|1.12x|\n|10000|648 μs|4.25 ms|6.57x|667 μs|1.03x|\n|100000|11.2 ms|62.8 ms|5.62x|7.57 ms|0.68x|\n|1000000|150 ms|655 ms|4.37x|67.8 ms|0.45x|\n| **Hexadecimal** ||||||\n|1|94.7 ns|||43.9 ns|0.46x|\n|10|255 ns|||308 ns|1.21x|\n|100|1.72 μs|||2.88 μs|1.67x|\n|1000|18.9 μs|||27.7 μs|1.46x|\n|10000|250 μs|||283 μs|1.13x|\n|100000|6.94 ms|||3.53 ms|0.51x|\n|1000000|93.2 ms|||28.9 ms|0.31x|\n| **Double** ||||||\n|1|457 ns|||414 ns|0.91x|\n|10|3.94 μs|||4.23 μs|1.07x|\n|100|40.3 μs|||44.4 μs|1.10x|\n|1000|398 μs|||436 μs|1.10x|\n|10000|5.65 ms|||4.90 ms|0.87x|\n|100000|63.3 ms|||45.1 ms|0.71x|\n|1000000|673 ms|||436 ms|0.65x|\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbodigrim%2Flinear-builder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbodigrim%2Flinear-builder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbodigrim%2Flinear-builder/lists"}