{"id":16661089,"url":"https://github.com/kukimik/dhall-text-utils","last_synced_at":"2026-03-16T12:06:16.823Z","repository":{"id":243665723,"uuid":"732349875","full_name":"kukimik/dhall-text-utils","owner":"kukimik","description":"Utilities for validation and modification of Text values in the Dhall configuration language","archived":false,"fork":false,"pushed_at":"2024-06-11T06:58:51.000Z","size":131,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-19T13:43:42.101Z","etag":null,"topics":["dhall-lang","text","validation"],"latest_commit_sha":null,"homepage":"","language":"Dhall","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kukimik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-16T11:39:23.000Z","updated_at":"2025-01-11T20:39:51.000Z","dependencies_parsed_at":"2024-06-12T09:16:36.729Z","dependency_job_id":null,"html_url":"https://github.com/kukimik/dhall-text-utils","commit_stats":null,"previous_names":["kukimik/dhall-text-utils"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kukimik%2Fdhall-text-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kukimik%2Fdhall-text-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kukimik%2Fdhall-text-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kukimik%2Fdhall-text-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kukimik","download_url":"https://codeload.github.com/kukimik/dhall-text-utils/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243293766,"owners_count":20268139,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dhall-lang","text","validation"],"created_at":"2024-10-12T10:33:31.171Z","updated_at":"2025-12-25T13:20:15.858Z","avatar_url":"https://github.com/kukimik.png","language":"Dhall","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dhall-text-utils\n\nUtilities for validation and modification of `Text` values in the [Dhall](https://dhall-lang.org/) configuration language. Not recommended for use in production.\n\n## What is this all about? \n\nThe `Text` type in Dhall is opaque by design. It is impossible to inspect the value of a `Text` expression during evaluation. (One may, however, test `Text` values for equality during the type-checking phase, e.g. `assert : \"abc\" === \"ab\" ++ \"c\"`.) The `Text`-related language bulit-ins are also scarce: `Text/show`, `Text/replace` and the concatenation operator `++`.\n\nThis library contains a number of functions, implemented using the above toolset and a dose of creativity, that allow one to manipulate `Text` values based on their content:\n\n```dhall\nlet L = ./src/Logic/package.dhall\nlet P = ./src/Predicates/package.dhall\nlet Prelude = ./src/Prelude.dhall\n\nlet animals = [\"aardvark\", \"hippo\", \"dog\", \"chimpanzee\", \"cat\", \"chickadee\", \"flapper\", \"manatee\",\"aardwolf\"]\n\nlet f : Text -\u003e Text =\n    \\(t : Text)\n        -\u003e \"${Text/show t} \"\n            ++\n           L.ifThenElse\n            (L.or [P.hasPrefix \"aa\" t, P.hasSuffix \"ee\" t, P.contains \"pp\" t])\n            (\"starts with \\\"aa\\\", or ends with \\\"ee\\\", or contains \\\"pp\\\"!\")\n            (\"is not very interesting\")\n\nin  Prelude.Text.concatSep \"\\n\"\n        (Prelude.List.map Text Text f animals)\n```\n\nvalidate `Text` values at the type-checking phase:\n\n```dhall\nlet L = ./src/Logic/package.dhall\nlet P = ./src/Predicates/package.dhall\n\nlet badWords = [\"adults\", \"only\", \"really\", \"bad\", \"swear\", \"words\"]\n\nlet rhyme =\n    ''\n    As I went up the apple tree\n    All the apples fell on me.\n    Apple pudding, apple pie\n    Did you ever tell a lie?\n    ''\n\nlet test = assert : L.isFalse (P.containsOneOf badWords rhyme)\n\nin rhyme\n```\n\nor use dependent types to control allowed function input:\n\n```dhall\nlet L = ./src/Logic/package.dhall\nlet P = ./src/Predicates/package.dhall\nlet T = ./src/Transformations/package.dhall\n\nlet hexDigits = [\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\",\"8\",\"9\",\"A\",\"B\",\"C\",\"D\",\"E\",\"F\"]\n\nlet  hex2binary: ∀(hex : Text) -\u003e  (L.isTrue (P.consistsOf hexDigits hex)) -\u003e Text =\n    \\(hex : Text) -\u003e \\(_ : L.isTrue (P.consistsOf hexDigits hex)) -\u003e\n        T.applyAll\n            [Text/replace \"F\" \"1111\"\n            ,Text/replace \"E\" \"1110\"\n            ,Text/replace \"D\" \"1101\"\n            ,Text/replace \"C\" \"1100\"\n            ,Text/replace \"B\" \"1011\"\n            ,Text/replace \"A\" \"1010\"\n            ,Text/replace \"9\" \"1001\"\n            ,Text/replace \"8\" \"1000\"\n            ,Text/replace \"7\" \"0111\"\n            ,Text/replace \"6\" \"0110\"\n            ,Text/replace \"5\" \"0101\"\n            ,Text/replace \"4\" \"0100\"\n            ,Text/replace \"3\" \"0011\"\n            ,Text/replace \"2\" \"0010\"\n            ,Text/replace \"1\" \"0001\"\n            ,Text/replace \"0\" \"0000\"\n            ]\n            hex\n\nin hex2binary \"BADF00D\" L.QED\n```\n\nAnd some parser-like capabilities are in the making!\n\nMore examples can be found in the [`examples/`](./examples/) directory.\n\n## Some comments\n\nThe library was created out of curiosity. I wanted to see how far I can get with validation and manipulation of `Text`\nvalues using the limited `Text` toolbox of Dhall. It was great fun!\n\nThe following discussions gave me some inspiration:\n\n* https://github.com/dhall-lang/dhall-lang/issues/1035\n* https://github.com/dhall-lang/dhall-lang/pull/669#issuecomment-515748801\n* https://discourse.dhall-lang.org/t/text-manipulation-functions/176/55\n\nNote that some of the possibilities that the library gives are strongly against the philosophy of Dhall (and good programming\npractices), e.g. it allows to string-encode some of the business logic instead of keeping it at the type level.\n\nWhile writing the library I mostly thought about extending it with new capabilities, keeping it correct, and making its use\nsimple. Performance was never a goal.\n\nFor this reason I do not advise to use `dhall-text-utils` in production. The performance is probably very poor (it was never\nbenchmarked). While some improvements are definitely possible, the whole idea behind the library seems doomed to\nhave bad performance. The library was also not tested thoroughly.\n\nHowever, the library demonstrates that Dhall already has some text validation capabilities. This may be an argument to \nhave them included, in some form, in a future version of the language standard as built-in features.\n\n## How it works?\n\nThe library makes use of several properties of the `Text` related operations. Some of them are inherent to these operations\n(e.g. the empty string being the neutral element of concatenation), some are more incidental (e.g. the escaping rules used\nby `Text/show`).\n\nWhile the `Text` type is opaque in Dhall, i.e. we can not inspect `Text` values and get, say, a `Bool` or a `Natural` result\ndepending on their content. However, `Text` values may be modified. And the result of the `Text/replace : Text -\u003e Text -\u003e Text -\u003e Text` function\ndepends on the content of its arguments in an interesting way. In particular `Text/replace t s t` equals `s` for any non-empty string `t`, but it\nequals `\"\"` if `t === \"\"`. Using this fact we can, given a `Text` value `t`, obtain some fixed string, let it be `\"x\"`, if `t` is \nnon-empty, and obtain `\"\"` otherwise:\n\n```dhall\n-- returns \"\" if t is empty and \"x\" if t is not empty\nlet nonempty2x = \\(t : Text) -\u003e Text/replace t \"x\" t\n```\n\nIt also happens that `\"\"` is the neutral element of the `Text` concatenation monoid (i.e. appending or prepending `\"\"` does not change\na string), which allows us tu \"detect\" the empty string in a similar fashion:\n\n```dhall\n-- returns \"\" if t is not empty and \"x\" if t is empty\nlet empty2x = \\(t : Text) -\u003e Text/replace \"xx\" \"\" (\"x\" ++ Text/replace t \"x\" t)\n```\n\nNow it is easy to write a function that accepts three arguments and returns the second argument if the first argument is the empty string and\nreturns the third argument otherwise:\n\n```dhall\nlet ifEmptyThenElse = \n    \\(t : Text) -\u003e \\(whenEmpty : Text) -\u003e \\(whenNotEmpty : Text) -\u003e\n        (Text/replace \"x\" whenEmpty (empty2x t))\n        ++\n        (Text/replace \"x\" whenNotEmpty (nonempty2x t))\n```\n\nIf we are able to implement a function (let's call it a \"predicate\") `hasPropertyP : Text -\u003e Text` that converts a `Text` value `t` either to `\"x\"` or to `\"\"`, depending on `t` having or not having some property `P`, then we may use `ifEmptyThenElse` to construct a function `ifHasPropertyPThenElse : Text -\u003e Text -\u003e Text` which returns either the second argument if the first argument has property `P`, and the third argument otherwise. Moreover, we can use `hasPropertyP` to validate function arguments at the type-checking level:\n\n```dhall\nlet f : forall ( t : Text ) -\u003e (\"x\" === hasPropertyP t) -\u003e SomeType =\n    \\(t : Text) -\u003e \\(assert : (\"x\" === hasPropertyP t)) -\u003e [...]\n```\n\nor to write `assert`-based tests:\n\n```dhall\nlet test = assert : \"x\" === hasPropertyP someVariable\n```\n\nIt turns out such \"predicates\" exist for many properties. For example, to check whether `t` is equal to some given string, we can use:\n\n```dhall\nlet equals = \\(s : Text) -\u003e \\(t : Text) -\u003e\n    empty2x (Text/replace s \"\" t ++ Text/replace t \"\" s)\n```\n\nSee [`./src/Predicates`](./src/Predicates) for the \"predicates\" that the library provides.\n\nNow, if we assign names to `\"x\"` and `\"\"`:\n\n```dhall\nlet true : Text = \"x\"\nlet false : Text = \"\"\n```\n\nwe can implement Boolean operators:\n\n```dhall\nlet not : Text -\u003e Text = empty2x\nlet or : Text -\u003e Text -\u003e Text = \\(t1 : Text) -\u003e \\(t2 : Text) -\u003e nonempty2x (t1 ++ t2) \nlet and : Text -\u003e Text -\u003e Text = \\(t1 : Text) -\u003e \\(t2 : Text) -\u003e not (or (not t1) (not t2))\n```\n\nthat work as expected when their input is either `\"\"` or `\"x\"`.\n\nThe `dhall-text-utils` library wraps `true` and `false` in a record type (named `TextBool`, see [`./src/Logic`](./src/Logic)) and adds some sugar here and there, but this is the basic idea.\n\nOne other notable trick that is used in the [`stripPrefix`](./src/Transofrmations/stripPrefix.dhall) and [`stripSuffix`](./src/Transofrmations/stripSuffix.dhall) functions (and in the [histogram example](./examples/histogram.dhall)) is based on the fact that the output of `Text/show` will never contain certain substrings\n(e.g. three consecutive quotation marks) or characters (e.g. a tab or a newline, which are converted to `\\t` and `\\n`). The details are left to the interested\nreader.\n\n## How is the library structured?\n\nThe library consists of several interrelated sub-packages:\n\n* `Logic`\n* `Predicates`\n* `CharacterClasses`\n* `Transformations`\n\nHere we describe the contents of these packages. Studying the examples provided in this README and in the [`./examples`](./examples) directory should\nbe enough to start working with the library. \n\n### `Logic`\n\nContains the definition of the `Logic.TextBool` type, which is used to represent \"true\" and \"false\" encoded as `Text` wrapped in a record. Values of the type `Logic.TextBool` should never be constructed by hand. The details of the implementation of this type can change without notice and this is not\nconsidered a breaking change. The main source of `Logic.TextBool` values are the functions living in the `Predicates` package. However, the functions `Logic.true` and `Logic.false` can be used to create `Logic.TextBool` \"literals\".\n\nThe basic Boolean operations are provided by `Logic.not`, `Logic.or`, `Logic.and` and `Logic.xor`. An `if`-like conditional is given in `Logic.ifThenElse`.\n\nThe package also contains some sugar used for assertions and dependent types: `Logic.isTrue`, `Logic.isFalse` can be used instead of `=== Logic.true`\nor `=== Logic.false`. For example, instead of writing `Predicates.isEmpty x === Logic.true` we can write `Logic.isTrue (Predicates.isEmpty x)`. This\nis mostly a matter of style. There exists another helper `Logic.QED`, that provides a \"proof\" for dependently typed functions:\n\n```dhall\nlet f : forall (t : Text) -\u003e Logic.isFalse (Predicates.isEmpty t) -\u003e Text =\n      \\(t : Text) -\u003e \\(_ : Logic.isFalse (Predicates.isEmpty t)) -\u003e [...]\n\nin f \"abc\" Logic.QED\n```\n\nThe functions `Logic.any` and `Logic.all` can be used to map predicates over a list and aggregate the result using either alternative or conjunction.\n\nThe package also contains a number of less notable functions.\n\n### `CharacterClasses`\n\nWe defina a *character class* to be a list of `Text` values, each consisting of a single codepoint. For example `[\"a\",\"b\",\"3\"]` and `[] : List Text` are character classes, but `[\"a\",\"bc\"]` is not.\n\nThe package contains definitions of several common character classes (e.g. the list of all ASCII characters, the list of ASCII digits, the list of Unicode control characters).\n\nNote that if a function in the library accepts a `List Text` value that is expected to be a charcater class, then it should be a character class. **This is not\ncontrolled at the type level and may lead to errors!** One should pay special attention to [Unicode characters consisting of several codepoints](https://tonsky.me/blog/unicode/#why-is-a----).\n\n### `Predicates`\n\nBy a *predicate*, in the context of this library, we understand a `TextBool`-valued function. The package contains a number of predicates related to various properties of `Text` values, e.g. `Predicates.isEmpty : Text -\u003e TextBool`, `Predicates.hasPrefix : Text -\u003e Text -\u003e TextBool`, `Predicates.containsOneOf : List Text -\u003e Text -\u003e Text`.\n\nSome of these have versions with arguments flipped, e.g. `Predicates.contains` and `Predicates.isContainedIn`, or `Predicates.hasSuffix` and `Predicates.isSuffixOf`. Some may be easily constructed using the others and the functions from the `Logic` package (e.g. `Predicates.containsOneOf` is implemented using `Logic.any` and `Predicates.isContainedIn`). I do not strive for minimality. There is at present no rule which of these derived predicates are included in the library, and which are not. One exception is that I do not want to include negations of existsing predicates. For example:\n\n```dhall\nlet isNotEmpty : Text -\u003e TextBool = \\(t : Text) -\u003e Logic.not (Predicates.isEmpty t)\n```\n\nis not going to be included in the `Predicates` package.\n\nSome of the predicates (e.g. `Predicates.consistsOf` or `Predicates.hasLengthUsing`) accept an argument that is required to be a character class (see the section above). **This is not checked at the type level!**\n\nThere are a few dependently typed predicates: `Predicates.hasLengthAtLeastUsing`, `Predicates.hasLengthAtMostUsing`, `Predicates.hasLengthUsing`. These functions use the fact that a given string consists entirely of codepoints belonging to a given character class. To use these predicates you have to guarantee, at the type-checking level, that this property holds. So, for example:\n\n```dhall\nPredicates.hasLengthUsing 5 CharacterClasses.ASCII \"abcde\" Logic.QED === Logic.true\n```\n\nbut\n\n```dhall\nPredicates.hasLengthUsing 5 CharacterClasses.ASCII \"ąbćdę\" Logic.QED\n```\n\nwill fail.\n\nIf you need to use a non-dependently typed version of these functions, then you may either try to use `Predicates.hasSubstringOfLengthAtLeastConsistingOf`. Or, when you are sure that your `Text` values consist only of character belonging to a certain class, create your own versions of these predicates, by copying the source and getting rid of the last argument.\n\n### `Transformations`\n\nThe package contains a miscellany of functions that may be useful when manipulating `Text` values, like `Transformations.stripPrefix`, `Transformations.stripSuffix` or `Transformations.applyAll`, which consecutively applies a list of `Text -\u003e Text` functions to a given `Text` value.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkukimik%2Fdhall-text-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkukimik%2Fdhall-text-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkukimik%2Fdhall-text-utils/lists"}