{"id":13735562,"url":"https://github.com/zero-functional/zero-functional","last_synced_at":"2025-05-08T11:33:44.022Z","repository":{"id":42710457,"uuid":"109588686","full_name":"zero-functional/zero-functional","owner":"zero-functional","description":"A library providing zero-cost chaining for functional abstractions in Nim.","archived":false,"fork":false,"pushed_at":"2022-09-19T06:05:08.000Z","size":2136,"stargazers_count":315,"open_issues_count":3,"forks_count":17,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-05-18T03:33:49.339Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zero-functional.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-05T14:55:16.000Z","updated_at":"2024-05-15T12:34:26.000Z","dependencies_parsed_at":"2023-01-18T14:00:35.411Z","dependency_job_id":null,"html_url":"https://github.com/zero-functional/zero-functional","commit_stats":null,"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zero-functional%2Fzero-functional","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zero-functional%2Fzero-functional/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zero-functional%2Fzero-functional/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zero-functional%2Fzero-functional/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zero-functional","download_url":"https://codeload.github.com/zero-functional/zero-functional/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":213758654,"owners_count":15634354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T03:01:08.207Z","updated_at":"2024-08-03T03:05:03.375Z","avatar_url":"https://github.com/zero-functional.png","language":"C++","funding_links":[],"categories":["Language Features"],"sub_categories":["Functional Programming"],"readme":"# zero-functional [![Build Status](https://github.com/zero-functional/zero-functional/workflows/CI/badge.svg?branch=master)](https://github.com/zero-functional/zero-functional/actions?query=branch%3Amaster)\n\nA library providing (almost) zero-cost chaining for functional abstractions in Nim.\n\n\nTable of Contents\n=================\n\n\u003c!-- toc --\u003e\n\u003c!-- add toc with markdown-toc --\u003e\n- [Initial Example](#initial-example)\n- [Installation](#installation)\n- [Rationale](#rationale)\n- [Variable names](#variable-names)\n- [Seq and arrays](#seq-and-arrays)\n- [Other types](#other-types)\n- [Supported methods](#supported-methods)\n  * [map](#map)\n  * [filter](#filter)\n  * [zip](#zip)\n  * [split](#split)\n  * [exists](#exists)\n  * [all](#all)\n  * [index](#index)\n  * [indexedMap](#indexedmap)\n  * [enumerate](#enumerate)\n  * [fold](#fold)\n  * [reduce](#reduce)\n    + [max](#max)\n    + [min](#min)\n    + [product](#product)\n    + [sum](#sum)\n  * [indexedReduce](#indexedreduce)\n  * [foreach](#foreach)\n    + [changing in-place](#changing-in-place)\n  * [sub](#sub)\n    + [drop](#drop)\n    + [dropWhile](#dropwhile)\n    + [take](#take)\n    + [takeWhile](#takewhile)\n  * [flatten](#flatten)\n    + [indexedFlatten](#indexedflatten)\n  * [combinations](#combinations)\n    + [indexedCombinations](#indexedcombinations)\n  * [concat](#concat)\n  * [group](#group)\n  * [partition](#partition)\n  * [to](#to)\n    + [createIter](#createiter)\n- [Extending zero-functional](#extending-zero-functional)\n  * [Extending with plain nim](#extending-with-plain-nim)\n  * [Writing extensions with Zero-DSL](#writing-extensions-with-zero-dsl)\n    + [Special variables](#special-variables)\n    + [Setting the result](#setting-the-result)\n  * [Defering to Zero-DSL inside plain nim](#defering-to-zero-dsl-inside-plain-nim)\n    + [Creating compound commands with other commands](#creating-compound-commands-with-other-commands)\n- [Overview Table](#overview-table)\n- [Debugging using `--\u003e\u003e`](#debugging-using---)\n- [Compile flags](#compile-flags)\n- [LICENSE](#license)\n- [Contributors](#contributors)\n\n\u003c!-- tocstop --\u003e\n\n## Initial Example\nThe example:\n\n```nim\nvar n = zip(a, b) --\u003e\n            map(f(it[0], it[1])).\n            filter(it mod 4 \u003e 1).\n            map(it * 2).\n            all(it \u003e 4)\n```\n\nis expanded on compile time - additional compile-time checks omitted - to the equivalent of:\n\n```nim\n(proc (): auto =\n  var minHigh134598 = min([high(a), high(b)])\n  var empty = true\n  for z in low(a) .. minHigh134598:\n    var it0 = (a[z], b[z])\n    var it1 = f(it0[0], it0[1])\n    if it1 mod 4 \u003e 1:\n      var it2 = it1\n      var it3 = it2 * 2\n      result = true\n      if not(it3 \u003e 4):\n        return false)()\n```\n\nCompared to:\n```nim\nimport sequtils\n\nvar n = zip(a, b).\n            mapIt(f(it[0], it[1])).\n            filterIt(it mod 4 \u003e 1).\n            mapIt(it * 2).\n            allIt(it \u003e 4)\n```\n\nwhich is roughly equivalent to:\n\n```nim\nvar m = min(a.len, b.len)\nvar result0: seq[(int, int)]\nnewSeq(result0, m)\nfor i in 0 .. \u003cm:\n  result0[i] = (m[i], m[i])\nvar result1: seq[int]\nlet t0 = result0\nvar i0 = 0\nresult1 = newSeq[int](result0.len)\nfor it in t0:\n  result1[i0] = f(it[0], it[1])\n  i0 += 1\nvar result2 = newSeq[int]()\nfor it in items(result1):\n  if it mod 4 \u003e 1:\n    result.add(it)\nvar result3: seq[int]\nlet t1 = result1\nvar i1 = 0\nresult3 = newSeq[int](result2.len)\nfor it in t1:\n  result3[i1] = it * 2\nresult = true\nfor it in items(result3):\n  if not (it \u003e 4):\n    return false\n```\n\n\n## Installation\n\n```\nnimble install zero_functional\n```\n\nNote: the correct name is `zero_functional` (with an underscore).\n\n\n## Rationale\n\nFunctional style handling of sequences is awesome, and Nim is supposed to be fast and smart.\nAllocating new sequences on each method in a chain can be extremely wasteful and there are not a lot of technical reasons to punish functional style like that.\n\nThis library can expand functional chains to simple loops fusing the method bodies one after another.\nIt is still very experimental, but it shows that a purely metaprogramming approach can be used to optimize functional Nim code.\n\n\n## Variable names\n\nThe supported variable names (can be changed at the beginning of the [zero_functional.nim](zero_functional.nim) file) are:\n\n* `it` is used for the iterator variable\n* `idx` is used as integer index of current iteration\n* `a` is used as the accumulator in `fold`\n\n\n## Seq and arrays\n\nAll supported methods work on finite indexable types and arrays.\n\nIf a handler returns a collection, it will be of the same shape as the input for seq-s, arrays and DoublyLinkedList-s.\nOther collections are mapped to seq if it cannot be automatically converted. (e.g. array.map returns an array).\n\nYou can always get a seq if you use `\u003chandler\u003eSeq`, e.g. `mapSeq` - or `to(seq)`.\nSome of the supported methods default to seq-output, e.g. `map` when changing the result type, `flatten` and `indexedMap`.\n\nWe can describe the supported types as\n\n```nim\ntype\n  FiniteIndexable[T] = concept a\n    a.low is int\n    a.high is int\n    a[int] is T\n```\n\n\n## Other types\n\nEnums are supported and mapped to `seq[enumtype]`.\n\nGeneric objects are supported if they are of any type:\n + FiniteIndexable - contains `high`/`low` and `[]`-access (see above)\n + FiniteIndexableLen - contains `len` and `[]`\n\nCollection types that will be generated as a result type need to implement either one of\n + Appendable (contains the `append` function as in DoublyLinkedList)\n + Addable (contains the `add` function as in `seq`)\n + `[]=` operator\n\nSome of the supported methods will only work when the `[]=` operator is defined - except when using DoublyLinkedList or SinglyLinkedList types.\nThis is needed for `zip`, `combinations` and `foreach` when changing elements.\n\nFor the creation of a generic type as result, the type needs to implement\n```nim\nproc zfInit(a: MyType): MyType =\n  MyType(...)\n  # the `a` is not actually used but is needed for overloading.\n```\n\n\n## Supported methods\n\nThese are not exactly the functions from sequtils, they have the same naming and almost the same behavior.\n\nThe macros work with `--\u003e`, `zfun` or `connect`. Multiple `--\u003e` may be used or `.`.\n\n```nim\nsequence --\u003e map(..) --\u003e all(..)\n```\n\nor\n\n```nim\nzip(a, b, c) --\u003e map(..).\n                 all(..)\n```\n\nYou can also use the call with sections - with the above calls omitting the dots or arrows or with several statements applied to the same function in one section.\n```nim\nlet res = sequence.zfun:\n  map:\n    f1(it)\n    f2(it)\n  all(...)\n```\n\nor as simple arguments to a function:\n\n```nim\nconnect(collection, map(..), all(..))\n```\n\nThe methods work with the auto `it` variable.\n\n\n### map\n\n```nim\ncollection --\u003e map(op)\n```\n\nMap each item in the collection to a new value.\nExample:\n\n```nim\nlet x = [1,2,3] --\u003e map(it * 2)\ncheck(x == [2,4,6])\n```\nMap also supports converting the type of iterator item and thus of the collection.\n\nMap is (currently) the only command supporting value definitions inside the map call that may be used instead or additionally to the `it` value in subsequent calls. Assignment to simple values or tuple assignments are possible as well.\n```nim\nlet idx = rows --\u003e map(row = it) --\u003e index(row \u003e someValue)\nlet posCoords = coords --\u003e map((x,y) = it) --\u003e filter(x \u003e 0 and y \u003e 0)\n```\n\nA shortcut syntax is also supported - actually replacing `it` with the given variable names:\n```nim\n# a iterates on x\ncheck(x --\u003e (a) --\u003e exists(y --\u003e exists(a == it)))\n# this is equivalent to\ncheck(x --\u003e map(a = it) --\u003e exists(y --\u003e exists(a == it)))\n```\n\n`zfun` also supports this shortcut either as single line `(a)` or as a parameter to `zfun`:\n```nim\nlet b = x.zfun(a):\n  exists:\n    z.zfun(b):\n      exists(a == b)\ncheck(not b)\n```\n\n### filter\n\n```nim\nsequence --\u003e filter(cond)\n```\n\nFilter the collection items with the given condition.\nExample:\n\n```nim\nlet x = @[-1,2,-3] --\u003e filter(it \u003e 0)\ncheck(x == @[2])\n```\n\n### zip\n\n`zip` can work with n sequences. `zip` is (roughly) internally translated to:\n\n```nim\nzip(a,b,c) \u003c=\u003e\na --\u003e zip(b,c) \u003c~\u003e\nlet minHigh = min([a.high(), b.high(), c.high()])\na --\u003e filter(idx \u003c= minHigh) --\u003e map(a[idx], b[idx], c[idx])\n```\n\nOn the right side of `--\u003e` (or as 2nd and later command) the left side of `--\u003e` is added to the zip result.\nFor `zip` in order to work properly all arguments have to support access with `[]` and the `high` procedure.\nIf those procedures are not available the macro tries to call the procedure `mkIndexable` on that parameter.\nUsing this helper the parameter can be wrapped with a new type that supports `[]` and `high`.\n\n### split\n`split` is kind of the opposite of `zip`. It works with a collection of n-tuples and splits it up into an n-tuple of sequences.\n\n```nim\ncheck([(1, \"one\"), (2, \"two\")] --\u003e split() == (@[1,2], @[\"one\",\"two\"])\n```\n\n### exists\n\nCheck if the given condition is true for at least one element of the collection.\n\n`exists` can be used only at the end of the command chain.\n\n```nim\nsequence --\u003e otherOperations(..) --\u003e exists(cond): bool\n```\n\n\n### all\n\nCheck if the given condition is true for all the elements of the collection.\n\n`all` can be used only at the end of the command chain.\n\n```nim\nsequence --\u003e otherOperations(..) --\u003e all(cond): bool\n```\n\n\n### index\n\nGet the first index of the item in the collection, where the given condition is true.\n\n`index` can be used only at the end of the command chain.\n\n```nim\nsequence --\u003e otherOperations(..) --\u003e index(cond): int\n```\n\n\n### indexedMap\n\nAdds or prepends the index of each element in the collection to the element itself and generates a named tuple `(idx: index, elem: it)` for each element in the collection. \n\n```nim\nvar n = zip(a, b, c) --\u003e\n            indexedMap(f(it[0], it[1])).\n            filter(it.idx \u003c 10 and it.elem mod 4 \u003e 1).\n            map(it.elem * 2).\n            all(it \u003e 4)\n```\n\n### enumerate\n\nPrepends the index to the value of the current iteration (unlike the `indexedMap`, which indexes the initial collection/iteration) and generates a named tuple `(idx: index, elem: it)` for each element in the iterable. Does not take any parameters.\n\nIs similar to [`enumerate`](https://docs.python.org/3/library/functions.html#enumerate) in Python or [`enumerate`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.enumerate) in Rust.\n\n```nim\n0..10 --\u003e filter(it mod 2 == 0).enumerate().map(it.idx) == @[0, 1, 2, 3, 4, 5]\n# enumerate vs indexedMap\n(1..7 --\u003e filter(it mod 2 == 0) --\u003e enumerate())    == @[(0, 2), (1, 4), (2, 6)])\n(1..7 --\u003e filter(it mod 2 == 0) --\u003e indexedMap(it)) == @[(1, 2), (3, 4), (5, 6)])\n```\n\n\n### fold\n\nCurrently a left fold (as easier to combine with the implementation).\n\nThe sequtils `a` is `a`, `b` is `it`.\n\n```nim\nvar n = zip(a, b) --\u003e map(it[0] + it[1]) --\u003e fold(0, a + it)\n```\n\n\n### reduce\n\nSame as fold, but with the iterator converted to a tuple where `it[0]` or `it.accu` is the accumulated result and `it[1]` or `it.elem` the actual iterator on the collection.\n\nThe first item of the collection is used as initial value - the other items are then accumulated to it.\nThis is also useful when a type does not define the neutral element for the given operation.\nE.g. for integers and `+` the neutral element is 0 but for user defined types the neutral element might not exist.\n\n```nim\nvar n = a --\u003e reduce(it.accu + it.elem)\n```\n\nThere are a few commands that are simply mapped to reduce\n\n#### max\nReturn the maximum value in the collection (`\u003e` is needed)\n\n#### min\nReturn the minimum value in the collection (`\u003c` is needed)\n\n#### product\nReturn the product of the (filtered) elements (`*`)\n\n#### sum\nReturn the sum of the (filtered) elements (`+`).\n\n\n### indexedReduce\n\nBy adding the `indexed` prefix to `reduce` or to the reduce commands above, the index of the last value that was used for the `result` and the actual result of the operation are returned.\n\nFor `sum` and `product` this is not actually helpful but it can be used to find the indices of the `min` and `max` elements.\n\n```nim\ncheck(@[11,2,0,-2,1,3,-1] --\u003e indexedMin() == (3,-2))\ncheck(@[11,2,0,-2,1,3,-1] --\u003e indexedMax() == (0,11))\n```\n\nNote that a named tuple is created and the index is also accessible via `.idx` and the actual element with `.elem`.\n\n### foreach\n\nCan only be used with functions that have side effects.\nWhen used as the last command in the chain the result is void.\nAs in-between element the code is simply executed on each element.\n\n\n#### changing in-place\nThe iterator content may be changed in `foreach` resulting in changing the original collection.\nHowever there are a few restrictions (see [test.nim](test.nim#L455)):\n+ the `[]=` operator has to be available for the underlying collection type (exception: the std LinkedList types)\n+ functions that alter the collection elements may not be used in the chain before (e.g. `map` is not allowed, but `filter` is).\n\n```nim\n@[1,2,3] --\u003e\n    foreach(echo($it))\n\nvar a = @[1,2,3]\na --\u003e foreach(it = it * 2)\ncheck (a == @[2,4,6]\n```\n\n\n### sub\n\nWorks on a part of the input collection - `sub(fromIndex, toIndex)` or `drop(fromIndex)` - similarly to ranges, starting with `fromIndex` and ending (inclusive) with `endIndex` or runs til the end, when `endIndex` is not given.\n```nim\ncheck((1..10) --\u003e sub(2,5) --\u003e to(list) == @[2,3,4,5])\n```\n\nThe `endIndex` may be a `BackwardsIndex` like `^1`, but then the collection has to have a `len`.\n\n`sub` is similar to the `filter` function working on the `idx` variable, however `sub` uses an internal index that is not affected by the outcome of preceding filtering functions.\n\n```nim\n# in filter `idx` counts the iterated items\ncheck(@[-1,2,-3,4,-5,6,-7,8] --\u003e filter(it \u003e 0) --\u003e filter(idx \u003e= 3) == @[4,6,8])\n# sub increments its own index when `it \u003e 0`\ncheck(@[-1,2,-3,4,-5,6,-7,8] --\u003e filter(it \u003e 0) --\u003e sub(3) == @[8])\n```\n\nSimilar commands like `sub` that result in parts of the lists being iterated on or generated are: `drop`, `dropWhile`, `take` and `takeWhile`.\n\n\n#### drop\n`drop(n)` drops n items before working on the collection. This is equivalent to `sub(n)`.\n\n#### dropWhile\n`dropWhile(cond)` drops the items as long as the condition in `cond` is met - it starts working on the collection when the condition is not fulfilled any more.\nAs opposed to `filter` the condition in `drop` is ignored, once it was not true any more.\n```nim\ncheck(@[-1,2,-3,4,-5] --\u003e dropWhile(it \u003c 0) --\u003e sum() == -2)\ncheck(@[-1,2,-3,4,-5] --\u003e filter(it \u003e= 0)   --\u003e sum() == 6)\n```\n\n#### take\n`take(n)` works on n items of the collection and then breaking. This is useful for very large (infinite) collections or iterators - the same for `takeWhile`.\n\n#### takeWhile\n`takeWhile(cond)` works on the collection as long as the condition in `cond` is met. Otherwise it breaks the processing.\n\n\n### flatten\n\nWorking on a collection of iterable items, the `flatten` function flattens out the elements of the collection.\n\n```nim\ncheck(@[@[1,2],@[3],@[4,5,6]] --\u003e flatten() == @[1,2,3,4,5,6])\n```\n\n\n#### indexedFlatten\n\nIs similar to `flatten`, except that it returns the index inside original sub-lists with the actual content.\n```nim\ncheck(@[@[1,2],@[3],@[4,5,6]] --\u003e indexedFlatten()            == @[(0,1),(1,2),(0,3),(0,4),(1,5),(2,6)])\ncheck(@[@[1,2],@[3],@[4,5,6]] --\u003e flatten() --\u003e map((idx,it)) == @[(0,1),(1,2),(2,3),(3,4),(4,5),(5,6)])\n```\nNote that as in the other `indexed` commands the tuples are named tuples and the index is also accessible via `.idx` and the actual element with `.elem`.\n\n### combinations\n\nCombines each element of the original collection with each other - the resulting variable is an array of 2 containing the combined iterator values. In case no other collection is supplied the combinations are done on the input collection - only combining different elements with each other.\n```nim\n# combine collection with itself results in unordered combinations - only different elements are combined\ncheck(@[1,2,3] --\u003e combinations() == @[[1,2],[1,3],[2,3])\n# combine collection with the same collection again as a parameter results in comparing all elements in \n# all possible ordered combinations\ncheck(@[1,2,3] --\u003e combinations(@[1,2,3]) == @[[1,1],[1,2],[1,3],[2,1],[2,2],[2,3],[3,1],[3,2],[3,3]])\n\n# combine all elements of first collection with elements of the second collections\ncheck(@[1,2,3] --\u003e combinations(@[4,5]) == @[[1,4],[1,5],[2,4],[2,5],[3,4],[3,5]])\n```\n\n#### indexedCombinations\n\nSame as `combinations` with the additional indices of the resulting combined elements. The resulting iterator is a named tuple with the combined elements and their indices `(idx: [idx1, idx2], elem: [combined_element1, combined_element2])`.\n```nim\n# find the indices of the elements in the collection, where the diff to the other element is 1\ncheck(@[11,2,7,3,4] --\u003e indexedCombinations() --\u003e filter(abs(it.elem[1]-it.elem[0]) == 1) --\u003e map(it.idx) == @[[1,3],[3,4]])\n#          ^   ^ ^\n```\n\n### concat\n\nConcatenates the given arguments to one iterator. Can only be used as first command preceeding `--\u003e`.\n```nim\ncheck(concat([1,2],[3,4]) --\u003e to(seq) == @[1,2,3,4])\n```\nAlternatively `zf_concat` can be used to create an iterator that concatenates the given iterators to one.\n```nim\nzf_concat(con, @[1], (2,3,4))\ncheck(con() --\u003e to(seq) == @[1,2,3,4])\n```\n\n### group\nCreates a table containing result of each element applied to a discriminator function as keys and the corresponding elements in sequences as values.\n\n```nim\n# taking the last character of the strings as key\nlet m = @[(1,\"one\"),(2,\"two\"),(3,\"three\")] --\u003e group(it[1][^1])\ncheck(m['e'] == @[(1, \"one\"), (3, \"three\")])\ncheck(m['o'] == @[(2, \"two\")])\n```\n\n### partition\nSimplified version of `group` and alternative version to `filter` that returns a tuple with all elements that match the filter condition as first tuple element named `yes` and all non-matching as the second named `no`.\n\n```nim\nproc isEven(i: int): bool = (i and 1) == 0\n  let a = @[1,2,3,4,5,6]\nlet p = a --\u003e partition(it.isEven())\ncheck(p.yes == @[2,4,6])\ncheck(p.no == @[1,3,5])\n```\n### to\n\nFinally, it is possible to force the result type to the type given in `to` - which is only allowed as last argument when generating collection results (e.g. `map` or `filter` are the last arguments before `to`).\nThis method is handled differently from the others and removed internally so the command before `to` is the actual last command.\n\nWhen the result type is given as `seq`, `array` or `list` (the latter is mapped to `DoublyLinkedList`) then the template argument can be determined automatically.\nHowever, when all auto detection fails, the result type may be given explicitly here - the resulting code is also a bit more efficient for the compilation process.\n\n```nim\ncheck([1,2,3]) --\u003e to(seq) == @[1,2,3])\nvar l = @[1,2,3] --\u003e map($it) --\u003e to(list)\nlet l2: DoublyLinkedList[string] = l\necho(l2)\n```\n\n`to` supports a second parameter `autoConvert`. This actually only suppresses a warning when the actual result type and the added type are not identical but can be cast to another.\nA bit more complex example:\n```nim\n# doing a complex conversion from `seq[seq[uint8]]` to `seq[seq[int]]`\ncheck(@[@[1u8, 2u8], @[3u8]] --\u003e map(it --\u003e to(seq[int],true)) --\u003e to(seq[seq[int]]) == @[@[1,2], @[3]])\n```\n* the `map` call itself contains a new `--\u003e` call, i.e.: each internal list is converted from `seq[uint8]` to `seq[int]`\n* the parameter `true` is necessary to suppress a warning that the conversion is done by casting (each element)\n* the final result is a `seq[seq[int]]` - this conversion is optional (maybe to document) and can be determined automatically.\n\n#### createIter\n\nWhen using `createIter(name:string,closure:bool=false)` as last function then an iterator `name` is created which can be used for further processing with zero-functional with only a small overhead.\nSimilar to `to` this is also a virtual function which is internally replaced and only used to check the output type.\nThe generated iterator is inline by default and can not be returned from a proc or given to another proc (see [Nim: Iterators](https://nim-lang.org/docs/manual.html#iterators-and-the-for-statement-first-class-iterators)).\n\nTo create a closure iterator, the optional argument `closure` has to be set to `true`. A closure iterator is created which can be used as a return result from a procedure or as a parameter to a procedure. *This does not work with JS backend!*\n\n```nim\nimport zero_functional\nimport strutils\n# filter all lines containing the word hint in the iterator\nlines(\"nim.cfg\") --\u003e filter(\"hint\" in it.string) --\u003e createIter(errorLines)\nerrorLines() --\u003e foreach(echo it)\n```\n\n## Extending zero-functional\n\nExtending zero-functional with own functions is probably more complicated than with other fp-libraries as the functions have to be implemented with macros producing inlined imperative code. \nSome good examples from basic to more complicated can be found in [test.nim: registerExtension](test.nim) and in the source code of [zero-functional](zero-functional.nim)\n\n### Extending with plain nim\nWhen adding your own `foo` implementation you can write your own `inlineFoo` proc and register it with zero_functional. It should look like this:\n```nim\nimport macros # do not forget to import macros when extending zero_functional \n\nproc inlineFoo*(ext: ExtNimNode) {.compileTime.} =\n  # do some parameter checks\n  if ext.node.len \u003e SOME_MAX:\n    zfFail(\"too many arguments in \\'$1\\', got $2 but expected only $3\" %\n        [ext.node.repr, $(ext.node.len - 1), $SOME_MAX])\n  # do some stuff\n  # ...\n  # the actual 'loop' section code\n  ext.node = quote:\n    # do something\n\n# ....\n# after the function declarations register the extensions during compile time for use with zero_functional\nstatic:\n  zfCreateExtension()\n  \n# use foo after the registration\n```\n\nThe vanilla `inline`-proc implementations should follow certain rules. \n- `ext: ExtNimNode` as parameter\n- ExtNimNode should be used when implementing the functions with some helpers:\n  - `ext.node` = place the actual code here that is being generated inside the current block \n\tinitially `ext.node` contains the current function call and its parameters (section: loop)\n  - `ext.initials` = add the initialization code for variable definitions (section: init)\n  - `ext.endLoop` = add code that can be inserted at the end of the loop (section: end)\n  - `ext.finals` = add code after the loop - e.g. to calculate a result (section: final)\n  - `ext.res` = helper: access to the function's result\n  - `ext.prevItNode()` = access to the iterator generated in the previous statement or loop\n  - `ext.nextItNode()` = generates a new iterator for the current block. This is the (intermediate) result of the current operation that can be used with the next function\n- check of parameters / number of parameters has to be done in the implementation \n- use `zfFail()` if any checks fail\n- register functions that use neither `zfInline` nor `zfInlineCall` using `zfAddFunction`\n- functions that create another sequence have to be registered with `zfAddSequenceHandlers`\n- finally call `zfCreateExtension` after all `zfInline...` definitions and `zfAddFunction` calls - before using the actual function implementation\n\n### Writing extensions with Zero-DSL\nExample - implement filterNot function:\n```nim\nimport macros # do not forget to import macros when extending zero_functional\n\nzfInline filterNot(condition: bool):\n  loop:\n    if not condition:\n      yield it   \n\n# ....\n# after the function declarations register the extensions during compile time for use with zero_functional\nstatic:\n  zfCreateExtension()\n  \n# use foo after the registration\n```\n\n`zfInline` is the actual macro that takes the created function name (here: `map`) and its parameters and a body with different sections as input.\n`zfInline_dbg` will print the generated code `proc inlineFilterMap` (see example for map below).\n\nThe sections directly map to their counterparts in `ExtNimNode`:\n- `pre` prepare section: initialize variables and constants. It is possible to do the entire implementation in the `pre` section.\n\t- all variables that are used in other sections have to be defined here!\n- `init` variable initialization before the loop\n- `loop` the actual loop action (maps to `ext.node`)\n- `delegate` delegate to other functions (like map, filter, etc.)\n- `endLoop` added to end of the loop\n- `final` after the loop section - e.g. to set the result or yield the remaining data\n\nLet's dissect the (simple) map implementation:\n``` nim\nzfInline map(f):\n  loop:\n    let it = f # create the next iterator in the loop setting it to the given parameter of the map function\n```\n\nThe above `map` definition will be translated to:\n```nim\nproc inlineMap*(ext: ExtNimNode) {.compileTime.} =\n  # do some parameter checks\n  if ext.node.len - 1 \u003e 1:\n    zfFail(\"too many arguments in \\'$1\\', got $2 but expected only $3\" %\n        [ext.node.repr, $(ext.node.len - 1), $1])\n  let f = \n    if ext.node.len \u003e 1:\n      adapt(ext, 1) # replace all occurences if internal iterator \"it\" with __it__0, __it__1, etc.\n    else: # assert that the argument 'f' is supplied\n      zfFail(\"missing argument \\'$1\\' for \\'$2\\'\" % [\"f\", \"map\"])\n      newIntLitNode(0)\n  let nextIdent = ext.nextItNode() # create the next iterator\n  # the actual 'loop' section\n  ext.node = quote:\n    let `nextIdent` = `f` # here the `it` is replaced by the next iterator\n```\n\nThe Zero-DSL `map` function does not set the `result` as opposed to the `count` or `index` definition below - hence the result type is a collection result type, which is determined automatically by the zero_functional framework.\nThe `it` again is seen as keyword and the definition `let it = ...` will internally set the new iterator value which is consequently used by the next functions. In the generated macro it is replaced by the call `ext.nextItNode`.\n\nWhile Zero-DSL is quite powerful, not all possibilities can be handled by it when implementing a function. For instance the `foreach` implementation is done completely manually and `reduce` and other functions use the macro `zfInlineCall` which provides Zero-DSL within a manual function implementation and also registers the function name.\nThe signature for creating an inline function is as in the `inlineMap` example above - each function `foo` is implemented by its `inlineFoo*(ext: ExtNimNode)` counterpart.\n\nIf the Zero-DSL should fail to create an own implementation of a function then `zfInline_dbg` instead of `zfInline` can be used to print the created function to the console, copy it - remember to add the `*` to the name - and adapt the code.\n\nIt is possible to set parameter types for the functions - for example in the `index` implementation:\n```nim\nzfInline index(cond: bool):\n  init:\n    result = -1 # index not found\n  loop:\n    if cond:\n      return idx\n```\nThe `cond: bool` definition adds additional compile time checks to the generated macros, so that when using the `index`-function with a different type than `bool` a compile error with the wrong parameter and the expected type is created.\nIn this example also the `idx` variable is replaced automatically with the running index that is increment during the loop.\n\nNote: zfCreateExtension must be called in a compile time context. This could be in a static block or macro for example:\n```nim\n# Register above extensions during compile time\nstatic:\n  zfCreateExtension()\n```\n\n\n#### Special variables\nSpecial variables for `zfInline` statements are:\n- `it`: when used is the previous iterator, when defined with `let it = ` creates a new iterator, in the `init` section `it` refers to the first element in the underlying collection\n- `idx`: the running index in the loop\n- `result`: the overall result and return type of the operation\nAll other variables have to be defined in the `pre`-section, also when automatically assigned, e.g. when overriding the `idx` variable or when accessing a reference to the list as `listRef`.\nSee `intersect` or `removeDoubles` implementation in [test.nim](test.nim) as an example.\n- `yield it`: opposed to setting the result this means that an iterator or a collection result is returned with `it` being added to it.\n- `yield \u003cvarname\u003e` in `final` section: the `\u003cvarname\u003e` is added to the result at the end\n\n#### Setting the result\nExample of `count` that sets a result:\n```nim\nzfInline count():\n  init:\n    result = 0\n  loop:\n    result += 1 # add one in each loop\n```\nFunctions that set a result in any section are considered final functions - no other function may follow. Opposed to using `yield` used for iterator results:\n\nExample of `filter` that returns a collection / iterator:\n```nim\nzfInline filter(cond: bool):\n  loop:\n    if cond:\n      yield it # add the current iterator to the resulting collection\n```\n\n### Defering to Zero-DSL inside plain nim\nAs Zero-DSL is limited in its expressiveness - e.g. there is no possibility to defer to a different loop implementation depending on the input type - it is possible to defer to different Zero-DSL implementations in the nim code using `zfInlineCall`. Example calling two implementations - one specialized to work on lists, the other for sequences:\n```nim\nproc inlineFoo*(ext: ExtNimNode) {.compileTime.} =\n  if ext.isListType():\n    # provide list implementation\n    zfInlineCall foo(param):\n      ... \n  else:\n    # provide plain implementation (e.g. using [])\n    zfInlineCall reduce(op):\n      ...\n```\n\n#### Creating compound commands with other commands\nZero-DSL can create commands as a sequence from already existing commands using the `delegate` section - which essentially calls the functions in that section.\nExample is the `removeDoubles` function that returns the input sequence with unique elements. See code in [test.nim](test.nim).\n```nim\nzfInline removeDoubles():\n# remove double elements. \n  pre:\n    # initialize variables in `ext` that are used below\n    let listRef = ext.listRef\n  delegate:\n    # this actually only works only on the original list / iterator\n    indexedCombinations(listRef) # combine with itself - all elements\n    # this is the tricky one: remove later elements that already are in the list\n    # this actually translates in the inner for loop of combinations as:\n    # if idx[0] \u003e idx[1] and it[0] == it[1]: break\n    takeWhile(not(it.elem[0] == it.elem[1] and it.idx[0] \u003e it.idx[1])) \n    # go back to the original elements\n    filter(it.idx[0] == it.idx[1]) \n    map(it.elem[0])\n```\n\n## Overview Table\n\nThe result type depends on the function used as last parameter.\n\n| Command       | 1st Param | in-between | Last Param | Result Type                 |\n| ------------- | --------- | ---------- | ---------- | --------------------------- |\n|all            |           |            |     +      | `bool`                      |\n|combinations   |   +       |    (+)     |    (+)     | `coll[Combination]`         |\n|concat         |   +       |            |            | coll                        |\n|exists         |           |            |     +      | bool                        |\n|filter         |   +       |     +      |     +      | part coll / zeroed array    |\n|find           |           |            |     +      | `int`                       |\n|flatten        |   +       |     +      |     +      | coll                        |\n|fold           |           |            |     +      | *                           |\n|foreach        |   +       |     +      |     +      | `void`                      |\n|group          |           |            |     +      | `table[*, seq[*]]`          |\n|index          |           |            |     +      | `int`                       |\n|indexedMap     |   +       |     +      |     +      | `seq[(int,*)]`              |\n|enumerate      |           |     +      |     +      | `seq[(int,*)]`              |\n|createIter     |           |            |  virtual   | iterator of given type      |\n|map            |   +       |     +      |     +      | `collType[*]`               |\n|partition      |           |            |     +      | `(yes:seq[*], no:seq[*])`   |\n|reduce         |           |            |     +      | *                           |\n|sub            |   +       |     +      |     +      | part coll / zeroed array    |\n|split          |           |            |     +      | `(seq[*],...seq[*])`        |\n|zip            |   +       |     +      |     +      | `seq[(*,..,*)]`             |\n|to             |           |            |   virtual  | given type                  |\n\n+ *: any type depending on given function parameters\n+ coll: is the input collection\n+ collType is the input collection type (without template argument)\n+ \"virtual\" function: can only be given as last argument, but does not count as last argument.\n\n## Debugging using `--\u003e\u003e`\n\nAs `zero_functional` macros are sometimes tricky to use, it can happen that the compiler crashes or that compile errors are hard to understand.\n\nTo see the actual code that is generated (provided the generation itself does not crash) you can use the `--\u003e\u003e` operator which prints the representation `repr` of the actual generated nim code.\nIt is also useful for checking what the macros are generating under the hood.\n\n```nim\nlet a = [1,2,3]\na --\u003e\u003e foreach(echo(it))\n```\n\nwill print during compilation:\n```nim\nblock:\n  for it0 in a:\n    echo(it0)\n```\nThe printed code can be copied to your actual program for further investigation.\n\n## Compile flags\n\nThe following compile flags are supported:\n* `-d:zf_iter` defaults to generating closure iterators instead of `seq` outputs.\n* `-d:zf_list` same as above but generating `DoublyLinkedList`\n* `-d:zf_debug_all` print all generated code\n\n## LICENSE\n\nMIT, Alexander Ivanov\n\n\n## Contributors\n\nCo-maintainers and authors: [Michael Schulte](https://github.com/michael72), Alexander Ivanov\n\nCreator: Alexander Ivanov\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzero-functional%2Fzero-functional","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzero-functional%2Fzero-functional","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzero-functional%2Fzero-functional/lists"}