{"id":22762494,"url":"https://github.com/sabiwara/iter","last_synced_at":"2025-04-05T07:02:53.298Z","repository":{"id":65102968,"uuid":"581988959","full_name":"sabiwara/iter","owner":"sabiwara","description":"A blazing fast compile-time optimized alternative to the `Enum` and `Stream` modules","archived":false,"fork":false,"pushed_at":"2024-12-13T10:47:32.000Z","size":100,"stargazers_count":64,"open_issues_count":0,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-29T06:02:35.388Z","etag":null,"topics":["elixir","enum","enumerable","metaprogramming","performance","stream"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sabiwara.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-25T07:26:10.000Z","updated_at":"2025-01-28T23:11:56.000Z","dependencies_parsed_at":"2023-12-23T08:19:31.157Z","dependency_job_id":"9f4eba5c-eb0c-4591-98cf-71eeb8776099","html_url":"https://github.com/sabiwara/iter","commit_stats":{"total_commits":14,"total_committers":3,"mean_commits":4.666666666666667,"dds":0.1428571428571429,"last_synced_commit":"b8fd85e86c4b6a6e0a0b3a133020f77aac27a35b"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sabiwara%2Fiter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sabiwara%2Fiter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sabiwara%2Fiter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sabiwara%2Fiter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sabiwara","download_url":"https://codeload.github.com/sabiwara/iter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247299831,"owners_count":20916190,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","enum","enumerable","metaprogramming","performance","stream"],"created_at":"2024-12-11T10:08:17.097Z","updated_at":"2025-04-05T07:02:53.279Z","avatar_url":"https://github.com/sabiwara.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Iter\n\n[![Hex Version](https://img.shields.io/hexpm/v/iter.svg)](https://hex.pm/packages/iter)\n[![docs](https://img.shields.io/badge/docs-hexpm-blue.svg)](https://hexdocs.pm/iter/)\n[![CI](https://github.com/sabiwara/iter/workflows/CI/badge.svg)](https://github.com/sabiwara/iter/actions?query=workflow%3ACI)\n\n\u003c!-- MDOC !--\u003e\n\nA blazing fast compile-time optimized alternative to the `Enum` and `Stream`\nmodules.\n\n## Overview\n\n`Iter` allows you to effortlessly write highly-efficient pipelines to process\nenumerables, in a familiar and highly readable style.\n\n```elixir\niex\u003e require Iter\niex\u003e 1..10 |\u003e Iter.map(\u0026 \u00261 ** 2) |\u003e Iter.sum()\n385\n```\n\n`Iter` will merge both the `map` and `sum` steps and perform both in one single\npass. Unlike the same pipeline written with `Enum`, it won't build any\nintermediate list, therefore saving memory and CPU cycles.\n\nYou can think of `Iter` as compile-time streams, or as comprehensions on\nsteroids. It should be highly efficient compared to the same pipeline written\nwith `Stream`, since it does most of the work at compile time without any\nruntime overhead. And while it actually works very similarly to `for/1` under\nthe hood and basically emits the same code, it offers a much more flexible,\ncomposable and extensive API.\n\nBecause `Iter` is compile-time, these are macros and not functions. This has\nseveral implications:\n\n- you have to `require` the module first before using it\n- they won't appear in the stacktrace in case of errors (but `Iter` tries to\n  make sure that stacktraces will point to the line of the step responsible)\n- if you \"break\" the pipeline, `Iter` won't be able to optimize it as a single\n  pass: it will suffer the same issue as `Enum`\n\n```elixir\n1..10\n|\u003e Iter.map(\u0026 \u00261 ** 2)\n|\u003e IO.inspect() # \u003c= pipeline broken, creates an intermediate list\n|\u003e Iter.sum()\n```\n\nWhen there is no possibility of merging steps, `Iter` is simply delegating to\n`Enum` which is optimized plenty on individual steps.\n\n\u003c!-- MDOC !--\u003e\n\n## Performance\n\nTo quote Jose Valim talking\n[about comprehensions](https://elixirforum.com/t/enum-fusion/36193/30):\n\n\u003e \"as soon as you start piping into multiple `Enum` and `Stream`, `for` should\n\u003e win by **considerable margins**.\"\n\nThe exact same thing can be said about `Iter`, which delivers the same\nperformance as comprehensions.\n\nThe `benchmarks` folder illustrates how `Iter` compares to `Enum` or `Stream`\nthrough some examples. Results might of course vary greatly depending on the\ninput (size, type) and the nature of the operations. Here is a brief summary\njust to illustrate the kind of multipliers that could be expected (input: list\nof length 100):\n\n|                   | `map \\|\u003e filter` | `map \\|\u003e reverse` | `map \\|\u003e find` | `map \\|\u003e sum`  |\n| ----------------- | ---------------- | ----------------- | -------------- | -------------- |\n| `Enum` / `Iter`   | 2.85x slower     | 2.35x slower      | 3.70x slower   | 2.96x slower   |\n|                   | 2.08x memory     | 1.74x memory      | 59.63x memory  | 41.00x memory  |\n| `Stream` / `Iter` | 2.92x slower     | 2.89x slower      | 4.99x slower   | 3.09x slower   |\n|                   | 7.25x memory     | 5.24x memory      | 26.25x memory  | 173.60x memory |\n\nFor operations like `sum/1` or `find/2`, space complexity is even changed from\nlinear to constant, yielding vast improvements for large inputs.\n\n## Installation\n\n`Iter` can be installed by adding `iter` to your list of dependencies in\n`mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:iter, \"~\u003e 0.1.2\"}\n  ]\nend\n```\n\nThe documentation can be found at\n[https://hexdocs.pm/iter](https://hexdocs.pm/iter).\n\n## Motivation\n\n`Iter` aims to provide a production-ready alternative to the `Enum` and `Stream`\nmodules, that allows to write enumerable pipelines in a familiar way without a\nneed to be concerned about efficiency.\n\nPremature optimization is the root of all evil. But by consistently providing\nbetter performance out of the box, `Iter` aims to help focusing more on writing\nreadable code, and to remove the tradeoff between readability and performance.\n\nWhile `Iter` is close to offer a drop-in replacement for `Enum`/`Stream`, it\ndoesn't aim to be an exact one. The [Consistency section](#consistency) below\ncovers the differences with the standard library.\n\n\u003c!-- MDOC !--\u003e\n\n## Consistency\n\n`Iter` is mostly consistent with the standard library, but it is prioritizing\nefficiency over absolute consistency with the `Enum` and `Stream` modules, which\nimplies some slight differences. These differences are always documented in the\nconcerned macro docs.\n\n### Negative indexes\n\n`Iter` only supports positive indexes when inside a pipeline, so most of\nfunctions like `at/1`, `slice/1` or `take/1` which would also accept negative\nindexes cannot be replaced in cases needing it. The reason is simple: working\nwith negative indexes implies to materialize the whole list once. If you need\nit, you should replace the relevant step to use `Enum`, or maybe call\n`Iter.reverse/1` before accessing it (see\n[_Collecting the pipeline_](#collecting-the-pipeline) section).\n\n### API differences\n\n`Iter` should cover most of the `Enum` API, but:\n\n- some operations are still missing\n- some operations won't be added because cannot be implemented efficiently\n- some extra functions are being provided: `Iter.mean/1`, `Iter.first/1`,\n  `Iter.last/1` (to compensate with the lack of negative indexes) and\n  `Iter.match/3` (pattern-match to filter and extract at once, like in\n  comprehensions)\n\n## Collecting the pipeline\n\nSome operations like `Iter.to_list/1`, `Iter.reverse/2`, `Iter.reduce/3`,\n`Iter.group_by/2`... need to materialize an intermediate list or accumulator and\nwill collect the pipeline.\n\nOperations that are collecting the pipeline are always mentioning it in their\ndocumentation.\n\nHere is a simple example:\n\n```elixir\nusers\n|\u003e Iter.map(\u0026fetch_user/1)\n|\u003e Iter.reject(\u0026is_nil/1)\n|\u003e Iter.each(\u0026process_user/1)\n```\n\nThe pipeline above will start processing users as they are retrieved, in one\nsingle pass. But assuming we want to make sure to be able to first retrieving\nall users before starting the processing step, `Iter.to_list/1` can be used to\nmake the intent explicit:\n\n```elixir\nusers\n|\u003e Iter.map(\u0026fetch_user/1)\n|\u003e Iter.reject(\u0026is_nil/1)\n|\u003e Iter.to_list()  # forcing the pipeline to collect\n|\u003e Iter.each(\u0026process_user/1)\n```\n\nForcing a pipeline to collect through `Enum.to_list/1` (or `Enum.reverse/1`,\nfaster) can also be used to circumvent some of `Iter`'s limitations like the\nabsence of negative indexes support:\n\n```elixir\nfoo\n|\u003e Iter.map(\u0026bar/1)\n|\u003e Iter.to_list()  # forcing the pipeline to collect\n|\u003e Iter.take(maybe_negative_index)\n```\n\nIn the example above, `Iter.take/2` is now the only step of its pipeline and can\nsupport negative indexing. The extra pass required is made explicit.\n\n\u003c!-- MDOC !--\u003e\n\n## Resources\n\nStill learning `Enum` and struggling to find the right function? Make sure to\ncheck our [Ultimate`Enum` cheatsheet](https://hexdocs.pm/iter/enum.html)!\n\n## Copyright and License\n\nIter is licensed under the [MIT License](LICENSE.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsabiwara%2Fiter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsabiwara%2Fiter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsabiwara%2Fiter/lists"}