{"id":16880603,"url":"https://github.com/squaremo/noodle","last_synced_at":"2026-05-17T13:15:52.211Z","repository":{"id":4781517,"uuid":"5933610","full_name":"squaremo/noodle","owner":"squaremo","description":"Lazy sequences, event streams and combinators, in portable-ish JavaScript","archived":false,"fork":false,"pushed_at":"2012-11-07T15:33:23.000Z","size":192,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-25T05:41:16.188Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"CoffeeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/squaremo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-09-24T11:35:27.000Z","updated_at":"2013-12-10T21:09:36.000Z","dependencies_parsed_at":"2022-08-18T04:10:11.432Z","dependency_job_id":null,"html_url":"https://github.com/squaremo/noodle","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squaremo%2Fnoodle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squaremo%2Fnoodle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squaremo%2Fnoodle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squaremo%2Fnoodle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/squaremo","download_url":"https://codeload.github.com/squaremo/noodle/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244544326,"owners_count":20469660,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T15:59:26.892Z","updated_at":"2026-05-17T13:15:47.188Z","avatar_url":"https://github.com/squaremo.png","language":"CoffeeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Noodling around with sequences\n\nSome modules with an encoding of sequences (streams, whatever) and the\naccompanying combinators. Usable from either CoffeeScript or Node.JS,\nand if you can figure out how, in browsers. Examples below are in\nCoffeeScript.\n\nThere's two kinds of sequences here, which I'll describe as\n*demand-driven* (new values are calculated by need, like lazy\nsequences in Clojure), and *producer-driven*, in which there's some\nprocess adding values onto the tail of the sequence, while another\nprocess chases the head of the sequence.\n\nOne can give them the same treatment given a careful encoding. The one\nI've chosen is that a sequence is a function, which when given a\n\"menu' of `Cons` and `Nil` continuations (callbacks, if you prefer),\nwill either provide a head and tail to cons, or call nil to indicate\nthe end of the sequence. Because the continuations are reified we can\nstow them away if we don't currently know the answer, which is why we\ncan encode producer-driven sequences. It also lets us define sequence\ncombinators (map, filter) rather simply.\n\nThere's an additional twist: writing certain combinators typically\nrequires some recursion, which in a language like JavaScript with no\nconstant-stack tail call will easily overflow the stack. For this\nreason, an additional continuation -- `Skip` -- is introduced to the\nmenu, which is called when the computation needs to proceed without\nconsidering an element (e.g., in filter).\n\n## Sequences\n\nLet's treat with demand-driven sequences first. With these, a value is\navailable every time you ask for one -- which means you can iterate\nthrough values in a loop (`doSeq` does this), so long as there aren't\ninfinitely many.\n\n### Constructors\n\n#### `NIL`\nis the sequence with no values.\n\n#### `cons(head, tailfn)` \nconstructs a sequence with the `head` and the tail yielded by the\nthunk `tailfn`. The tail is guarded this way so that one may construct\nrecursively-defined sequences. (These don't necessarily make a lot of\nsense for JavaScript, outside of fun examples)\n\nFor example,\n\n```coffeescript\nalternate = cons(true, -\u003e cons(false, -\u003e alternate))\n```\n\ngives the sequence `true, false, true, false, ...`\n\n#### `unfold(seed, fn)`\n\ngenerates an infinite sequence by applying fn, first to seed, then to\nsuccessive return values.\n\nFor example,\n\n```coffeescript\nunfold(0, (x) -\u003e x + 1)\n```\n\ngives a sequence of the natural numbers.\n\n#### `fromArray(values)`\nis a convenience for constructing a finite sequence of the values\ngiven. This is useful for writing tests (those tests there are) of\ncourse, but also for lifting results of \"regular\" procedures into\nsequences. See for example the implementation of `split`.\n\n### Combinators\n\n#### `map(fn, seq)`\ngives a sequence of the function `fn` applied to the values of `seq`.\n\n#### `filter(predicate, seq)`\ngives a sequence of the values of `seq` for which `predicate` returns\n`true`.\n\n#### `concatMap(fn, seq)`\nis like `map`, but expects `fn` to return a sequence; successive such\nvalues are concatenated to yield a \"flat\" sequence. Useful (possibly\nin conjunction with `fromArray`) for mapping a function which may\nreturn zero or more values.\n\n#### `zipWith(fn, a, b)`\ngives a sequence that consists of `fn` applied to values of `a` and\n`b` point-wise. For example,\n\n```coffeescript\nzipWith(((x, y) -\u003e x + y), a, b)\n```\n\ngives the sequence of the first value of a plus the first value of b,\nthen the second value of a plus the second value of b, and so\non. Either sequence ending will end the result.\n\n#### `lift`\ntakes a unary operation on values and gives an operation on sequences;\nin other words,\n\n```coffeescript\nlift(fn) === (a) -\u003e map(fn, a)\n```\n\n#### `lift2`\ntakes a binary operation on values and gives a binary operation on\nsequences.\n\n```coffeescript\nlift2(fn) === (a, b) -\u003e zipWith(fn, a, b)\n```\n\n### Operations\n\n#### `take(n, seq)`\ngives a sequence with only the first `n` values of `seq`, or fewer if\n `seq` has fewer than `n`.\n\n#### `drop(n, seq)`\ngives `seq` after discarding `n` values.\n\n#### `tail(seq)`\ndiscards the first value in `seq` and yields the remaining\nsequence. For simplicity a sequence with no values is treated as its\nown tail.\n\n#### `memoise(seq)`\ngives a sequence which remembers values once they have been computed;\nthis is sometimes necessary for avoiding exponential blowout in\nrecursively-defined sequences. Since this forces the whole sequence to\nbe kept in memory, it's a trade-off.\n\n#### `replace(a, b)`\n\nyields a value of `b` for each value of `a` realised. This is useful\nwhen `a` is \"driving\" the computation, but it's the values of `b` you\nwant; for example, if `a` is incoming events, and `b` is a count.\n\n#### `doSeq(proc, seq)`\napplies the (typically side-effecting) procedure `proc` with each\nvalue of `seq`. For example,\n\n```coffeescript\ndoSeq(console.log, take(100, unfold(0, (x) -\u003e x + 1)))\n```\n\n`doSeq` spins in a while loop, and for this reason is only suitable\nfor demand-driven sequences, and finite ones at that; otherwise it'll\nhappily spin forever.\n\n## Event streams\n\nEvent streams (for want of a better term) are producer-driven\nsequences. This means you get both the sequence and the means of\ninjecting new values into it (and of ending it). The idea is that the\nvalues come from I/O of some kind; for instance, a DOM event handler,\nor a socket.\n\nEvent streams work with the combinators and operations above. Using\n`doSeq` will spin because of the way it's written as a loop; a\nreplacement, `doEvents`, is given for event streams.\n\n#### `events()`\nyields an object `{stream, inject, stop}`. `stream` is the initial\nsequence head; `inject` adds another value to the sequence tail; and\n`stop` ends the sequence.\n\n#### `doEvents(proc, seq)`\napplies `proc` to each value of `seq`. This will recurse in the case\nof demand-driven sequences, so use only with event streams. It returns\na promise which is resolved at the (possibly never-arriving) end of\nthe stream.\n\n## Strings\n\n#### `split(seq, char)`\ngives a sequence of the concatenation of values in `seq` (assuming\nthey are strings), split into substrings at each instance of the\ncharacter `char`.\n\nFor example,\n\n```coffeescript\nsplit(fromArray(['foo\\nba', 'r\\nbaz\\n', 'boo']), '\\n')\n```\n\ngives the sequence `'foo', 'bar', 'baz', 'boo'`.\n\n## Node.JS integration\n\nThe obvious point of integration with Node.JS is with\n`stream.Stream`. Reduced to the essentials, streams are event emitters\nthat emit `'data'` until there are no more (for some reason), then\nemit `'end'`; and, they include a method `pipe` which will read from\none stream (the source) and write to another, until the source ends.\n\nTechnically, one can write all the combinators above for use with these\nstreams (well probably, I haven't tried all of them). But you end up\nwith funny little state machines, and explicit buffering; and, you\ncan't give demand-driven sequences the same treatment.\n\nLuckily the event streams are pretty easy to convert, both into\nwritable streams (using the `inject` bit) and readable streams (using\nthe stream head and `doEvents`). In fact, so that we can propagate\nbackpressure, we'll want to do both; however, the readable bit (where\nvalues come out) is *after* any computation, and the writable bit\n(where values go in) is before any computation. For this reason, we\nconstruct the whole thing at once, supplying the computation as a\ntransformation of the input sequence (writable) to the output sequence\n(readable).\n\n#### `stream(transformer)`\n\nconstructs a readable and writable stream; the function `transformer`\naccept a sequence (that will be written to) and returns a sequence\nthat will be read from. Backpressure is propagated, and `pipe` is\navailable.\n\nFor example, reading the comment lines from one file into another\nfile, assuming `infile` and `outfile` are already created (with, for\ninstance, `fs.createReadStream` and `fs.createWriteStream`\nrespectively):\n\n```coffeescript\nisComment = (x) -\u003e x.trim()[0] == '#'\nnewlines = lift((x) -\u003e x + '\\n')\ninfile.setEncoding('utf8')\ncomments = stream((s) -\u003e newlines(filter(isComment, split(s, '\\n'))))\ninfile.pipe(comments).pipe(outfile)\n```\n\nNote that using pipe will tend to end the downstream when the upstream\nends; so in the above example, `comments` gets ended (and thus so does\noutfile).\n\n## References\n\nThere's a great survey of streams (lazy sequences) in the preface to\n[SRFI-40](http://srfi.schemers.org/srfi-40/srfi-40.html), and\nadditional discussion in\n[SRFI-41](http://srfi.schemers.org/srfi-41/srfi-41.html).\n\nThe technique of having a `Skip` constructor to avoid recursion I have\nlifted from the paper [\"Stream\nFusions\"](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.104.7401). There\nit's used to give the compiler an opportunity to optimise away\nintermediate steps; here I've used it to avoid blowing the stack,\nsince JavaScript does not execute tail calls in constant stack. Sadly\nI don't get the same chance at optimisation with JavaScript, so\nthere's no deforestation effect.\n\nI've taken a couple of notions from Clojure's lazy sequences, though\nthey are of a slightly different nature; in particular, calling them\n\"sequences\", and the name of the procedure `doSeq`. Clojure's lazy\nsequences use a different scheme to avoid recursion (it doesn't have\nproper tail calls either): it delays the whole value (the cons cell,\nif you like), and forces it in the primitive `seq`, which will iterate\nuntil it gets a realised sequence. Thus, `filter` and the like are\nfree to keep returning delayed values that will themselves yield\nfurther delayed sequences. This is pretty similar in effect to the\nstream fusion scheme.\n\nI haven't seen the continuation-passing variety of streams elsewhere,\nbut of course this style is almost mandatory in typical JavaScript\nenvironments.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsquaremo%2Fnoodle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsquaremo%2Fnoodle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsquaremo%2Fnoodle/lists"}