{"id":17252052,"url":"https://github.com/ppetr/scala-conduit","last_synced_at":"2025-03-26T07:23:04.273Z","repository":{"id":7365773,"uuid":"8691375","full_name":"ppetr/scala-conduit","owner":"ppetr","description":"Scala library implementing the concept of functional pipes/conduits.","archived":false,"fork":false,"pushed_at":"2013-07-25T18:55:40.000Z","size":525,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-31T08:44:20.099Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ppetr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-03-10T20:53:58.000Z","updated_at":"2017-08-10T22:45:21.000Z","dependencies_parsed_at":"2022-09-21T11:02:43.265Z","dependency_job_id":null,"html_url":"https://github.com/ppetr/scala-conduit","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppetr%2Fscala-conduit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppetr%2Fscala-conduit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppetr%2Fscala-conduit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppetr%2Fscala-conduit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ppetr","download_url":"https://codeload.github.com/ppetr/scala-conduit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245606484,"owners_count":20643188,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-15T06:52:55.150Z","updated_at":"2025-03-26T07:23:04.252Z","avatar_url":"https://github.com/ppetr.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# scala-conduit\n\n_Currently the library is experimental and the interface is subject to change._\n\n[![Build Status](https://secure.travis-ci.org/ppetr/scala-conduit.png?branch=master)](https://travis-ci.org/ppetr/scala-conduit)\n\n## Overview\n\nA `Pipe` is a component that receives objects of a given type and responds with objects of another type. It is similar to [UNIX pipelines](https://en.wikipedia.org/wiki/Unix_pipeline), except that instead of receiving and sending bytes it receives and sends specific objects. When a pipe finishes processing, it also produces a final result.\n\nThere are two main ways how to compose pipes:\n\n- Sequence them. When the first pipe finishes processing, the second pipe continues. It's similar to simple sequencing of commands in a UNIX pipeline.\n- Fuse them. The output of the first pipe (called _upstream_) is used as the input for the second one (called _downstream_). And if the first pipe finishes processing, the second pipe also receives its final result. This is similar to combining UNIX pipelines with `|`.\n\nUnlike UNIX pipelines, when a pipeline is run, it is executed in a single thread and values are only generated on demand. A downstream pipe is only invoked when an upstream pipe requests input and is suspended again when it produces the requested value. So if an upstream pipe requests no input at all, the downstream pipe is never invoked.\n\nThis library aims at deterministic and correct handling of resources. Each pipe operation has a finalizer that is executed when the processing is interrupted for whatever reason.\n\nThe library is conceptually similar to Haskell's [conduit](http://hackage.haskell.org/package/conduit) library.\n\nRequires Scala 2.10.\n\n## Scaladoc\n\nGenerated documentation is available [here](http://ppetr.github.com/scala-conduit/#conduit.package). It is updated only from time to time so it might not reflect the latest changes.\n\n## Core concepts\n\n_Note:_ To understand how type parameters of pipes work, you need to understand Scala's [covariant and contravariant types][covariance].\n\n  [covariance]: https://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science)\n\n### Pipe\n\nA general pipe is defined as\n\n```scala\nsealed trait GenPipe[-U,-I,+O,+R]\n```\n\nwhere:\n\n- `U` is the type of the value received from upstream when there is no more input available. This is the final result of the upstream pipe.\n- `I` is the type of values received from upstream.\n- `O` is the type of output values produced by this pipe and sent downstream.\n- `R` is the final result of the pipe when it finishes processing.\n\nIn majority of cases a pipe doesn't care about the result of its upstream pipe, so `U` is set to `Any`. This is so common that we define alias\n\n```scala\ntype Pipe[-I,+O,+R]     = GenPipe[Any,I,O,R]\n```\n\n#### Sources and sinks\n\nA _source_ is a pipe that doesn't care about any possible input, so its input\ntype is `Any`.. Usually a source never requests input from upstream.\n\nA _sink_ is a pipe that never produces output.\n\nWe also need to distinguish pipes that cannot receive any input. Their `I` parameter is set to `Nothing`. We'll use this kind of pipes to express how to continue processing when input is exhausted.\n\nWe define type aliases for these concepts as:\n\n```scala\ntype Sink[-I,+R]        = Pipe[I,Nothing,R]\ntype Source[+O,+R]      = Pipe[Any,O,R]\ntype NoInput[-U,+O,+R]  = GenPipe[U,Nothing,O,R]\n```\n\n\nTODO\n\n## Examples\n\nSee Util.scala, IO.scala and NIO.scala for full code.\n\n### Read a file and output its contents as `String` lines.\n\nFirst we create a source pipe that takes no input and lists a given file as its output:\n\n```scala\ndef readLines(r: BufferedReader): Source[String,Unit] = {\n  implicit val fin = closeFin(r);\n  untilF[Any,Any,String](Option(r.readLine).map(respond[String] _));\n}\n```\n\nNote the implicit finalizer `fin`. Using Scala's `implicit` feature it is applied to all pipe operations so that the reader is closed properly when the pipe is interrupted. Here `closeFin` is simply\n\n```scala\nimplicit def closeFin(implicit c: Closeable): Finalizer =\n  Finalizer { System.err.println(\"Closing \" + c); c.close() }\n```\n\nUsing `readLines` we can construct a pipe that takes `BufferedReader`s at its input and outputs their contents:\n\n```scala\nval readLines: Pipe[BufferedReader,String,Unit] =\n  unfold[BufferedReader,String](readLines _);\n```\n\nFinally, we compose it with a simple mapping pipe that converts `File`s into `BufferedReader`s:\n\n```scala\nval readLinesFile: Pipe[File,String,Unit] = {\n  import Finalizer.empty\n  mapF((f: File) =\u003e new BufferedReader(\n                      new InputStreamReader(\n                        new FileInputStream(f), \"UTF-8\"))\n    ) \u003e-\u003e readLines\n}\n```\n\n\n\n### Display the contents of all Scala files in the current directory\n\n```scala\nrunPipe(\n  listRec(new File(\".\")) \u003e-\u003e\n  filter[File](_.getName().endsWith(\".scala\")) \u003e-\u003e\n  readLinesFile \u003e-\u003e\n  log\n)\n```\n\nwhere \n\n```scala\ndef listRec(dir: File): Source[File,Unit] =\n```\n\nis a source pipe that doesn't take any input and outputs recursively a list of all files in a directory (similar to UNIX [find](https://en.wikipedia.org/wiki/Find)),\n\n```scala\ndef filter[A,R](p: A =\u003e Boolean): GenPipe[R,A,A,R] = ...\n```\n\nis a pipe that checks its input values and passes trough only those that satisfy a given predicate,\n\n```scala\nval readLines: Pipe[BufferedReader,String,Unit] = ...\n```\n  \n`readLinesFile` is a pipe that for each `File` received on its input outputs all its lines as `String`s,\n\n```scala\ndef writeLines(w: Writer): Sink[String,Unit] = ...\nval log = writeLines(new OutputStreamWriter(System.out));\n```\n    \n\n`log` receives `String`s and prints them to [standard output](https://en.wikipedia.org/wiki/Standard_output#Standard_output_.28stdout.29).\n\n\n# Copyright\n\nCopyright 2013, Petr Pudlák\n\nContact: [petr.pudlak.name](http://petr.pudlak.name/).\n\n![LGPLv3](https://www.gnu.org/graphics/lgplv3-88x31.png)\n\nThis program is free software: you can redistribute it and/or modify it under\nthe terms of the GNU Lesser General Public License as published by the Free\nSoftware Foundation, either version 3 of the License, or (at your option) any\nlater version.\n\nThis program is distributed in the hope that it will be useful, but WITHOUT ANY\nWARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A\nPARTICULAR PURPOSE.  See the GNU Lesser General Public License for more\ndetails.\n\nYou should have received a copy of the GNU Lesser General Public License along\nwith this program.  If not, see \u003chttp://www.gnu.org/licenses/\u003e.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppetr%2Fscala-conduit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fppetr%2Fscala-conduit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppetr%2Fscala-conduit/lists"}