{"id":21074471,"url":"https://github.com/longevityframework/stream-adapter","last_synced_at":"2026-05-19T18:36:43.856Z","repository":{"id":88534520,"uuid":"82330814","full_name":"longevityframework/stream-adapter","owner":"longevityframework","description":"Conversions between various Scala streaming libraries","archived":false,"fork":false,"pushed_at":"2018-06-06T00:17:45.000Z","size":77,"stargazers_count":3,"open_issues_count":14,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-20T22:37:44.250Z","etag":null,"topics":["akka-streams","fs2","iteratee-io","play-framework","reactive-streams","scala"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/longevityframework.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-02-17T19:28:30.000Z","updated_at":"2018-06-06T00:17:46.000Z","dependencies_parsed_at":"2023-03-07T03:15:54.004Z","dependency_job_id":null,"html_url":"https://github.com/longevityframework/stream-adapter","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/longevityframework%2Fstream-adapter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/longevityframework%2Fstream-adapter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/longevityframework%2Fstream-adapter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/longevityframework%2Fstream-adapter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/longevityframework","download_url":"https://codeload.github.com/longevityframework/stream-adapter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243517180,"owners_count":20303579,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["akka-streams","fs2","iteratee-io","play-framework","reactive-streams","scala"],"created_at":"2024-11-19T19:16:23.343Z","updated_at":"2025-12-27T22:26:41.365Z","avatar_url":"https://github.com/longevityframework.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A collection of converters between various Scala streaming libraries\n\nDid you ever find yourself with a stream from one streaming library, but some other part of your\nprogram expects a stream from some other library? Do you want to provide streaming as part of your\nAPI, don't want to limit yourself to a single streaming library? `stream-adapter` provides tooling\nto help address both of these situations.\n\n## Converting between two streaming libraries\n\n`stream-adapter` provides conversions between the following Scala streaming libraries:\n\n- [Akka Streams](http://doc.akka.io/docs/akka/2.4.17/scala/stream/index.html)\n- [FS2](https://github.com/functional-streams-for-scala/fs2)\n- [iteratee.io](https://github.com/travisbrown/iteratee)\n- [Play enumerators](https://www.playframework.com/documentation/2.5.x/Enumerators)\n\nMore can be added relatively easily, as we will see below. Let's start with an Akka `Source`:\n\n```scala\nimport akka.actor.ActorSystem\nimport akka.stream.ActorMaterializer\nimport akka.stream.scaladsl.Source\nimplicit val actorSystem = ActorSystem(\"streamadapter\")\nimplicit val materializer = ActorMaterializer()\nval akkaSource = Source(0.until(10))\n```\n\nWe can convert this to an `fs2.Stream` like so:\n\n```scala\nval fs2Stream: fs2.Stream[cats.effect.IO, Int] = {\n  import streamadapter._\n  import streamadapter.akka._\n  import streamadapter.fs2._\n  adapt[AkkaSource, FS2Stream, Int](akkaSource)\n}\n```\n\nConverting streams requires a small handful of wildcard imports. I like to put these in an anonymous\nblock, as above, so they don't apply to the rest of the code in the file.\n\nUnfortunately, the effects for `fs2.Stream` is hardwired to `fs2.Task` for the moment. We should be\nable to handle other effects in the future.\n\nLet's now convert that FS2 stream into an `io.iteratee.Enumerator`:\n\n```scala\nval iterateeIoEnumerator: io.iteratee.Enumerator[cats.Eval, Int] = {\n  import scala.concurrent.ExecutionContext.Implicits.global\n  import streamadapter._\n  import streamadapter.fs2._\n  import streamadapter.iterateeio._\n  adapt[FS2Stream, EvalEnumerator, Int](fs2Stream)\n}\n```\n\nAll three of the type arguments to `adapt` should be inferrable by the compiler. The first and the\nthird argument are inferrable from the argument, `fs2Stream`, and the second argument should be\ninferrable from the left-hand side. But I haven't figured out how to get the compiler to infer any\nof them yet. Can you help? It's not as simple as you might think, because the implicit resolution\nhas to navigate between types with two type parameters, such as `Enumerator[Eval, Int]`, and types\nwith a single type parameter, such as `streamadapter.iterateeio.EvalEnumerator[Int]`.\n\nLet's in turn convert this iteratee.io enumerator into a Play enumerator:\n\n```scala\nval playEnumerator: play.api.libs.iteratee.Enumerator[Int] = {\n  import streamadapter._\n  import streamadapter.iterateeio._\n  import streamadapter.play._\n  import scala.concurrent.ExecutionContext.Implicits.global\n  adapt[EvalEnumerator, PlayEnumerator, Int](iterateeIoEnumerator)\n}\n```\n\nAll four enumerators above will produce the same elements, zero through nine, as you can see for\nyourself by running [Usage.scala](https://github.com/longevityframework/stream-adapter/blob/master/core/src/test/scala/streamadapter/usage/Usage.scala).\n\n## How it works\n\nThe `stream-adapter` provides a class `Chunkerator` that captures most of the functionality of a\nstream in a synchronous data structure. Streams are re-runnable, so we provide a `Function0` that\nproduces an iterator:\n\n```scala\ntrait Chunkerator[+A] extends Function0[CloseableChunkIter[A]] { // ...\n```\n\nThe `CloseableChunkIter` is an iterator over \"chunks\", which are used by various streaming libraries\nfor performance reasons:\n\n```scala\ntrait CloseableChunkIter[+A] extends CloseableIter[Seq[A]]\n```\n\nA `CloseableIter` is just an iterator that supports a `close` operation, so the consumer can\ncommunicate to the producer that it finished early, allowing the producer to free up resources:\n\n```scala\ntrait CloseableIter[+A] extends Iterator[A] {\n  def close: Unit\n}\n```\n\nFor each streaming library, we provide two adapters: one from a `Chunkerator` to a stream, and the\nother from a stream to a `Chunkerator`. We can then combine two adapters to produce an adapter\nbetween any two streaming libraries, using `Chunkerator` as a mediator. This way, we can have eight\nconverters - two for each of four streaming libraries - instead of the twelve we would need if we\nwere going to provide custom adapters for every pair of streaming libraries. And if we add another\nstreaming library to the mix, we only need to add two adapters, instead of eight.\n\nOne advantage to this approach is for people who are writing libraries that provide a streaming API,\nbut don't want to lock it down to a single streaming library. In this case, they can just produce a\nsingle `Chunkerator`, and use `stream-adapter` to produce streams from multiple libraries. In fact,\nthis is how I am using it. I only use the stream to `Chunkerator` converters for testing.\n\nOf course, this approach does not rule out the possibility of providing custom adapters that remove\nthe mediating step. We may have to juggle the implicits a bit to make this work, but it shouldn't be\na big deal.\n\n## Limitations\n\nThis early release is very raw, and there are a number of improvements that could be made. I've\ncreated GitHub issues to keep track of all the ideas I've come up with. Rather than repeating myself\nhere, I'll ask you to browse the issues yourself:\n\n- [https://github.com/longevityframework/stream-adapter/issues](https://github.com/longevityframework/stream-adapter/issues)\n\n## Usage\n\nWe provide artifacts for Scala 2.11 and 2.12. We don't have 2.10 artifacts because there is not a\nfull suite of 2.10 artifacts for the four streaming libraries.\n\n```scala\nlibraryDependencies += \"org.longevityframework\" %% \"streamadapter\" % \"0.1.0\"\n```\n\nAll the underlying streaming libraries are included here as optional dependencies, so you will need\nto bring in the libraries you want in your own project. Examples:\n\n```scala\nlibraryDependencies += \"com.typesafe.akka\" %% \"akka-stream\" % \"2.5.2\"\n\nlibraryDependencies += \"co.fs2\" %% \"fs2-core\" % \"0.9.6\"\n\nlibraryDependencies += \"org.typelevel\" %% \"cats\" % \"0.9.0\"\nlibraryDependencies += \"io.iteratee\" %% \"iteratee-core\" % \"0.12.0\"\n\nlibraryDependencies += \"com.typesafe.play\" %% \"play-iteratees\" % \"2.6.1\"\n```\n\n## License\n\nIt's [Apache 2](http://www.apache.org/licenses/). I don't really have any reasons to pick a\ndifferent license than this seemingly de-facto open source license. If you have some good reasons\nwhy this project should be released under a different license, please let me know.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flongevityframework%2Fstream-adapter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flongevityframework%2Fstream-adapter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flongevityframework%2Fstream-adapter/lists"}