https://github.com/odis-labs/streaming

Fast, safe and composable streaming abstractions.
https://github.com/odis-labs/streaming

iterators ocaml reasonml streaming transducers

Last synced: 2 months ago
JSON representation

Fast, safe and composable streaming abstractions.

• Host: GitHub
• URL: https://github.com/odis-labs/streaming
• Owner: odis-labs
• License: isc
• Created: 2019-12-10T12:17:53.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2022-08-07T14:02:38.000Z (over 2 years ago)
• Last Synced: 2024-08-04T01:28:41.036Z (6 months ago)
• Topics: iterators, ocaml, reasonml, streaming, transducers
• Language: OCaml
• Homepage: https://odis-labs.github.io/streaming
• Size: 229 KB
• Stars: 108
• Watchers: 4
• Forks: 8
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGES.md
  • License: LICENSE.md

Awesome Lists containing this project

• awesome-list - streaming - labs | 90 | (OCaml)

README

        
# Streaming

Streaming abstractions that combine, transform and reduce large amounts of

sequential data efficiently, in constant space and without leaking resources.

## Overview

_Streaming_ uses composable stream producers (sources), consumers (sinks) and

transformers (flows). The central model that abstracts over them is a `Stream`.

The following features are provided:

- **Constant memory usage**: large or infinite streams can be computed in constant

  and small space. Buffering of the input is possible when needed.

- **Excellent performance**: all models were designed with performance at the

  core. See [benchmarks](https://github.com/rizo/streams-bench) for detailed

  comparison with other libraries.

- **Resource safety**: resources in effectful streaming pipelines are allocated

  lazily and released as early as possible. Resources are guaranteed to

  be terminated even when streams rise exceptions.

- **Flexibility**: both push-based and pull-based models are implemented to

  allow efficient zipping, concatenation and other streaming operations.

- **Streaming notation**: build streams and sinks using a convenient

  comprehension and applicative notations.

Read the [library documenation](https://odis-labs.github.io/streaming) for more details.

## Acknowledgements

This library is based on ideas found in other libraries and research projects

such as: Haskell's [Pipes](https://github.com/Gabriel439/Haskell-Pipes-Library)

and [Foldl](https://github.com/Gabriel439/Haskell-Foldl-Library) libraries,

Scala's [ZIO Streams](https://zio.dev), Clojure's

[Transducers](https://clojure.org/reference/transducers) and the

[Iteratees](http://okmij.org/ftp/Streams.html) streaming model by Oleg

Kiselyov.