https://github.com/cgarciae/stream

A simple data processing library for Swift
https://github.com/cgarciae/stream

Last synced: 2 months ago
JSON representation

A simple data processing library for Swift

• Host: GitHub
• URL: https://github.com/cgarciae/stream
• Owner: cgarciae
• License: mit
• Created: 2019-07-14T17:50:27.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2019-07-23T06:08:02.000Z (over 5 years ago)
• Last Synced: 2024-10-15T15:35:54.929Z (3 months ago)
• Language: Swift
• Homepage:
• Size: 83 KB
• Stars: 28
• Watchers: 3
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Stream

Stream is a Swift library that enables you to create scalable data pipelines for medium or large datasets.

## Features

Stream pipelines allow you to process large or even infinite collections efficiently by:

* Performing computation in parallel within each Stream.

* Running each Stream concurrently within a pipeline.

* Providing back-pressure mechanisms to control memory growth.

## Installation

You can install it via SwiftPM via:

```swift

.package(url: "https://github.com/cgarciae/Stream", from: "0.0.7")

```

It might work on other compatible package managers.

## Example

Any `Sequence` can be converted into a `Stream` via the `.stream` property, after that you can use its custom functional methods like `map`, `filter`, etc, to process the data in parallel / concurrently:

```swift

import Stream

_ = getURLs()

    .stream

    .map {

        downloadImage($0)

    }

    .filter {

        validateImage($0)

    }

    .flatMap {

        getMultipleImageSizes($0)

    }

    .forEach {

        storeImage($0)

    }

```

`Stream` inherits from `LazySequence` so you can treat it like a normal Sequence for other purposes. By default the results of each stream may come in any order which has better performance, but if you do want to preserve order you can turn a `Stream` into an `OrderedStream` via the `.inOrder` property. 

```swift

import Stream

_ = getURLs()

    .stream

    .inOrder

    .map {

        downloadImage($0)

    }

    .filter {

        validateImage($0)

    }

    .flatMap {

        getMultipleImageSizes($0)

    }

```

#### Back-pressure

To manage resources you can use the `maxTasks` and `queueMax` parameters: 

```swift

import Stream

_ = getURLs()

    .stream

    .map(maxTasks: 4, queueMax: 10) {

        downloadImage($0)

    }

    .filter(maxTasks: 2, queueMax: 15) {

        validateImage($0)

    }

    .flatMap(maxTasks: 5, queueMax: 25) {

        getMultipleImageSizes($0)

    }

    .forEach(maxTasks: 3,queueMax: 20) {

        storeImage($0)

    }

```

`maxTasks` will control the number of GCD Tasks created by the Stream, and `queueMax` will limit maximum amount of elements allowed to live in the output queue simultaneously. If the output queue is full tasks will eventually block and the Stream will halt until its consumer requests more elements.

## Architecture

![](docs/Stream.png)

## Members

* `map`

* `flatMap`

* `filter`

* `forEach`

## Meta

Cristian Garcia – [email protected]

Distributed under the MIT license. See LICENSE for more information.