{"id":18676244,"url":"https://github.com/cgarciae/stream","last_synced_at":"2025-04-12T02:12:51.412Z","repository":{"id":63906865,"uuid":"196866949","full_name":"cgarciae/Stream","owner":"cgarciae","description":"A simple data processing library for Swift","archived":false,"fork":false,"pushed_at":"2019-07-23T06:08:02.000Z","size":85,"stargazers_count":28,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-25T21:51:19.197Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cgarciae.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-14T17:50:27.000Z","updated_at":"2023-06-03T12:58:55.000Z","dependencies_parsed_at":"2022-11-28T22:48:30.034Z","dependency_job_id":null,"html_url":"https://github.com/cgarciae/Stream","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cgarciae%2FStream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cgarciae%2FStream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cgarciae%2FStream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cgarciae%2FStream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cgarciae","download_url":"https://codeload.github.com/cgarciae/Stream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248317348,"owners_count":21083519,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T09:28:25.330Z","updated_at":"2025-04-12T02:12:51.387Z","avatar_url":"https://github.com/cgarciae.png","language":"Swift","readme":"# Stream\nStream is a Swift library that enables you to create scalable data pipelines for medium or large datasets.\n\n## Features\nStream pipelines allow you to process large or even infinite collections efficiently by:\n* Performing computation in parallel within each Stream.\n* Running each Stream concurrently within a pipeline.\n* Providing back-pressure mechanisms to control memory growth.\n\n## Installation\nYou can install it via SwiftPM via:\n```swift\n.package(url: \"https://github.com/cgarciae/Stream\", from: \"0.0.7\")\n```\nIt might work on other compatible package managers.\n\n## Example\nAny `Sequence` can be converted into a `Stream` via the `.stream` property, after that you can use its custom functional methods like `map`, `filter`, etc, to process the data in parallel / concurrently:\n```swift\nimport Stream\n\n_ = getURLs()\n    .stream\n    .map {\n        downloadImage($0)\n    }\n    .filter {\n        validateImage($0)\n    }\n    .flatMap {\n        getMultipleImageSizes($0)\n    }\n    .forEach {\n        storeImage($0)\n    }\n```\n`Stream` inherits from `LazySequence` so you can treat it like a normal Sequence for other purposes. By default the results of each stream may come in any order which has better performance, but if you do want to preserve order you can turn a `Stream` into an `OrderedStream` via the `.inOrder` property. \n\n```swift\nimport Stream\n\n_ = getURLs()\n    .stream\n    .inOrder\n    .map {\n        downloadImage($0)\n    }\n    .filter {\n        validateImage($0)\n    }\n    .flatMap {\n        getMultipleImageSizes($0)\n    }\n```\n\n#### Back-pressure\nTo manage resources you can use the `maxTasks` and `queueMax` parameters: \n```swift\nimport Stream\n\n_ = getURLs()\n    .stream\n    .map(maxTasks: 4, queueMax: 10) {\n        downloadImage($0)\n    }\n    .filter(maxTasks: 2, queueMax: 15) {\n        validateImage($0)\n    }\n    .flatMap(maxTasks: 5, queueMax: 25) {\n        getMultipleImageSizes($0)\n    }\n    .forEach(maxTasks: 3,queueMax: 20) {\n        storeImage($0)\n    }\n```\n`maxTasks` will control the number of GCD Tasks created by the Stream, and `queueMax` will limit maximum amount of elements allowed to live in the output queue simultaneously. If the output queue is full tasks will eventually block and the Stream will halt until its consumer requests more elements.\n\n## Architecture\n![](docs/Stream.png)\n## Members\n* `map`\n* `flatMap`\n* `filter`\n* `forEach`\n\n\n## Meta\nCristian Garcia – cgarcia.e88@gmail.com\n\nDistributed under the MIT license. See LICENSE for more information.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcgarciae%2Fstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcgarciae%2Fstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcgarciae%2Fstream/lists"}