{"id":20064246,"url":"https://github.com/mochidev/asyncsequencereader","last_synced_at":"2025-10-08T19:16:34.147Z","repository":{"id":46032228,"uuid":"429314292","full_name":"mochidev/AsyncSequenceReader","owner":"mochidev","description":"Building blocks to easily consume Swift's `AsyncSequence`.","archived":false,"fork":false,"pushed_at":"2024-10-08T13:36:42.000Z","size":38,"stargazers_count":11,"open_issues_count":4,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-09-15T11:27:13.988Z","etag":null,"topics":["async-sequence","serialization","swift","swift-concurrency","swift-package-manager","swift-serve","swift5-5"],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mochidev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-18T05:59:51.000Z","updated_at":"2025-03-13T12:05:37.000Z","dependencies_parsed_at":"2024-05-19T22:24:37.236Z","dependency_job_id":"89bded34-df88-4c60-898c-de25e62235fc","html_url":"https://github.com/mochidev/AsyncSequenceReader","commit_stats":{"total_commits":14,"total_committers":1,"mean_commits":14.0,"dds":0.0,"last_synced_commit":"5a882f1d294d90f9afbd00a885a02f7b0c16738d"},"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/mochidev/AsyncSequenceReader","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mochidev%2FAsyncSequenceReader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mochidev%2FAsyncSequenceReader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mochidev%2FAsyncSequenceReader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mochidev%2FAsyncSequenceReader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mochidev","download_url":"https://codeload.github.com/mochidev/AsyncSequenceReader/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mochidev%2FAsyncSequenceReader/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279000707,"owners_count":26082805,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async-sequence","serialization","swift","swift-concurrency","swift-package-manager","swift-serve","swift5-5"],"created_at":"2024-11-13T13:45:24.325Z","updated_at":"2025-10-08T19:16:34.119Z","avatar_url":"https://github.com/mochidev.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AsyncSequenceReader\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://swiftpackageindex.com/mochidev/AsyncSequenceReader\"\u003e\n\u003cimg src=\"https://img.shields.io/endpoint?url=https%3A%2F%2Fswiftpackageindex.com%2Fapi%2Fpackages%2Fmochidev%2FAsyncSequenceReader%2Fbadge%3Ftype%3Dswift-versions\" /\u003e\n\u003c/a\u003e\n\u003ca href=\"https://swiftpackageindex.com/mochidev/AsyncSequenceReader\"\u003e\n\u003cimg src=\"https://img.shields.io/endpoint?url=https%3A%2F%2Fswiftpackageindex.com%2Fapi%2Fpackages%2Fmochidev%2FAsyncSequenceReader%2Fbadge%3Ftype%3Dplatforms\" /\u003e\n\u003c/a\u003e\n\u003ca href=\"https://github.com/mochidev/AsyncSequenceReader/actions?query=workflow%3A%22Test+AsyncSequenceReader%22\"\u003e\n\u003cimg src=\"https://github.com/mochidev/AsyncSequenceReader/workflows/Test%20AsyncSequenceReader/badge.svg\" alt=\"Test Status\" /\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\n`AsyncSequenceReader` provides building blocks to easily consume Swift's `AsyncSequence`.\n\n## Quick Links\n\n- [Documentation](https://swiftpackageindex.com/mochidev/AsyncSequenceReader/documentation)\n\n## Installation\n\nAdd `AsyncSequenceReader` as a dependency in your `Package.swift` file to start using it. Then, add `import AsyncSequenceReader` to any file you wish to use the library in.\n\nPlease check the [releases](https://github.com/mochidev/AsyncSequenceReader/releases) for recommended versions.\n\n```swift\ndependencies: [\n    .package(url: \"https://github.com/mochidev/AsyncSequenceReader.git\", .upToNextMinor(from: \"0.3.1\")),\n],\n...\ntargets: [\n    .target(\n        name: \"MyPackage\",\n        dependencies: [\n            \"AsyncSequenceReader\",\n        ]\n    )\n]\n```\n\n## What is `AsyncSequenceReader`?\n\n`AsyncSequenceReader` is a collection of building blocks to make it easy to read information and transform `AsyncSequence` into data types your app understands.\n\nAlthough an `AsyncSequence` can be consumed via a `for await` loop, that isn't often the easiest way of consuming that data:\n\n```swift\n\nfor await byte in url.resourceBytes {\n    // Buffer enough bytes to read an int\n    // Buffer the amount of bytes specified by the int to read a frame\n    // Repeat until all frames are consumed...\n}\n\n```\n\nIf the serialization format is more complicated than that, it can be significantly harder to write easy to read and understandable code that can be easily maintained.\n\n`AsyncSequenceReader` provides 3 primary tools to help you with this: Iterator Maps, Counted Collections, and Terminated Collections.\n\n### Iterator Maps\n\nThe most basic building block this package provides is called an **Iterator Map**. Iterator maps allow you a way of reading a sequence a value at a time without worrying about buffering or state management. Additionally, they allow you to return complete objects as you build them, letting other parts of your app consume those objects as they become available!\n\nLet's build an iterator map:\n\n```swift\nstruct DataFrame {\n    var command: String\n    var payload: [UInt8]\n}\n\nlet url = ...\nlet sequence = url.resourceBytes\n\nlet results = sequence.iteratorMap { iterator -\u003e DataFrame? in\n    /// Reads go here\n}\n\n// Do something with the results:\nfor await dataFrame in results {\n    print(dataFrame)\n}\n```\n\nWithin the closure, you can do one of three things:\n- Read values and return an object (In this case a DataFrame),\n- Throw an error, cancelling the whole process,\n- Return `nil`, indicating the end of the sequence.\n\nReading values is as easy as calling `let value = try await iterator.next()`. This value will match the type of the sequence, which is `UInt8` in the above example. If the value is nil, you've reached the end of the sequence. We'll take a look at other ways to read values momentarily.\n\nNote: Resist the urge to catch errors within an iterator map, as once a value is read, it will no longer be available.\n\nReturning an object will make it available to whoever is consuming the resulting sequence, preparing your closure to be called again for the next object. Do note that your closure will not be called unless something consumes your `results` sequence, either via `for await`, or by using `.reduce` or other `AsyncSequence` methods.\n\nNote: Do not copy the iterator to other methods without marking it as `inout`, since as a value type, a copy will be made, and further reads may become out of sync.\n\n### Counted Collections\n\nReading values in an iterator map one at a time is useful, but often times we need to buffer larger amounts of data. There are several ways we can do that:\n\n```swift\nvar fourByteSequence = try await iterator.collect(4) // [UInt8, UInt8, UInt8, UInt8]?\nvar largeSequence = try await iterator.collect(max: 256) // Array of [UInt8]? with a max size of 256, but may be shorter if the sequence had less than 256 characters available.\nvar limitedSequence = try await iterator.collect(min: 128, max: 256) // Array of [UInt8]? that will throw if less than 128 bytes are available, but will be no larger than 256.\n```\n\nFor that last example, do note that the `limitedSequence` will only become available if and when all the bytes have been read. ie. you will not get results back if only 128 bytes are available _right now_, if the sequence is still ongoing.\n\nIf the minimum number of bytes cannot be collected, an `AsyncSequenceReaderError.insufficientElements` error will be thrown.\n\nYou can also collect elements into another async sequence using a **sequence transform**:\n\n```swift\nvar veryLargeSequence = try await iterator.collect(1024*1024*1024) { sequence -\u003e Summary in\n    let results = sequence.iteratorMap { iterator -\u003e DataFrame? in\n        guard let values = try await iterator.collect(count: 1024*1024) else { return nil }\n        \n        return DataFrame(values)\n    }\n    \n    let averages = try await results.reduce(into: []) { $0.append($1.average) }\n    return Summary(averages)\n}\n```\n\nIn the above example, our sequence transform gives us access to a sequence that will be at most `1024*1024*1024` bytes large, which is 1 GB! However, instead of accumulating that data into an array, we get a sequence back, which we can attach an iterator map to so we can process the data 1 MB at a time, combining that data into a `DataFrame` type. Then, we can consume this transformed sequence, reducing it to calculate averages for each data frame, and storing those averages in a `Summary` object.\n\nNote that this whole time, no more than around 1 MB of memory will be used at a time, because it'll only actually be consumes while reducing the results, which will only read 1 MB of data at a time, and will stop once a total of 1 GB of data has been read.\n\n### Terminated Collections\n\nTerminated collections actually work just like counted collections, but they read until a certain element (or sequence of elements) is encountered:\n\n```swift\nvar nullTerminatedString = try await iterator.collect(upToIncluding: 0, throwsIfOver: 1024) // [UInt8]?, ending in `\\0`\nvar httpHeaderEntry = try await iterator.collect(upToExcluding: [\"\\r\".asciiValue, \"\\n\".asciiValue], throwsIfOver: 1024) // [UInt8]?, without the `\\r\\n`\n```\n\nThis is especially useful when scanning for strings or other known boundaries, allowing you get get an array of elements either including or excluding the terminator you specified.\n\nNote how a `throwsIfOver` parameter is necessary — this is to prevent un-bounded reads from running out of control. If the terminator is not detected, or your maximum element allowance has been reached, an `AsyncSequenceReaderError.terminationNotFound` error will be thrown.\n\nYou can bypass the `throwsIfOver` parameter if you use a **sequence transform** instead, which may be a better option if your algorithm deals with large amounts of data. If you stop reading early, elements can still be read by subsequent requests, giving you more control over how to read your data.\n\nAlso note that if you use a **sequence transform**, you can only collect a sequence up to and including your terminator, and no error will be thrown if your terminator was never encountered, since you can easily check `result.suffix(termination.count) == termination` to verify this yourself, allowing you the possibility of handling different data lengths yourself.\n\n### Integration with Bytes\n\n`AsyncSequenceReader` really shines when you combine it with [Bytes](https://github.com/mochidev/Bytes), another package specialized in dealing with and transforming byte sequences. For instance, if you wanted to decode data frames that consist of a four byte payload size, a null terminated header string, and a payload, you could do so easily like this:\n\n```swift\nstruct DataFrame {\n    var command: String\n    var payload: [UInt8]\n}\n\nlet url = ...\nlet sequence = url.resourceBytes\n\nlet results = sequence.iteratorMap { iterator -\u003e DataFrame? in\n    guard let payloadCountBytes = try await iterator.count(4) else { throw DataFrameError.missingPayloadSize }\n    var payloadSize = try UInt32(bigEndianBytes: payloadCountBytes)\n    \n    guard let commandBytes = try await iterator.count(upToExcluding: 0, throwsIfOver: min(256, payloadSize)) else { throw DataFrameError.missingCommand }\n    let commandString = String(utf8Bytes: commandBytes)\n    payloadSize -= commandBytes.count - 1 // Don't forget the null byte we skipped\n    \n    guard let payloadBytes = try await iterator.count(payloadSize) else { throw DataFrameError.missingPayload }\n    \n    return DataFrame(command: commandString, payload: payloadBytes)\n}\n\n// Do something with the results:\nfor await dataFrame in results {\n    print(dataFrame)\n}\n```\n\nBetter yet, [Bytes](https://github.com/mochidev/Bytes) also has support for reading from `AsyncIteratorProtocol` directly, allowing you to simplify the above to:\n\n```swift\nstruct DataFrame {\n    var command: String\n    var payload: [UInt8]\n}\n\nlet url = ...\nlet sequence = url.resourceBytes\n\nlet results = sequence.iteratorMap { iterator -\u003e DataFrame? in\n    var payloadSize = try await iterator.next(bigEndian: UInt32.self)\n    \n    let commandString = try await iterator.next(utf8: String.self, upToExcluding: 0, throwsIfOver: min(256, payloadSize))\n    payloadSize -= commandString.utf8.count - 1 // Don't forget the null byte we skipped\n    \n    let payloadBytes = try await iterator.next(bytes: Bytes.self, count: payloadSize)\n    \n    return DataFrame(command: commandString, payload: payloadBytes)\n}\n\n// Do something with the results:\nfor await dataFrame in results {\n    print(dataFrame)\n}\n```\n\n### More\n\nFor more examples, please take a look at the unit tests provided in this package. If a good example isn't listed, please consider submitting a PR to show how it's done!\n\n## Contributing\n\nContribution is welcome! Please take a look at the issues already available, or start a new discussion to propose a new feature. Although guarantees can't be made regarding feature requests, PRs that fit within the goals of the project and that have been discussed beforehand are more than welcome!\n\nPlease make sure that all submissions have clean commit histories, are well documented, and thoroughly tested. **Please rebase your PR** before submission rather than merge in `main`. Linear histories are required, so merge commits in PRs will not be accepted.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmochidev%2Fasyncsequencereader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmochidev%2Fasyncsequencereader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmochidev%2Fasyncsequencereader/lists"}