{"id":15293521,"url":"https://github.com/jkolb/lilliput","last_synced_at":"2025-04-13T05:11:40.573Z","repository":{"id":18556585,"uuid":"21758329","full_name":"jkolb/Lilliput","owner":"jkolb","description":"Low overhead byte encoding/decoding for Swift","archived":false,"fork":false,"pushed_at":"2024-05-29T01:31:52.000Z","size":182,"stargazers_count":42,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-05-29T19:51:57.297Z","etag":null,"topics":["big-endian","buffer-memory","byte-order","endianness","file-format","lilliput","little-endian","swift"],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jkolb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-07-12T03:09:00.000Z","updated_at":"2024-06-22T20:54:58.556Z","dependencies_parsed_at":"2024-06-22T20:54:57.858Z","dependency_job_id":"ab48b90e-ca37-45f7-9855-3305e67c68d1","html_url":"https://github.com/jkolb/Lilliput","commit_stats":{"total_commits":77,"total_committers":3,"mean_commits":"25.666666666666668","dds":0.07792207792207795,"last_synced_commit":"227c8015491ac9a870463cff02aba8f7111d601b"},"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jkolb%2FLilliput","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jkolb%2FLilliput/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jkolb%2FLilliput/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jkolb%2FLilliput/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jkolb","download_url":"https://codeload.github.com/jkolb/Lilliput/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248665745,"owners_count":21142123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-endian","buffer-memory","byte-order","endianness","file-format","lilliput","little-endian","swift"],"created_at":"2024-09-30T16:49:51.503Z","updated_at":"2025-04-13T05:11:40.546Z","avatar_url":"https://github.com/jkolb.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lilliput\n\n[Lilliput](http://en.wikipedia.org/wiki/Lilliput_and_Blefuscu) is a native Swift framework for working with binary data.\n\n## Design decisions\n\n* Does not support streaming data, it only works with data that can fit in memory.\n* Allows types to have multiple encodings/decodings, this means that a UInt32 could be easily represented as little endian, big endian, or some custom compressed encoding.\n* Does not support the idea of \"native\" endianness, when working with data where endianess matters you must always be specific. \n* Makes use of the [swift-system](https://github.com/apple/swift-system) package for reading/writing files.\n\n## Basic usage\n\n```swift\nimport Lilliput\n\n// Allocate enough storage for a 32-bit unsigned integer\nlet buffer = ByteBuffer(count: 4)\ntry buffer.slice(...) { bytes in\n    // Write a UInt32 to that storage in little endian encoding\n    var writer = ByteSpanWriter(bytes)\n    try writer.write(UInt32(100), as: UInt32.LittleEndian.self)\n\n    // Read a UInt32 from that stroage in little endian encoding \n    var reader = ByteSpanReader(bytes)\n    let value = try reader.read(UInt32.LittleEndian.self)\n}\n```\n\n## Custom byte decodable/encodable types\n\n```swift\nstruct MyType {\n    var value0: Int\n    var value1: Int8\n    var value3: Double\n}\n\nextension MyType: ByteDecoder {\n    static func decode\u003cR: Reader\u003e(from reader: inout R) throws -\u003e MyType {\n        return MyType(\n            value0: Int(try reader.read(UInt32.LittleEndian.self)),\n            value1: try reader.read(Int8.self),\n            value2: try reader.read(Float64.LittleEndian.self)\n        )\n    }\n}\n\nextension MyType: ByteEncoder {\n    static func encode\u003cW: Writer\u003e(_ value: MyType, to writer: inout W) throws {\n        try writer.write(UInt32(value.value0), as: UInt32.LittleEndian.self)\n        try writer.write(value.value1)\n        try writer.write(value.value2, as: Float64.LittleEndian.self)\n    }\n}\n```\n\n## Mutiple ways to decode/encode the same type\n\n```swift\nextension MyType {\n    @frozen enum Custom {}\n}\n\nextension MyType.Custom: ByteDecoder {\n    static func decode\u003cR: Reader\u003e(from reader: inout R) throws -\u003e MyType {\n        // Values are loaded/stored in reverse order from the type definition and using big endian\n        let value2 = try reader.read(Float64.BigEndian.self)\n        let value1 = try reader.read(Int8.self)\n        try reader.alignTo(4) // Make sure next read occurs after three padding bytes\n        let value0 = Int(try reader.read(UInt32.BigEndian.self))\n         \n        return MyType(\n            value0: value0,\n            value1: value1,\n            value2: value2\n        )\n    }\n}\n\nextension MyType.Custom: ByteEncoder {\n    static func encode\u003cW: Writer\u003e(_ value: MyType, to writer: inout W) throws {\n        // Values are loaded/stored in reverse order from the type definition and using big endian\n        try writer.write(value.value2, as: Float64.BigEndian.self)\n        try writer.write(value.value1)\n        try writer.write((0, 0, 0), as: UInt8.Tuple3.self) // Add three padding bytes\n        try writer.write(UInt32(value.value0), as: UInt32.BigEndian.self)\n    }\n}\n```\n\n## Reading types from disk\n\n```swift\nimport Lilliput\nimport SystemPackage\n\nlet file = try FileDescriptor.open(path, .readOnly)\nlet buffer = ByteBuffer(count: 13)\nlet myType = try buffer.slice(...) { bytes in\n    try file.readAll(into: bytes)\n    var reader = ByteSpanReader(bytes)\n    return try reader.read(MyType.self)\n}\n```\n\nIf the data was stored on disk using the Custom encoding instead\n```swift\nreturn try reader.read(MyType.Custom.self)\n```\n\n## Writing types to disk\n```swift\nimport Lilliput\nimport SystemPackage\n\nlet myType = MyType(value0: 100, value1: 8, value2: 300.56)\nlet buffer = ByteBuffer(count: 13)\ntry buffer.slice(...) { bytes in\n    var writer = SpanByteWriter(buffer)\n    try writer.write(myType)\n}\nlet file = try FileDescriptor.open(path, .writeOnly)\nfile.writeAll(buffer)\n```\n\nIf we wanted to store the data using the Custom encoding instead\n```swift\ntry writer.write(myType, as: MyType.Custom.self)\n```\n\n## Reading arrays of decodable types\n\n```swift\nlet count = Int(try reader.read(UInt32.LittleEndian.self))\nlet arrayOfMyType = try reader.read(Element\u003cMyType\u003e.self, count: count)\n```\n\n## Writing arrays of encodable types\n\n```swift\ntry writer.write(UInt32(arrayOfMyType.count), as: UInt32.LittleEndian.self)\ntry writer.write(arrayOfMyType, as: Element\u003cMyType\u003e.self)\n```\n\n## Custom array decoding/encoding\n\nNote: This is not built in to the library due to decisions on how to encode the count value could differ wildly across use cases.\n```swift\n@frozen struct MyArray\u003cE\u003e {}\n\nextension MyArray: ByteDecoder where E: ByteDecoder {\n    static func decode\u003cR: Reader\u003e(from reader: inout R) throws -\u003e [E.Decodable] {\n        let count = Int(try reader.read(UInt32.LittleEndian.self))\n        return try reader.read(Element\u003cE\u003e.self, count: count)\n    }\n}\n\nextension MyArray: ByteEncoder where E: ByteEncoder {\n    static func encode\u003cW: ByteWriter\u003e(_ value: [E.Encodable], to writer: inout W) throws {\n        try writer.write(UInt32(value.count), as: UInt32.LittleEndian.self)\n        \n        for element in value {\n            try writer.write(element, as: E.self)\n        }\n    }\n}\n```\n\nUsage (note that the count value is handled without extra work now)\n```swift\nlet arrayOfMyType = try reader.read(MyArray\u003cMyType\u003e.self)\n\ntry writer.write(arrayOfMyType, as: MyArray\u003cMyType\u003e.self)\n```\n\n## Foundation Integration\n```swift\nlet data = Data(...) // Get data somehow\nvar reader = DataReader(data: data, maxReadCount: 8)\nlet myType = try reader.read(MyType.self)\n```\n\nChoose a value for `maxReadCount` that represents the maximum amount of bytes read in one call to `read`. In the example above the longest read is a `Double` which is 8 bytes. In the worst case you can omit the `maxReadCount` parameter and then the entire length of the Data would be used as the default.\n\n## Small Gotcha In Version 12.0.0+ API\n\nAll of the `read` methods on `ByteReader` and `write` methods on `ByteWriter` that do not involve a `ByteDecoder`/`ByteEncoder` no longer check to see if there are enough bytes to complete the operation. This is now handled separately be calling `ensure`. For example `try reader.ensure(5)` will throw if there are not enough bytes left to read.\n\nA side effect of this change is that when you write a single `UInt8` it no longer triggers the `UInt8` implementation of `ByteEncoder`.\n\nPreviously you would do this and it would throw if there was not enough space to write:\n```swift\ntry writer.write(UInt8(7))\n```\n\nNow the new `write` method on `ByteWriter` that takes a single `UInt8` superceeds this.\nTo get the new behavior do this:\n\n```swift\ntry ensure(1)\nwriter.write(UInt8(7))\n```\n\nor do this which triggers the `ByteEncoder` implementation manually:\n\n```swift\ntry writer.write(UInt8(7), as: UInt8.self)  \n```\n\n## Adding `Lilliput` as a Dependency\n\nTo use the `Lilliput` library in a SwiftPM project, add the following line to the dependencies in your `Package.swift` file:\n\n```swift\n.package(url: \"https://github.com/jkolb/Lilliput.git\", from: \"12.0.0\"),\n```\n\nFinally, include `\"Lilliput\"` as a dependency for your target:\n\n```swift\nlet package = Package(\n    // name, platforms, products, etc.\n    dependencies: [\n        .package(url: \"https://github.com/jkolb/Lilliput.git\", from: \"12.0.0\"),\n        // other dependencies\n    ],\n    targets: [\n        .target(name: \"MyTarget\", dependencies: [\n            .product(name: \"Lilliput\", package: \"Lilliput\"),\n        ]),\n        // other targets\n    ]\n)\n```\n\n## License\n\nLilliput is available under the MIT license. See the [LICENSE](LICENSE) file for more info.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjkolb%2Flilliput","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjkolb%2Flilliput","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjkolb%2Flilliput/lists"}