{"id":13871897,"url":"https://github.com/adam-fowler/compress-nio","last_synced_at":"2025-03-17T11:30:39.312Z","repository":{"id":63901419,"uuid":"258282710","full_name":"adam-fowler/compress-nio","owner":"adam-fowler","description":"Compression/Decompression support for Swift NIO ByteBuffer","archived":false,"fork":false,"pushed_at":"2024-10-21T15:20:18.000Z","size":145,"stargazers_count":22,"open_issues_count":1,"forks_count":4,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-10-22T11:53:04.100Z","etag":null,"topics":["bytebuffer","compression","decompression","lz4","swift","swift-nio","zlib"],"latest_commit_sha":null,"homepage":null,"language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adam-fowler.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"adam-fowler"}},"created_at":"2020-04-23T17:37:25.000Z","updated_at":"2024-05-04T15:24:15.000Z","dependencies_parsed_at":"2024-10-25T12:30:16.322Z","dependency_job_id":"f5e9a689-6a6f-4ff5-a05d-cc02e77dd087","html_url":"https://github.com/adam-fowler/compress-nio","commit_stats":{"total_commits":53,"total_committers":1,"mean_commits":53.0,"dds":0.0,"last_synced_commit":"e1dd251dbc551051fe86494386b9b48c3d0dc23c"},"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adam-fowler%2Fcompress-nio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adam-fowler%2Fcompress-nio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adam-fowler%2Fcompress-nio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adam-fowler%2Fcompress-nio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adam-fowler","download_url":"https://codeload.github.com/adam-fowler/compress-nio/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243858056,"owners_count":20359271,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bytebuffer","compression","decompression","lz4","swift","swift-nio","zlib"],"created_at":"2024-08-05T23:00:29.792Z","updated_at":"2025-03-17T11:30:38.880Z","avatar_url":"https://github.com/adam-fowler.png","language":"Swift","funding_links":["https://github.com/sponsors/adam-fowler"],"categories":["Swift"],"sub_categories":[],"readme":"# Compress NIO\n\nA compression library for Swift NIO ByteBuffers.\n\n# Compress and Decompress\nCompress NIO contains a number of methods for compressing and decompressing `ByteBuffers`. A simple usage would be \n```swift\nvar compressedBuffer = buffer.compress(with: .gzip)\nvar uncompressedBuffer = buffer.decompress(with: .gzip)\n```\nThese methods allocate a new `ByteBuffer` for you. The `decompress` method can allocate multiple `ByteBuffers` while it is uncompressing depending on how well compressed the original `ByteBuffer` is. It is preferable to know in advance the size of buffer you need and allocate it yourself just the once and use the following functions.\n```swift\nlet uncompressedSize = buffer.readableBytes\nvar compressedBuffer = ByteBufferAllocator().buffer(capacity: knownCompressedSize)\ntry buffer.compress(to: \u0026compressedBuffer, with: .deflate)\nvar uncompressedBuffer = ByteBufferAllocator().buffer(capacity: uncompressedSize)\ntry compressedBuffer.decompress(to: \u0026uncompressedBuffer, with: .deflate)\n```\nThis returns the maximum size of buffer required to write out compressed data for the `deflate` compression algorithm.\n\nIf you provide a buffer that is too small a `CompressNIO.bufferOverflow` error is thrown. You will need to provide a larger `ByteBuffer` to complete your operation.\n\n# Streaming\nThere are situations where you might want to or are required to compress/decompress a block of data in smaller slices. If you have a large file you want to compress it is probably best to load it in smaller slices instead of loading it all into memory in one go. If you are receiving a block of compressed data via HTTP you cannot guarantee it will be delivered in one slice. Swift NIO Compress provides a streaming api to support these situations. \n\n## Compressing \n\nThere are three methods for doing stream compressing: window, allocating and raw. All of them start with calling `compressor.startStream` and end with calling `compressor.finishStream`. \n\n#### Window method\nFor the window method you provide a working buffer for the compressor to use. When you call `compressStream` it compresses into this buffer and when the buffer is full it will call a `process` closure you have provided.\n```swift\nlet compressor = ZlibCompressor(algorithm: .gzip)\nvar window = ByteBufferAllocator().buffer(capacity: 64*1024)\nwhile var buffer = getData() {\n    try buffer.compressStream(with: compressor, window: window, flush: .finish) { buffer in\n        // process your compressed data\n    }\n}\ntry compressor.reset()\n```\n#### Allocation method\nWith the allocating method you leave the compressor to allocate the ByteBuffers for output data. It will calculate the maximum possible size the compressed data could be and allocates that amount of space for each compressed data block. The last compressed block needs to have the `flush` parameter set to `.finish`\n```swift\nlet compressor = ZlibCompressor(algorithm: .gzip)\nwhile var buffer = getData() {\n    let flush: CompressNIOFlush = isThisTheFinalBlock ? .finish : .sync\n    let compressedBuffer = try buffer.compressStream(with: compressor, flush: flush, allocator: ByteBufferAllocator())\n}\ntry compressor.reset()\n```\nIf you don't know what is your final data block you can always compress an empty `ByteBuffer` with the `flush` set to `.finish` to get your final block. Also note that the flush parameter is set to `.sync` in the loop. This is required otherwise the next `compressStream` cannot successfully estimate its buffer size as there might be buffered data still waiting to be output.\n\n#### Raw method\n\nWith this mehod you call the lowest level function and deal with `.bufferOverflow` errors thrown whenever you run out of space in your output buffer. You will need a loop for receiving data and then you will need an inner loop for compressing that data. You call the `compress` until you have no more data to compress. Everytime you receive a `.bufferOverflow` error you have to provide a new output data. Once you have read all the input data you do the same again but with the `flush` parameter set to `.finish`.\n\n## Decompressing\n\nThe same three methods window, allocation, raw are available for decompressing streamed data but you don't need to set a `flush` parameter to `.finish` while decompressing which makes everything a little easier. \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadam-fowler%2Fcompress-nio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadam-fowler%2Fcompress-nio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadam-fowler%2Fcompress-nio/lists"}