https://github.com/mlhpdx/seekable-s3-stream
Code library that uses S3's API to provide an efficient random-access (seekable) Stream implementation for use in code where efficient network I/O is paramount.
https://github.com/mlhpdx/seekable-s3-stream
dotnet dotnet-core http-range random-access s3 stream
Last synced: 2 months ago
JSON representation
Code library that uses S3's API to provide an efficient random-access (seekable) Stream implementation for use in code where efficient network I/O is paramount.
- Host: GitHub
- URL: https://github.com/mlhpdx/seekable-s3-stream
- Owner: mlhpdx
- License: mit
- Created: 2020-11-10T22:31:33.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2025-12-19T01:08:37.000Z (3 months ago)
- Last Synced: 2025-12-21T23:37:11.365Z (3 months ago)
- Topics: dotnet, dotnet-core, http-range, random-access, s3, stream
- Language: C#
- Homepage:
- Size: 36.1 KB
- Stars: 38
- Watchers: 1
- Forks: 9
- Open Issues: 0
-
Metadata Files:
- Readme: Readme.md
- License: LICENSE
Awesome Lists containing this project
README
# Seekable S3 Stream
Some S3 objects are big enough that working on them in memory isn't desirable, or even possible. This library performs selective, efficient data transfer from S3 that is orders of magnitude faster and more efficient than naiively using `MemoryStream` while maintaining compatibility with libraries and packages that work with a `Stream` interface. Examples for reading ISO, Zip, JPG and Parquet files are included in the repo.
For the full explaination, check out the article on [Medium](https://medium.com/circuitpeople/random-access-seekable-streams-for-amazon-s3-in-c-bd2414255dcd).