https://github.com/mlhpdx/s3-upload-stream
Code library that provides a Stream implementation that makes working with uploads to S3 easier where the size of the content isn't known a priori. It holds only partial content in memory (works with large objects), and is compatible with code libraries that work with output streams.
https://github.com/mlhpdx/s3-upload-stream
content-length dotnet-core multi-part s3 streams upload
Last synced: 5 months ago
JSON representation
Code library that provides a Stream implementation that makes working with uploads to S3 easier where the size of the content isn't known a priori. It holds only partial content in memory (works with large objects), and is compatible with code libraries that work with output streams.
- Host: GitHub
- URL: https://github.com/mlhpdx/s3-upload-stream
- Owner: mlhpdx
- License: mit
- Created: 2021-01-13T22:04:56.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2025-12-21T02:44:38.000Z (6 months ago)
- Last Synced: 2025-12-22T21:51:03.397Z (6 months ago)
- Topics: content-length, dotnet-core, multi-part, s3, streams, upload
- Language: C#
- Homepage:
- Size: 42 KB
- Stars: 32
- Watchers: 2
- Forks: 13
- Open Issues: 2
-
Metadata Files:
- Readme: Readme.md
- License: LICENSE
Awesome Lists containing this project
README
# S3 Upload Stream
This library performs uploads to S3 without holding the entire object content in memory (or storage) beforehand. This approach is more efficient than using `MemoryStream` and offers compatibility with libraries and packages that work with a `Stream` interface. Examples for extracting GZIP, decrypting AES and PGP, and uploading dynamically-generated CSV are provided.
This project is a follow-on to `SeekableS3Stream`, also on [Github](https://github.com/mlhpdx/seekable-s3-stream). Using both libraries allows for very efficient, and simple, large-file processing to and from S3 via AWS Lambda, as demonstrated in one of the [Examples](https://github.com/mlhpdx/s3-upload-stream/tree/main/Examples).
For a more detailed explaination, check out the article on [Medium](https://medium.com/circuitpeople/stream-to-stream-s3-uploads-with-aws-lambda-578fe710ac1e).