https://github.com/arindas/laminarmq-io
Generic I/O abstraction over local File storage and AWS S3 for laminarmq
https://github.com/arindas/laminarmq-io
append-only async-io async-storage aws-s3 generic-associated-types
Last synced: 2 months ago
JSON representation
Generic I/O abstraction over local File storage and AWS S3 for laminarmq
- Host: GitHub
- URL: https://github.com/arindas/laminarmq-io
- Owner: arindas
- License: mit
- Created: 2024-04-10T06:39:51.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-30T11:05:18.000Z (5 months ago)
- Last Synced: 2025-01-23T12:32:31.433Z (4 months ago)
- Topics: append-only, async-io, async-storage, aws-s3, generic-associated-types
- Language: Rust
- Homepage: https://arindas.github.io/laminarmq-io/docs/laminarmq_io/
- Size: 2.35 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
laminarmq-io
An attempt to rewrite the laminarmq I/O layer
to accommodate bounded memory overhead during read/write ops.
## Usage
`laminarmq-io` is a library crate. In order to use it, add the following to your `Cargo.toml`
```toml
[dependencies]
laminarmq-io = { git = "https://github.com/arindas/laminarmq-io.git" }
```
## Overview
This crate provides the following I/O related traits:
| **Trait** | **Type** | **Wrapper Implemented On** |
| ----------------- | ---------------------------------- | ------------------------------------------------------------------------ |
| [`AsyncRead`] | `Read I/O` | [`AsyncBufRead`], [`AsyncRead`]
(struct [`BufReader`]) |
| [`AsyncBufRead`] | `Read I/O` | - |
| [`StreamRead`] | _Streaming_ `Read I/O` | [`AsyncRead`] (struct [`AsyncReadStreamRead`],
struct [`BufReader`]) |
| [`AsyncAppend`] | `Write I/O` | [`AsyncAppend`] (struct [`BufAppender`]) |
| [`StreamAppend`] | _Streaming_ `Write I/O` | [`AsyncAppend`] (_trait impl_) |
| [`AsyncTruncate`] | `Write I/O` | - |
| [`AsyncRemove`] | Management | - |
| [`AsyncClose`] |
Management ![]()
|
- ![]()
|
> The "Wrapper Implemented on" column denotes on which underlying trait, the current trait has an impl with the help of a wrapper struct wrapping the mentioned trait.
> For example, [`StreamRead`] is implemented by a wrapper struct [`AsyncReadStreamRead`] which wraps an [`AsyncRead`] instance.
>
> Some traits in this table also have direct impls on other trait types e.g:
>
> ```text
> impl StreamAppend for T where T: AsyncAppend { /* ... */ }
> ```
>
> They are marked with (_trait impl_).
This library makes the following improvements over existing I/O primitives in `laminarmq`:
- Provides traits at individual operaton level i.e `Read` / `Append` level as opposed to a unified `Storage` trait
- All operations are exclusive with a `&mut self` receiver to avoid internal locks
- Supports both streaming read and streaming write operations
- Provides impls on both filessytem based APIs and cloud object storage APIs such as S3
## License
This repository is licensed under the same terms as [`laminarmq`](https://github.com/arindas/laminarmq).
See [LICENSE](https://raw.githubusercontent.com/arindas/laminarmq-io/main/LICENSE) for more details.