https://github.com/horusiath/srs

Scalable Redis Streams
https://github.com/horusiath/srs

Last synced: 5 days ago
JSON representation

Scalable Redis Streams

• Host: GitHub
• URL: https://github.com/horusiath/srs
• Owner: Horusiath
• License: other
• Created: 2024-11-04T10:22:03.000Z (2 months ago)
• Default Branch: main
• Last Pushed: 2024-11-05T05:06:12.000Z (2 months ago)
• Last Synced: 2024-11-05T06:18:23.224Z (2 months ago)
• Language: Rust
• Size: 10.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Scalable Redis Streams

A library to provide a way of reading Redis streams in a way to scales up to thousands concurrent stream readers.

## Example

```rust

#[tokio::main]

async fn main() -> Result<(), Error> {

    // create router

    let client = Client::open("redis://127.0.0.1/")?;

    let router = StreamRouter::with_options(&client, StreamRouterOptions {

        // number of reader worker threads - each worker gets its own

        // Redis connection which gets blocked while XREAD is pending

        worker_count: 8,

        // number of streams per worker - total number of Redis streams

        // that can be awaited on concurrently is `worker_count * xread_streams`

        xread_streams: 100,

        // how long does worker awaits before returning from XREAD

        xread_block_millis: Some(0),

        // how many messages should be returned in a single XREAD batch

        // this value ideally should be greater than `xread_streams`

        xread_count: Some(2 * 100),

    })?;

    // if you want to continue from specific point in Redis stream, specify

    // message id to continue reading from

    let last_message_id = Some("0-0".to_string());

    let mut reader = router.observe("test:stream".into(), last_message_id);

    while let Some((message_id, data)) = reader.recv().await {

        let payload: redis::Value = data["field"];

    }

}

```