https://github.com/rkuhn/batch_queue

A single-producer single-consumer Rust queue with smart batching
https://github.com/rkuhn/batch_queue

concurrency queue rust spsc-queue

Last synced: about 2 months ago
JSON representation

A single-producer single-consumer Rust queue with smart batching

• Host: GitHub
• URL: https://github.com/rkuhn/batch_queue
• Owner: rkuhn
• License: apache-2.0
• Created: 2021-12-19T16:02:19.000Z (about 3 years ago)
• Default Branch: master
• Last Pushed: 2021-12-19T21:19:09.000Z (about 3 years ago)
• Last Synced: 2024-03-15T11:23:04.352Z (10 months ago)
• Topics: concurrency, queue, rust, spsc-queue
• Language: Rust
• Homepage: https://docs.rs/batch_queue
• Size: 19.5 KB
• Stars: 3
• Watchers: 1
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE-APACHE2.md

Awesome Lists containing this project

README

        
# Batching Queue

A library that implements smart batching between a producer and a consumer.

In other words, a single-producer single-consumer queue that tries to ensure that the either side runs for a while (to make use of L1 caches) before hibernating again.

One nice feature of this queue is that all buckets are preallocated, so unless you use the `recv_batch` or `try_recv_batch` methods no allocations are done as part of sending and receiving items.

## Example

```rust

# tokio_test::block_on(async {

use batch_queue::{batch_queue, pipe};

let (tx, mut rx) = batch_queue::(20);

const N: u32 = 10_000_000;

tokio::spawn(async move {

    let stream = futures::stream::iter(0..N);

    pipe(stream, tx).await;

});

let mut x = 0;

while let Ok(iter) = rx.recv().await {

    for y in iter {

        assert_eq!(y, x);

        x += 1;

    }

}

assert_eq!(x, N);

# })

```

Here, the iterator returned by `recv()` allows you to consume the bucket contents without allocating.

On my AMD Hetzner box it takes about 6ns per item.

## How it works

The queue is modeled as a boxed slice of buckets, which each is an appropriately sized array of `MaybeUninit`-ialized items and a fill level.

Reader and writer maintain a current position each, which includes the bucket index as well as a cycle counter (to detect when the writer is exactly one full round ahead of the reader).

Inserting an item:

- check whether the writer’s bucket is available for writing (i.e. it is not the reader’s bucket in the previous cycle)

- if so, insert into next slot in bucket; if full, move write bucket position

Declaring end of current batch:

- check whether current write bucket has elements ⇒ move write bucket position

Taking a bucket:

- check whether the writer’s bucket is ahead of the reader’s bucket ⇒ take it and move read bucket position