{"id":37302495,"url":"https://github.com/slickquant/slick-queue","last_synced_at":"2026-03-02T10:30:45.207Z","repository":{"id":133274844,"uuid":"309004696","full_name":"SlickQuant/slick-queue","owner":"SlickQuant","description":"A robust, ring buffer based, lock-free, C++ MPMC queue with shared memory support.","archived":false,"fork":false,"pushed_at":"2026-01-28T01:35:39.000Z","size":184,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-01-28T17:26:42.682Z","etag":null,"topics":["cplusplus-20","header-only","lock-free","mpmc-queue","shared-memory"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SlickQuant.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"SlickQuant","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2020-11-01T01:47:06.000Z","updated_at":"2026-01-28T01:35:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"19fd98ac-27ef-4ce0-bad4-ddc31acd70cf","html_url":"https://github.com/SlickQuant/slick-queue","commit_stats":null,"previous_names":["slickquant/slick_queue","slickquant/slick-queue"],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/SlickQuant/slick-queue","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SlickQuant%2Fslick-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SlickQuant%2Fslick-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SlickQuant%2Fslick-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SlickQuant%2Fslick-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SlickQuant","download_url":"https://codeload.github.com/SlickQuant/slick-queue/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SlickQuant%2Fslick-queue/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29092985,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T03:31:03.593Z","status":"ssl_error","status_checked_at":"2026-02-04T03:29:50.742Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cplusplus-20","header-only","lock-free","mpmc-queue","shared-memory"],"created_at":"2026-01-16T02:50:18.626Z","updated_at":"2026-02-04T19:01:40.049Z","avatar_url":"https://github.com/SlickQuant.png","language":"C++","funding_links":["https://github.com/sponsors/SlickQuant"],"categories":[],"sub_categories":[],"readme":"# SlickQueue\n\n[![C++20](https://img.shields.io/badge/C%2B%2B-20-blue.svg)](https://en.cppreference.com/w/cpp/20)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Header-only](https://img.shields.io/badge/header--only-yes-brightgreen.svg)](#installation)\n[![Lock-free](https://img.shields.io/badge/concurrency-lock--free-orange.svg)](#architecture)\n[![CI](https://github.com/SlickQuant/slick-queue/actions/workflows/ci.yml/badge.svg)](https://github.com/SlickQuant/slick-queue/actions/workflows/ci.yml)\n[![GitHub release](https://img.shields.io/github/v/release/SlickQuant/slick-queue)](https://github.com/SlickQuant/slick-queue/releases)\n\nSlickQueue is a header-only C++ library that provides a lock-free,\nmulti-producer multi-consumer (MPMC) queue built on a ring buffer. It is\ndesigned for high throughput concurrent messaging and can optionally operate\nover shared memory for inter-process communication.\n\n## Features\n\n- **Lock-free operations** for multiple producers and consumers\n- **Header-only implementation** - just include and go\n- **Zero dynamic allocation** on the hot path for predictable performance\n- **Shared memory support** for inter-process communication\n- **Cross-platform** - supports Windows, Linux, and macOS\n- **Modern C++20** implementation\n\n## Requirements\n\n- C++20 compatible compiler\n- CMake 3.10+ (for building tests)\n\n## Installation\n\nSlickQueue is header-only. Simply add the `include` directory to your project's include path:\n\n```cpp\n#include \"slick/queue.h\"\n```\n\n### Using vcpkg\n\nSlickQueue is available in the [vcpkg](https://github.com/microsoft/vcpkg) package manager:\n\n```bash\nvcpkg install slick-queue\n```\n\nThen in your CMakeLists.txt:\n\n```cmake\nfind_package(slick-queue CONFIG REQUIRED)\ntarget_link_libraries(your_target PRIVATE slick::queue)\n```\n\n### Using CMake FetchContent\n\n```cmake\ninclude(FetchContent)\n\n# Disable tests for slick-queue\nset(BUILD_SLICK_QUEUE_TESTS OFF CACHE BOOL \"\" FORCE)\nFetchContent_Declare(\n    slick-queue\n    GIT_REPOSITORY https://github.com/SlickQuant/slick-queue.git\n    GIT_TAG v1.1.0.0  # See https://github.com/SlickQuant/slick-queue/releases for latest version\n)\nFetchContent_MakeAvailable(slick-queue)\n\ntarget_link_libraries(your_target PRIVATE slick::queue)\n```\n\n## Usage\n\n### Basic Example\n\n```cpp\n#include \"slick/queue.h\"\n\n// Create a queue with 1024 slots (must be power of 2)\nslick::SlickQueue\u003cint\u003e queue(1024);\n\n// Producer: reserve a slot, write data, and publish\nauto slot = queue.reserve();\n*queue[slot] = 42;\nqueue.publish(slot);\n\n// Consumer: read from queue using a cursor\nuint64_t cursor = 0;\nauto result = queue.read(cursor);\nif (result.first != nullptr) {\n    int value = *result.first;  // value == 42\n}\n```\n\n### Shared Memory Example (IPC)\n\n```cpp\n#include \"slick/queue.h\"\n\n// Process 1 (Server/Writer)\nslick::SlickQueue\u003cint\u003e server(1024, \"my_queue\");\nauto slot = server.reserve();\n*server[slot] = 100;\nserver.publish(slot);\n\n// Process 2 (Client/Reader)\nslick::SlickQueue\u003cint\u003e client(\"my_queue\");\nuint64_t cursor = 0;\nauto result = client.read(cursor);\nif (result.first != nullptr) {\n    int value = *result.first;  // value == 100\n}\n```\n\n### Multi-Producer Multi-Consumer\n\n```cpp\n#include \"slick/queue.h\"\n#include \u003cthread\u003e\n\nslick::SlickQueue\u003cint\u003e queue(1024);\n\n// Multiple producers\nauto producer = [\u0026](int id) {\n    for (int i = 0; i \u003c 100; ++i) {\n        auto slot = queue.reserve();\n        *queue[slot] = id * 1000 + i;\n        queue.publish(slot);\n    }\n};\n\n// Multiple consumers (each maintains independent cursor)\n// Note: Each consumer will see ALL published items (broadcast pattern)\nauto consumer = [\u0026](int id) {\n    uint64_t cursor = 0;\n    int count = 0;\n    while (count \u003c 200) {  // 2 producers × 100 items each\n        auto result = queue.read(cursor);\n        if (result.first != nullptr) {\n            int value = *result.first;\n            ++count;\n        }\n    }\n};\n\nstd::thread p1(producer, 1);\nstd::thread p2(producer, 2);\nstd::thread c1(consumer, 1);\nstd::thread c2(consumer, 2);\n\np1.join(); p2.join();\nc1.join(); c2.join();\n```\n\n### Work-Stealing with Shared Atomic Cursor\n\n```cpp\n#include \"slick/queue.h\"\n#include \u003cthread\u003e\n#include \u003catomic\u003e\n\nslick::SlickQueue\u003cint\u003e queue(1024);\nstd::atomic\u003cuint64_t\u003e shared_cursor{0};\n\n// Multiple producers\nauto producer = [\u0026](int id) {\n    for (int i = 0; i \u003c 100; ++i) {\n        auto slot = queue.reserve();\n        *queue[slot] = id * 1000 + i;\n        queue.publish(slot);\n    }\n};\n\n// Multiple consumers sharing atomic cursor (work-stealing/load-balancing)\n// Each item is consumed by exactly ONE consumer\nauto consumer = [\u0026]() {\n    int count = 0;\n    for (int i = 0; i \u003c 100; ++i) {\n        auto result = queue.read(shared_cursor);\n        if (result.first != nullptr) {\n            int value = *result.first;\n            ++count;\n        }\n    }\n    return count;\n};\n\nstd::thread p1(producer, 1);\nstd::thread p2(producer, 2);\nstd::thread c1(consumer);\nstd::thread c2(consumer);\n\np1.join(); p2.join();\nc1.join(); c2.join();\n// Total items consumed: 200 (each item consumed exactly once)\n```\n\n## API Overview\n\n### Constructor\n\n```cpp\n// In-process queue\nSlickQueue(uint32_t size);\n\n// Shared memory queue\nSlickQueue(uint32_t size, const char* shm_name);  // Writer/Creator\nSlickQueue(const char* shm_name);                  // Reader/Attacher\n```\n\n### Core Methods\n\n- `uint64_t reserve(uint32_t n = 1)` - Reserve `n` slots for writing (non-blocking; may overwrite old data if consumers lag)\n- `T* operator[](uint64_t slot)` - Access reserved slot\n- `void publish(uint64_t slot, uint32_t n = 1)` - Publish `n` written items to consumers\n- `std::pair\u003cT*, uint32_t\u003e read(uint64_t\u0026 cursor)` - Read next available item (independent cursor)\n- `std::pair\u003cT*, uint32_t\u003e read(std::atomic\u003cuint64_t\u003e\u0026 cursor)` - Read next available item (shared atomic cursor for work-stealing)\n- `std::pair\u003cT*, uint32_t\u003e read_last()` - Read the most recently published item without a cursor\n- `uint32_t size()` - Get queue capacity\n- `uint64_t loss_count() const` - Get count of skipped items due to overwrite (debug-only if enabled)\n- `void reset()` - Reset the queue, invalidating all existing data\n\n### Important Constraints\n\n**Lock-Free Atomics Implementation**: SlickQueue uses a packed 64-bit atomic internally to guarantee lock-free operations on all platforms. This packs both the write index (48 bits) and the reservation size (16 bits) into a single atomic value.\n\n**Lossy Semantics**: SlickQueue does not apply backpressure. If producers advance by at least the queue size before a consumer reads, older entries will be overwritten and the consumer will skip ahead to the latest value for a slot. Size the queue and read frequency to bound loss.\n\n**Debug Loss Detection**: Define `SLICK_QUEUE_ENABLE_LOSS_DETECTION=1` to enable a per-instance skipped-item counter (enabled by default in Debug builds). Use `loss_count()` to inspect how many items were skipped.\n\n**CPU Relax Backoff**: Define `SLICK_QUEUE_ENABLE_CPU_RELAX=0` to disable the pause/yield backoff used on contended CAS loops (default is enabled). Disabling may reduce latency in very short contention bursts but can increase CPU usage under load.\n\n**⚠️ Reserve Size Limitation (legacy shared memory)**: Older shared-memory segments used the 16-bit size stored in the packed reservation atomic to compute `read_last()`. New segments track the last published index separately, so this limit no longer applies in normal use.\n\n- If you attach to a shared-memory segment created by older versions, keep `reserve(n) \u003c= 65,535` when using `read_last()`\n- For typical use cases with `reserve()` or `reserve(1)`, this limit is not a concern\n- The 48-bit index supports up to 2^48 (281 trillion) iterations, sufficient for any practical application\n\n## Performance Characteristics\n\n- **Lock-free**: No mutex contention between producers/consumers\n- **Wait-free reads**: Consumers never block each other\n- **Cache-friendly**: Ring buffer design with power-of-2 sizing\n- **Predictable**: No allocations or system calls on hot path (except for initial reserve when full)\n\n## Building and Testing\n\n### Build Tests\n\n```bash\ncmake -S . -B build\ncmake --build build\n```\n\n### Run Tests\n\n```bash\n# Using CTest\ncd build\nctest --output-on-failure\n\n# Or run directly\n./build/tests/slick_queue_tests\n```\n\n### Build Options\n\n- `BUILD_SLICK_QUEUE_TESTS` - Enable/disable test building (default: ON)\n- `CMAKE_BUILD_TYPE` - Set to `Release` or `Debug`\n\n## License\n\nSlickQueue is released under the [MIT License](LICENSE).\n\n**Made with ⚡ by [SlickQuant](https://github.com/SlickQuant)**\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslickquant%2Fslick-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslickquant%2Fslick-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslickquant%2Fslick-queue/lists"}