{"id":15047888,"url":"https://github.com/codeexmachina/blockingcollection","last_synced_at":"2025-04-10T01:06:09.153Z","repository":{"id":144159570,"uuid":"151661390","full_name":"CodeExMachina/BlockingCollection","owner":"CodeExMachina","description":"C++11 thread safe, multi-producer, multi-consumer blocking queue, stack \u0026 priority queue class","archived":false,"fork":false,"pushed_at":"2019-11-05T13:20:34.000Z","size":141,"stargazers_count":64,"open_issues_count":2,"forks_count":23,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-10T01:06:00.296Z","etag":null,"topics":["cplusplus-11","cpp11","header-only","queue"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CodeExMachina.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-10-05T02:27:01.000Z","updated_at":"2025-02-25T14:42:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"ca5610b0-22b7-435b-8673-f701ccfe36ab","html_url":"https://github.com/CodeExMachina/BlockingCollection","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeExMachina%2FBlockingCollection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeExMachina%2FBlockingCollection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeExMachina%2FBlockingCollection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeExMachina%2FBlockingCollection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CodeExMachina","download_url":"https://codeload.github.com/CodeExMachina/BlockingCollection/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248137888,"owners_count":21053775,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cplusplus-11","cpp11","header-only","queue"],"created_at":"2024-09-24T21:05:44.181Z","updated_at":"2025-04-10T01:06:09.144Z","avatar_url":"https://github.com/CodeExMachina.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BlockingCollection\nBlockingCollection is a C++11 thread safe collection class that provides the following features:\n- Modeled after .NET BlockingCollection class. \n- Implementation of classic Producer/Consumer pattern (i.e. condition variable, mutex);   \n- Adding and taking of items from multiple threads.\n- Optional maximum capacity.\n- Insertion and removal operations that block when collection is empty or full.\n- Insertion and removal \"try\" operations that do not block or that block up to a specified period of time.\n- Insertion and removal 'bulk\" operations that allow more than one element to be added or taken at once. \n- Priority-based insertion and removal operations. \n- Encapsulates any collection type that satisfy the ProducerConsumerCollection requirement.\n- [Minimizes](#performance-optimizations) sleeps, wake ups and lock contention by managing an active subset of producer and consumer threads.\n- Pluggable condition variable and lock types.\n- Range-based loop support.\n\n## Bounding and Blocking Support\nBlockingCollection\u003cT\u003e supports bounding and blocking. Bounding means that you can set the maximum capacity\nof the collection. Bounding is important in certain scenarios because it enables you to control the maximum\nsize of the collection in memory, and it prevents the producing threads from moving too far ahead of the consuming threads. \n  \nMultiple threads or tasks can add elements to the collection, and if the collection reaches its specified maximum capacity, the producing threads will block until an element is removed. Multiple consumers can remove elements, and if the collection becomes empty, the consuming threads will block until a producer adds an item. A producing thread can call the complete_adding method to indicate that no more elements will be added. Consumers monitor the is_completed property to know when the collection is empty and no more elements will be added. The following example shows a simple BlockingCollection with a bounded capacity of 100. A producer task adds items to the collection as long as some external condition is true, and then calls complete_adding. The consumer task takes items until the is_completed property is true.  \n```C++\n// A bounded collection. It can hold no more \n// than 100 items at once.\nBlockingCollection\u003cData*\u003e collection(100);\n\n// a simple blocking consumer\nstd::thread consumer_thread([\u0026collection]() {\n\n  while (!collection.is_completed())\n  {\n      Data* data;\n      \n      // take will block if there is no data to be taken\n      auto status = collection.take(data);\n      \n      if(status == BlockingCollectionStatus::Ok)\n      {\n          process(data);\n      }\n      \n      // Status can also return BlockingCollectionStatus::Completed meaning take was called \n      // on a completed collection. Some other thread can call complete_adding after we pass \n      // the is_completed check but before we call take. In this example, we can ignore that\n      // status since the loop will break on the next iteration.\n  }\n});\n\n// a simple blocking producer\nstd::thread producer_thread([\u0026collection]() {\n\n  while (moreItemsToAdd)\n  {\n      Data* data = GetData(data);\n      \n      // blocks if collection.size() == collection.bounded_capacity()\n      collection.add(data);\n  }\n  \n  // let consumer know we are done\n  collection.complete_adding();\n});\n```\n## Timed Blocking Operations\nIn timed blocking try_add and try_take operations on bounded collections, the method tries to add or take an item. If an item is available it is placed into the variable that was passed in by reference, and the method returns Ok. If no item is retrieved after a specified time-out period the method returns TimedOut. The thread is then free to do some other useful work before trying again to access the collection.\n```C++\n  BlockingCollection\u003cData*\u003e collection(100);\n  \n  Data *data;\n  \n  // if the collection is empty this method returns immediately\n  auto status = collection.try_take(data);\n  \n  // if the collection is still empty after 1 sec this method returns immediately\n  status = collection.try_take(data, std::chrono::milliseconds(1000));\n  \n  // in both case status will return BlockingCollectionStatus::TimedOut if \n  // try_take times out waiting for data to become available\n```\n## Bulk Operations\nBlockingCollection's add and take operations are all thread safe. But it accomplishes this by using a mutex. To minimize mutex contention when adding or taking elements BlockingCollection supports bulk operations. It is usually much cheaper to acquire the mutex and then to add or take a whole batch of elements in one go, than it is to acquire and release the mutex for every add and take.\n```C++\nBlockingCollection\u003cData*\u003e collection(100);\n\nstd::array\u003cData*, 20\u003e arr;\n\nsize_t taken;\n\nauto status = collection.try_take_bulk(arr, arr.size(), taken);\n\n// try_take_bulk will update taken with actual number of items taken\n```\n## Specifying the Collection Type\nWhen you create a BlockingCollection object, you can specify not only the bounded capacity but also the type of collection to use. \nFor example, you could specify a ```QueueContainer\u003cT\u003e``` object for first in, first out (FIFO) behavior, or a ```StackContainer\u003cT\u003e``` object for last in, first out (LIFO) behavior. You can use any collection class that supports the ProducerConsumerCollection requirement. The default collection type for BlockingCollection is ```QueueContainer\u003cT\u003e```. The following code example shows how to create a BlockingCollection of strings that has a capacity of 1000 and uses a ```StackContainer\u003cT\u003e```\n```C++\nBlockingCollection\u003cstd::string, StackContainer\u003cstd::string\u003e\u003e stack(1000);\n```\nType aliases are also available:\n```C++\nBlockingQueue\u003cstd::string\u003e blocking_queue;\nBlockingStack\u003cstd::string\u003e blocking_stack;\n```\n## Priority based Insertion and Removal\nPriorityBlockingCollection offers the same functionality found in BlockingCollection\u003cT, PriorityQueue\u003e.\nBut the add/try_add methods add items to the collection based on their priority - (0 is lowest priority).\n\nFIFO order is maintained when items of the same priority are added consecutively. And the take/try_take methods\nreturn the highest priority items in FIFO order.\n\nIn addition, PriorityBlockingCollection adds additional methods (i.e. take_prio/try_take_prio) for taking\nthe lowest priority items.\n\nPriorityBlockingCollection's default priority comparer expects that the objects being compared have\noverloaded \u003c and \u003e operators. If this is not the case then you can provide your own comparer implementation\nlike in the following example.\n```C++\nstruct PriorityItem {\n  PriorityItem(int priority) : Priority(priority) \n  {}\n  \n  int Priority;\n};\n\nclass CustomComparer {\npublic:\n  CustomComparer() {\n  }\n\n  int operator() (const PriorityItem \u0026item, const PriorityItem \u0026new_item) {\n    if (item.Priority \u003c new_item.Priority)\n      return -1;\n    else if (item.Priority \u003e new_item.Priority)\n      return 1;\n    else\n      return 0;\n  }\n};\n\nusing CustomPriorityContainer = PriorityContainer\u003cPriorityItem, CustomComparer\u003e;\n\nPriorityBlockingCollection\u003cPriorityItem, CustomPriorityContainer\u003e collection;\n```\n## Range-based for loop Support\nBlockingCollection provides an iterator that enables consumers to use ```for(auto item : collection) { ... }```to remove items until the collection is completed, which means it is empty and no more items will be added. For more information, see\n```C++\nBlockingCollection\u003cData\u003e collection(100);\n\n// a simple blocking consumer using range-base loop\nstd::thread consumer_thread([\u0026collection]() {\n        \n    for(auto data : collection) {\n        process(data);\n    }    \n});\n```\n## ProducerConsumerCollection Requirement  \nIn order for a container to be used with the BlockingCollection\u003cT\u003e it must meet the ProducerConsumerCollection requirement.\nThe ProducerConsumerCollection requires that all the following method signatures must be supported:\n  \n- size_type size()\n- bool try_add(const value_type\u0026 element)\n- bool try_add(value_type\u0026\u0026 element)\n- bool try_take(value_type\u0026 element)\n- template \u003ctypename... Args\u003e bool try_emplace(Args\u0026\u0026... args)\n\nBlockingCollection currently supports three containers:\n\n- QueueContainer\n- StackContainer\n- PriorityContainer\n\n## Performance Optimizations\nBlockingCollection can behave like most condition variable based collections. That is, it will by default issue a signal each time a element is added or taken from its underlying Container. But this approach leads to poor application scaling and performance. \n\nSo in the interest of performance BlockingCollection can be configured to maintain a subset of active threads that are currently adding and taking elements. This is important because it allows BlockingCollection not to have to issue a signal each time an element is added or taken. Instead in the case of consumers it issues a signal only when an element is taken and there are no active consumers, or when the Container's element count starts to grow beyond a threshold level. And in the case of producers, BlockingCollection will issue a signal only when an element is added and there are no active producers or when the Container's available capacity starts to grow beyond a threshold level. \n\nIn both cases, this approach greatly improves performance and makes it more predictable.\n\nTwo strategy classes are responsible for implementing the behavior just described.\n\n1. NotEmptySignalStrategy\n   - implements conditions under which a \"not empty\" condition variable should issue a signal\n   \n2. NotFullSignalStrategy\n   - implements conditions under which a \"not full\" condition variable should issue a signal\n\n### NotEmptySignalStrategy\nThis strategy will return true under two conditions.\n\n1. All consumers are currently not active (i.e. waiting)\n2. Number of active consumers \u003c total consumers AND number of item in collection per active consumer is \u003e threshold value\n```C++\ntemplate\u003csize_t ItemsPerThread = 16\u003e struct NotEmptySignalStrategy {\n\n    bool should_signal(size_t active_workers, size_t total_workers, size_t item_count, size_t /*capacity*/) const {\n        return active_workers == 0 || (active_workers \u003c total_workers \u0026\u0026 item_count / active_workers \u003e ItemsPerThread);\n    }\n};\n```\n### NotFullSignalStrategy\nThis strategy will return true under two conditions.\n\n1. All producers are currently not active (i.e. waiting)\n2. Number of active producers \u003c total producers AND current available capacity per active producer is \u003e threshold value\n```C++\ntemplate\u003csize_t ItemsPerThread = 16\u003e struct NotFullSignalStrategy {\n\n    bool should_signal(size_t active_workers, size_t total_workers, size_t item_count, size_t capacity) const {\n        return (active_workers == 0 || (active_workers \u003c total_workers \u0026\u0026 (capacity - item_count) / active_workers \u003e ItemsPerThread));\n    }\n};\n```\nNote that in both strategies the threshold value (i.e. ItemsPerThread) can be specified. And that completely new strategies can be used for both \"no empty\" and \"not full\" use cases by creating a new strategy that implements the required method signature.\n```C++\nbool should_signal(size_t active_workers, size_t total_workers, size_t item_count, size_t capacity)\n```\n### Attaching/Detaching Consumers \u0026 Producers\nIn order for BlockingCollection to maintain a subset of active threads it exposes attach_producer and attach_consumer member function. The calling thread can call either of those functions to attach itself to the BlockingCollection as either a producer or consumer respectively. Note that the thread should remember to detach itself in both cases.\n```C++\nBlockingCollection\u003cData\u003e collection(100);\n  \nstd::thread consumer_thread([\u0026collection]() {\n\n  collection.attach_consumer();\n  \n  int item;\n  \n  for(int i = 0; i \u003c 10; i++)\n    collection.take(item);\n    \n  collection.detach_consumer();\n});\n```\n### Consumer \u0026 Producer Guards\nIn order to mitigate forgetting to attach or detach from a BlockingCollection the BlockingCollection Guard classes (i.e. ProducerGuard\u003cT\u003e and ConsumerGuard\u003cT\u003e) can be used for this purpose. Both Guard classes are a RAII-style mechanism for attaching a thread to the BlockingCollection and detaching it when the thread terminates. As well as in exception scenarios.\n  \nIn the following examples, ConsumerGuard and ProducerGuard automatically attach and detach the std::threads to the BlockingCollection.\n```C++\nBlockingCollection\u003cint\u003e collection;\n  \nstd::thread consumer_thread([\u0026collection]() {\n\n  ConsumerGuard\u003cBlockingCollection\u003cint\u003e\u003e Guard(collection);\n  \n  int item;\n  \n  for(int i = 0; i \u003c 10; i++)\n    collection.take(item);\n});\n```\n```C++\nstd::thread producer_thread([\u0026collection]() {\n\n  ProducerGuard\u003cBlockingCollection\u003cint\u003e\u003e Guard(collection);\n\n  for(int i = 0; i \u003c 10; i++)\n    collection.add(i+1);\n});\n```\n## Pluggable Condition Variable and Lock Types\nThe BlockingCollection class by default will use std::condition_variable and std::mutex classes. But those two synchronization primitives can be overridden by specializing ConditionVarTraits.\n```C++\ntemplate \u003ctypename ConditionVarType,  typename LockType\u003e\nstruct ConditionVarTraits;\n\ntemplate \u003c\u003e\nstruct ConditionVarTraits\u003cstd::condition_variable, std::mutex\u003e\n{\n    static void initialize(std::condition_variable\u0026 cond_var) {\n    }\n\n    static void signal(std::condition_variable\u0026 cond_var) {\n        cond_var.notify_one();\n    }\n\n    static void broadcast(std::condition_variable\u0026 cond_var) {\n        cond_var.notify_all();\n    }\n\n    static void wait(std::condition_variable\u0026 cond_var, std::unique_lock\u003cstd::mutex\u003e\u0026 lock) {\n        cond_var.wait(lock);\n    }\n\n    template\u003cclass Rep, class Period\u003e static bool wait_for(std::condition_variable\u0026 cond_var, std::unique_lock\u003cstd::mutex\u003e\u0026 lock, const std::chrono::duration\u003cRep, Period\u003e\u0026 rel_time) {\n        return std::cv_status::timeout == cond_var.wait_for(lock, rel_time);\n    }\n};\n```\nIn the following example, ConditionVarTraits is specialized to use Win32 CONDITION_VARIABLE and SRW_LOCK.\n\nNote that Win32's SRWLOCK synchronization primitive requires a wrapper class so that it can meet the BasicLockable requirements needed by std::unique_lock.\n```C++\nclass WIN32_SRWLOCK {\npublic:\n    WIN32_SRWLOCK() {\n        InitializeSRWLock(\u0026srw_);\n    }\n\n    void lock() {\n        AcquireSRWLockExclusive(\u0026srw_);\n    }\n\n    void unlock() {\n        ReleaseSRWLockExclusive(\u0026srw_);\n    }\n\n    SRWLOCK\u0026 native_handle() {\n        return srw_;\n    }\nprivate:\n    SRWLOCK srw_;\n};\n\ntemplate \u003c\u003e\nstruct ConditionVarTraits\u003cCONDITION_VARIABLE, WIN32_SRWLOCK\u003e\n{\n    static void initialize(CONDITION_VARIABLE\u0026 cond_var) {\n      InitializeConditionVariable(\u0026cond_var);\n    }\n\n    static void signal(CONDITION_VARIABLE\u0026 cond_var) {\n      WakeConditionVariable(\u0026cond_var);\n    }\n\n    static void broadcast(CONDITION_VARIABLE\u0026 cond_var) {\n      WakeAllConditionVariable(\u0026cond_var);\n    }\n\n    static void wait(CONDITION_VARIABLE\u0026 cond_var, std::unique_lock\u003cWIN32_SRWLOCK\u003e\u0026 lock) {\n      SleepConditionVariableSRW(\u0026cond_var, \u0026lock.mutex()-\u003enative_handle(), INFINITE, 0);\n    }\n\n    template\u003cclass Rep, class Period\u003e static bool wait_for(CONDITION_VARIABLE\u0026 cond_var, std::unique_lock\u003cWIN32_SRWLOCK\u003e\u0026 lock, const std::chrono::duration\u003cRep, Period\u003e\u0026 rel_time) {\n    \n      DWORD milliseconds = static_cast\u003cDWORD\u003e(rel_time.count());\n\n      if (!SleepConditionVariableSRW(\u0026cond_var, \u0026lock.mutex()-\u003enative_handle(), milliseconds, 0)) {\n        if (GetLastError() == ERROR_TIMEOUT)\n          return true;\n      }\n      return false;\n    }\n};\n```\n## Condition Variable Generator\nThe BlockingCollection class uses the ConditionVariableGenerator template to generate the condition variable and lock types it will use. In addition, the template also generates the strategy classes.\n```C++\ntemplate\u003ctypename ThreadContainerType, typename NotFullSignalStrategy, typename NotEmptySignalStrategy, typename ConditionVarType, typename LockType\u003e \nstruct ConditionVariableGenerator {\n\n    using NotFullType = ConditionVariable\u003cThreadContainerType, NotFullSignalStrategy, ConditionVarType, LockType\u003e;\n    using NotEmptyType = ConditionVariable\u003cThreadContainerType, NotEmptySignalStrategy, ConditionVarType, LockType\u003e;\n\n    using lock_type = LockType;\n};\n```\nBy default, the BlockingCollection class will use the following ConditionVariableGenerator type alias.\n```C++\nusing StdConditionVariableGenerator = ConditionVariableGenerator\u003cThreadContainer\u003cstd::thread::id\u003e, NotFullSignalStrategy\u003c16\u003e, NotEmptySignalStrategy\u003c16\u003e, std::condition_variable, std::mutex\u003e;\n```\nBut it can easily be replaced by something else such as the following.\n```C++\nusing Win32ConditionVariableGenerator = ConditionVariableGenerator\u003cThreadContainer\u003cstd::thread::id\u003e, NotFullSignalStrategy\u003c16\u003e, NotEmptySignalStrategy\u003c16\u003e, CONDITION_VARIABLE, WIN32_SRWLOCK\u003e;\n```\nA custom condition variable generator can be used like so:\n```C++\nBlockingCollection\u003cint, QueueContainer\u003cint\u003e, Win32ConditionVariableGenerator\u003e collection;\n```\n## References\n1. ^ [a](#bounding-and-blocking-support) [b](#timed-blocking-operations) [c](#specifying-the-collection-type) Microsoft Docs, *\"BlockingCollection Overview\"* [link](https://docs.microsoft.com/en-us/dotnet/standard/collections/thread-safe/blockingcollection-overview)\n\nBlockingCollection implements optimizations described in the following paper:\n\n[2007] Hewlett Packard Development Company, L.P *\"Techniques for Improving the Scalability of Applications Using POSIX Thread Condition Variables\"* [pdf](https://www.yumpu.com/en/document/view/18689696/making-condition-variables-perform-hp)\n\n## License\nBlockingCollection uses the GPLv3 license that is available [here](https://github.com/CodeExMachina/BlockingCollection/blob/master/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodeexmachina%2Fblockingcollection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodeexmachina%2Fblockingcollection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodeexmachina%2Fblockingcollection/lists"}