{"id":16150725,"url":"https://github.com/robinfriedli/rusty_pool","last_synced_at":"2025-04-13T06:41:53.814Z","repository":{"id":52203003,"uuid":"260061551","full_name":"robinfriedli/rusty_pool","owner":"robinfriedli","description":"Self growing / shrinking ThreadPool implementation based on crossbeam's multi-producer multi-consumer channels that enables awaiting the result of a task and offers async support.","archived":false,"fork":false,"pushed_at":"2024-05-28T13:39:53.000Z","size":86,"stargazers_count":45,"open_issues_count":5,"forks_count":6,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-12T11:58:59.105Z","etag":null,"topics":["async","concurrency","concurrent","executor","futures","rust","thread","thread-pool","threadpool"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robinfriedli.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["robinfriedli"],"patreon":null,"open_collective":null,"ko_fi":"meteora98","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"custom":null}},"created_at":"2020-04-29T22:41:14.000Z","updated_at":"2025-02-25T18:57:02.000Z","dependencies_parsed_at":"2024-10-25T17:08:09.744Z","dependency_job_id":"d4d0d44d-c9e3-45f3-8a35-1048cefe672f","html_url":"https://github.com/robinfriedli/rusty_pool","commit_stats":{"total_commits":38,"total_committers":2,"mean_commits":19.0,"dds":"0.052631578947368474","last_synced_commit":"d56805869ba3cbe47021d5660bbaf19ac5ec4bfb"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robinfriedli%2Frusty_pool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robinfriedli%2Frusty_pool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robinfriedli%2Frusty_pool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robinfriedli%2Frusty_pool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robinfriedli","download_url":"https://codeload.github.com/robinfriedli/rusty_pool/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248675435,"owners_count":21143763,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","concurrency","concurrent","executor","futures","rust","thread","thread-pool","threadpool"],"created_at":"2024-10-10T00:51:12.572Z","updated_at":"2025-04-13T06:41:53.783Z","avatar_url":"https://github.com/robinfriedli.png","language":"Rust","readme":"# rusty_pool\n\nSelf growing / shrinking `ThreadPool` implementation based on crossbeam's\nmulti-producer multi-consumer channels that enables awaiting the result of a\ntask and offers async support.\n\nThis `ThreadPool` has two different pool sizes; a core pool size filled with\nthreads that live for as long as the channel and a max pool size which describes\nthe maximum amount of worker threads that may live at the same time.\nThose additional non-core threads have a specific keep_alive time described when\ncreating the `ThreadPool` that defines how long such threads may be idle for\nwithout receiving any work before giving up and terminating their work loop.\n\nThis `ThreadPool` does not spawn any threads until a task is submitted to it.\nThen it will create a new thread for each task until the core pool size is full.\nAfter that a new thread will only be created upon an `execute()` call if the\ncurrent pool is lower than the max pool size and there are no idle threads.\n\nFunctions like `evaluate()` and `complete()` return a `JoinHandle` that may be used\nto await the result of a submitted task or future. JoinHandles may be sent to the\nthread pool to create a task that blocks a worker thread until it receives the\nresult of the other task and then operates on the result. If the task panics the\n`JoinHandle` receives a cancellation error. This is implemented using a futures\noneshot channel to communicate with the worker thread.\n\nThis `ThreadPool` may be used as a futures executor if the \"async\" feature is enabled,\nwhich is the case by default. The \"async\" feature includes the `spawn()` and\n`try_spawn()` functions which create a task that polls the future one by one and\ncreates a waker that re-submits the future to the pool when it can make progress.\nWithout the \"async\" feature, futures can simply be executed to completion using\nthe `complete` function, which simply blocks a worker thread until the future has\nbeen polled to completion.\n\nThe \"async\" feature can be disabled if not need by adding the following to your\nCargo dependency:\n```toml\n[dependencies.rusty_pool]\ndefault-features = false\nversion = \"*\"\n```\n\nWhen creating a new worker this `ThreadPool` tries to increment the worker count\nusing a compare-and-swap mechanism, if the increment fails because the total worker\ncount has been incremented to the specified limit (the core_size when trying to\ncreate a core thread, else the max_size) by another thread, the pool tries to create\na non-core worker instead (if previously trying to create a core worker and no idle\nworker exists) or sends the task to the channel instead. Panicking workers are always\ncloned and replaced.\n\nLocks are only used for the join functions to lock the `Condvar`, apart from that\nthis `ThreadPool` implementation fully relies on crossbeam and atomic operations.\nThis `ThreadPool` decides whether it is currently idle (and should fast-return\njoin attempts) by comparing the total worker count to the idle worker count, which\nare two values stored in one `AtomicUsize` (both half the size of usize) making sure\nthat if both are updated they may be updated in a single atomic operation.\n\nThe thread pool and its crossbeam channel can be destroyed by using the shutdown\nfunction, however that does not stop tasks that are already running but will\nterminate the thread the next time it will try to fetch work from the channel.\nThe channel is only destroyed once all clones of the `ThreadPool` have been\nshut down / dropped.\n\n# Installation\n\nTo add rusty_pool to your project simply add the following Cargo dependency:\n```toml\n[dependencies]\nrusty_pool = \"0.7.0\"\n```\n\nOr to exclude the \"async\" feature:\n```toml\n[dependencies.rusty_pool]\nversion = \"0.7.0\"\ndefault-features = false\n```\n\n# Usage\nCreate a new `ThreadPool`:\n```rust\nuse rusty_pool::Builder;\nuse rusty_pool::ThreadPool;\n// Create default `ThreadPool` configuration with the number of CPUs as core pool size\nlet pool = ThreadPool::default();\n// Create a `ThreadPool` with default naming:\nuse std::time::Duration;\nlet pool2 = ThreadPool::new(5, 50, Duration::from_secs(60));\n// Create a `ThreadPool` with a custom name:\nlet pool3 = ThreadPool::new_named(String::from(\"my_pool\"), 5, 50, Duration::from_secs(60));\n// using the Builder struct:\nlet pool4 = Builder::new().core_size(5).max_size(50).build();\n```\n\nSubmit a closure for execution in the `ThreadPool`:\n```rust\nuse rusty_pool::ThreadPool;\nuse std::thread;\nuse std::time::Duration;\nlet pool = ThreadPool::default();\npool.execute(|| {\n    thread::sleep(Duration::from_secs(5));\n    print!(\"hello\");\n});\n```\n\nSubmit a task and await the result:\n```rust\nuse rusty_pool::ThreadPool;\nuse std::thread;\nuse std::time::Duration;\nlet pool = ThreadPool::default();\nlet handle = pool.evaluate(|| {\n    thread::sleep(Duration::from_secs(5));\n    return 4;\n});\nlet result = handle.await_complete();\nassert_eq!(result, 4);\n```\n\nSpawn futures using the `ThreadPool`:\n```rust\nasync fn some_async_fn(x: i32, y: i32) -\u003e i32 {\n    x + y\n}\n\nasync fn other_async_fn(x: i32, y: i32) -\u003e i32 {\n    x - y\n}\n\nuse rusty_pool::ThreadPool;\nlet pool = ThreadPool::default();\n\n// simply complete future by blocking a worker until the future has been completed\nlet handle = pool.complete(async {\n    let a = some_async_fn(4, 6).await; // 10\n    let b = some_async_fn(a, 3).await; // 13\n    let c = other_async_fn(b, a).await; // 3\n    some_async_fn(c, 5).await // 8\n});\nassert_eq!(handle.await_complete(), 8);\n\nuse std::sync::{Arc, atomic::{AtomicI32, Ordering}};\n\n// spawn future and create waker that automatically re-submits itself to the threadpool if ready to make progress, this requires the \"async\" feature which is enabled by default\nlet count = Arc::new(AtomicI32::new(0));\nlet clone = count.clone();\npool.spawn(async move {\n    let a = some_async_fn(3, 6).await; // 9\n    let b = other_async_fn(a, 4).await; // 5\n    let c = some_async_fn(b, 7).await; // 12\n    clone.fetch_add(c, Ordering::SeqCst);\n});\npool.join();\nassert_eq!(count.load(Ordering::SeqCst), 12);\n```\n\nJoin and shut down the `ThreadPool`:\n```rust\nuse std::thread;\nuse std::time::Duration;\nuse rusty_pool::ThreadPool;\nuse std::sync::{Arc, atomic::{AtomicI32, Ordering}};\n\nlet pool = ThreadPool::default();\nfor _ in 0..10 {\n    pool.execute(|| { thread::sleep(Duration::from_secs(10)) })\n}\n// wait for all threads to become idle, i.e. all tasks to be completed including tasks added by other threads after join() is called by this thread or for the timeout to be reached\npool.join_timeout(Duration::from_secs(5));\n\nlet count = Arc::new(AtomicI32::new(0));\nfor _ in 0..15 {\n    let clone = count.clone();\n    pool.execute(move || {\n        thread::sleep(Duration::from_secs(5));\n        clone.fetch_add(1, Ordering::SeqCst);\n    });\n}\n\n// shut down and drop the only instance of this `ThreadPool` (no clones) causing the channel to be broken leading all workers to exit after completing their current work\n// and wait for all workers to become idle, i.e. finish their work.\npool.shutdown_join();\nassert_eq!(count.load(Ordering::SeqCst), 15);\n```\n\n# Performance\nIn terms of performance from the perspective of a thread submitting tasks to the pool, rusty_pool should offer better\nperformance than any pool using std::sync::mpsc (such as rust-threadpool) in most scenarios thanks to the great work of\nthe crossbeam team. In some cases with extreme contention rusty_pool might fall behind rust-threadpool, though the scenarios\nwhere this has been found to be the case are hardly practical as they require to submit empty tasks in a loop and it depends\non the platform. macOS seems to perform particularly well in the tested scenario, presumably macOS has spent a lot of\neffort optimising atomic operations as Swift's reference counting depends on it. Apparently this should be amplified on\nApple Silicon but rusty_pool has not been tested on that platform. The following tests were executed on a PC with an\nAMD Ryzen 9 3950X for Linux and Windows and on a MacBook Pro 15\" 2019 with an Intel i9-9880H for macOS.\n\n### Test 1: No contention\nAll tasks are submitted by the same thread and the task lasts longer than the test, meaning all atomic operations\n(reading and incrementing the worker counter) are performed by the main thread, since newly created workers do not\nalter the counter until after they completed their initial task and increment the idle counter.\n\n```rust\nfn main() {\n    let now = std::time::Instant::now();\n\n    let pool = rusty_pool::Builder::new().core_size(10).max_size(10).build();\n    //let pool = threadpool::ThreadPool::new(10);\n\n    for _ in 0..10000000 {\n        pool.execute(|| {\n            thread::sleep(std::time::Duration::from_secs(1));\n        });\n    }\n\n    let millis = now.elapsed().as_millis();\n    println!(\"millis: {}\", millis);\n}\n```\n\nResults (in milliseconds, average value):\n\nrusty_pool 0.5.1:\n\n| Windows | MacOS | Linux |\n|---------|-------|-------|\n| 221.6   | 293.07| 183.73|\n\nrusty_pool 0.5.0:\n\n| Windows | MacOS | Linux |\n|---------|-------|-------|\n| 224.6   | 315.6 | 187.0 |\n\nrust-threadpool 1.8.1:\n\n| Windows | MacOS | Linux |\n|---------|-------|-------|\n| 476.4   | 743.4 | 354.3 |\n\nrusty_pool 0.4.3:\n\n| Windows | MacOS | Linux |\n|---------|-------|-------|\n| 237.5   | 318.1 | 181.3 |\n\n### Test 2: Multiple producers\nNext to the main thread there are 10 other threads submitting tasks to the pool. Unlike the previous test, the task no\nlonger lasts longer than the test, thus there not only is contention between the producers for the worker counter but also\nbetween the worker threads updating the idle counter. This is a somewhat realistic albeit extreme example.\n\n```rust\nfn main() {\n    let now = std::time::Instant::now();\n\n    let pool = rusty_pool::Builder::new().core_size(10).max_size(10).build();\n    //let pool = threadpool::ThreadPool::new(10);\n\n    for _ in 0..10 {\n        let pool = pool.clone();\n\n        std::thread::spawn(move || {\n            for _ in 0..10000000 {\n                pool.execute(|| {\n                    std::thread::sleep(std::time::Duration::from_secs(1));\n                });\n            }\n        });\n    }\n\n    for _ in 0..10000000 {\n        pool.execute(|| {\n            std::thread::sleep(std::time::Duration::from_secs(1));\n        });\n    }\n\n    let millis = now.elapsed().as_millis();\n    println!(\"millis: {}\", millis);\n}\n```\n\nResults (in milliseconds, average value):\n\nrusty_pool 0.5.1:\n\n| Windows* | MacOS  | Linux  |\n|----------|--------|--------|\n| 7692.4   | 3656.2 | 7514.53|\n\nrusty_pool 0.5.0:\n\n| Windows  | MacOS  | Linux  | Windows*|\n|----------|--------|--------|---------|\n| 6251.0   | 4417.7 | 7903.1 | 7774.67 |\n\nrust-threadpool 1.8.1:\n\n| Windows  | MacOS  | Linux  |\n|----------|--------|--------|\n| 10030.5  | 5810.5 | 9743.3 |\n\nrusty_pool 0.4.3:\n\n| Windows  | MacOS  | Linux  | Windows*|\n|----------|--------|--------|---------|\n| 6342.2   | 4444.6 | 7962.0 | 8564.93 |\n\n\u0026ast; When testing 0.5.1 the performance for Windows appeared to be considerably worse, so the results for previous versions\nof rusty_pool were recalculated and also found to be worse than when originally recorded, probably due to external\ninfluence (e.g. background task taking a lot of CPU time, though the test was retried with realtime priority with similar\nresults). The results for rust-threadpool 1.8.1 were not fully recalculated as they appeared to be similar to the last\nrecording.\n\n### Test 3: Worst case\nThis test case highlights the aforementioned worst-case scenario for rusty_pool where the pool is spammed with empty\ntasks. Since workers increment the idle counter after completing a task and the task is executed practically immediately,\nthe increment of the idle counter coincides with the next execute() call in the loop reading the counter. The higher the\nnumber of workers the higher contention gets and the worse performance becomes.\n\n```rust\nfn main() {\n    let now = std::time::Instant::now();\n\n    let pool = rusty_pool::Builder::new().core_size(10).max_size(10).build();\n    //let pool = threadpool::ThreadPool::new(10);\n\n    for _ in 0..10000000 {\n        pool.execute(|| {});\n    }\n\n    let millis = now.elapsed().as_millis();\n    println!(\"millis: {}\", millis);\n}\n```\n\nrusty_pool 0.5.1:\n\n| Windows  | MacOS  | Linux  |\n|----------|--------|--------|\n| 1967.93  | 698.8  | 2150.0 |\n\nrusty_pool 0.5.0:\n\n| Windows  | MacOS  | Linux  |\n|----------|--------|--------|\n| 1991.6   | 679.93 | 2175.1 |\n\nrust-threadpool 1.8.1:\n\n| Windows  | MacOS  | Linux  |\n|----------|--------|--------|\n| 980.33   | 1224.6 | 677.0  |\n\nrusty_pool 0.4.3:\n\n| Windows  | MacOS  | Linux  |\n|----------|--------|--------|\n| 2016.8   | 683.13 | 2175.1 |\n\nCuriously, macOS heavily favours rusty_pool in this case while Windows and Linux favour rust-threadpool. However, this test\ncase should hardly occur in a real world scenario. In all other tested scenarios rusty_pool performs better when submitting\ntasks, where macOS seems to gain a lead in cases where there is a lot of contention but falling behind in other cases,\npossibly due to the weaker hardware of the specific device used for testing. Linux seems to perform best in cases with\nlittle to no contention but performs the worst when contention is high.","funding_links":["https://github.com/sponsors/robinfriedli","https://ko-fi.com/meteora98"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinfriedli%2Frusty_pool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobinfriedli%2Frusty_pool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinfriedli%2Frusty_pool/lists"}