{"id":13503107,"url":"https://github.com/kanidm/concread","last_synced_at":"2025-05-14T09:12:28.251Z","repository":{"id":40584828,"uuid":"153856000","full_name":"kanidm/concread","owner":"kanidm","description":"Concurrently Readable Data Structures for Rust","archived":false,"fork":false,"pushed_at":"2025-04-11T00:04:24.000Z","size":1408,"stargazers_count":357,"open_issues_count":5,"forks_count":19,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-10T16:05:50.044Z","etag":null,"topics":["concurrency","multiple-readers","single-writers"],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kanidm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-10-20T00:41:21.000Z","updated_at":"2025-05-09T21:48:43.000Z","dependencies_parsed_at":"2023-02-14T05:00:24.170Z","dependency_job_id":"68d93549-0ceb-48b1-b177-712fca44a4fe","html_url":"https://github.com/kanidm/concread","commit_stats":{"total_commits":196,"total_committers":8,"mean_commits":24.5,"dds":"0.15816326530612246","last_synced_commit":"8ff50d44fb78e150708afc445d70bc4362b3c519"},"previous_names":[],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanidm%2Fconcread","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanidm%2Fconcread/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanidm%2Fconcread/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanidm%2Fconcread/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kanidm","download_url":"https://codeload.github.com/kanidm/concread/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254110377,"owners_count":22016391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency","multiple-readers","single-writers"],"created_at":"2024-07-31T22:02:37.318Z","updated_at":"2025-05-14T09:12:28.222Z","avatar_url":"https://github.com/kanidm.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"Concread\n========\n\nConcurrently readable datastructures for Rust.\n\nConcurrently readable is often referred to as Copy-On-Write, Multi-Version-Concurrency-Control.\n\nThese structures allow multiple readers with transactions\nto proceed while single writers can operate. A reader is guaranteed the content\nwill remain the same for the duration of the read, and readers do not block writers.\nWriters are serialised, just like a mutex.\n\nThis library contains concurrently readable Cell types and Map/Cache types.\n\nWhen do I want to use these?\n----------------------------\n\nYou can use these in place of a RwLock, and will likely see improvements in\nparallel throughput.\n\nThe best use is in place of mutex/rwlock, where the reader exists for a\nnon-trivial amount of time.\n\nFor example, if you have a RwLock where the lock is taken, data changed or read, and dropped\nimmediately, this probably won't help you.\n\nHowever, if you have a RwLock where you hold the read lock for any amount of time,\nwriters will begin to stall - or inversely, the writer will cause readers to block\nand wait as the writer proceeds.\n\nConcurrently readable avoids this because readers never stall readers/writers, writers\nnever stall or block a readers. This means that you gain in parallel throughput\nas stalls are reduced.\n\nThis library also has a concurrently readable BTreeMap, HashMap and Adaptive Replacement Cache.\nThese are best used when you have at least 512 bytes worth of data in your Cell, as they only copy\nwhat is required for an update.\n\nIf you do not required key-ordering, then the HashMap will likely be the best choice\nfor most applications.\n\nWhat is concurrently readable?\n------------------------------\n\nIn a multithread application, data is commonly needed to be shared between threads.\nIn sharing this there are multiple policies for this - Atomics for single integer\nreads, Mutexes for single thread access, RwLock for many readers or one writer,\nall the way to Lock Free which allows multiple read and writes of queues.\n\nLock Free however has the limitation of being built on Atomics. This means it can\nreally only update small amounts of data at a time consistently. It also means\nthat you don't have transactional behaviours. While this is great for queues,\nit is not so good for a tree or hashmap where you want the state to be consistent\nfrom the state to the end of an operation. In the few places that lock free trees\nexist, they have the properly that as each thread is updating the tree, the changes\nare visible immediately to all other readers. Your data could change before you\nknow it.\n\nMutexes and RwLock on the other hand allow much more complex structures to be protected.\nThe guarantee that all readers see the same data, always, and that writers are\nthe only writer. But they cause stalls on other threads waiting to access them.\nRwLock for example can see large delays if a reader won't yield, and OS policy\ncan cause reader/writer to starve if the priority favours the other.\n\nConcurrently readable structures sit in between these two points. They provide\nmultiple concurrent readers, with transactional behaviour, while allowing single\nwriters to proceed simultaneously.\n\nThis is achieved by having writers copy the internal data before they modify\nit. This allows readers to access old data, without modification, and allows\nthe writer to change the data inplace before committing. Once the new data is\nstored, old readers continue to access their old data - new readers will\nsee the new data.\n\nThis is a space-time trade off, using more memory to achieve better parallel\nbehaviour.\n\nSafety\n------\n\nThis library has extensive testing, and passes its test suite under [miri], a rust\nundefined behaviour checker. If you find an issue however, please let us know so we can\nfix it!\n\nTo check with miri OR asan on nightly:\n\n    # Follow the miri readme setup steps\n    cargo clean \u0026\u0026 MIRIFLAGS=\"-Zmiri-disable-isolation -Zmiri-disable-stacked-borrows\" cargo +nightly miri test\n    RUSTC_FLAGS=\"-Z sanitizer=address\" cargo test\n\nNote: Miri requires isolation to be disabled so that clock monotonic can be used in ARC for cache channels.\n\n[miri]: https://github.com/rust-lang/miri\n\nSIMD\n----\n\nThere is support for SIMD in ARC if you are using a nightly compiler. To use this, you need to compile\nwith:\n\n    RUSTFLAGS=\"-C target-feature=+avx2,+avx\" cargo ... --features=concread/simd_support\n\nContributing\n------------\n\nPlease open an issue, pr or contact me directly by email (see github)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkanidm%2Fconcread","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkanidm%2Fconcread","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkanidm%2Fconcread/lists"}