{"id":16299102,"url":"https://github.com/kvark/choir","last_synced_at":"2025-03-16T13:31:43.732Z","repository":{"id":38314810,"uuid":"479258948","full_name":"kvark/choir","owner":"kvark","description":"Task Orchestration Framework","archived":false,"fork":false,"pushed_at":"2023-11-08T06:49:58.000Z","size":92,"stargazers_count":58,"open_issues_count":5,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-12T02:08:17.178Z","etag":null,"topics":["task-scheduler"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kvark.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-08T05:41:09.000Z","updated_at":"2025-01-22T10:41:19.000Z","dependencies_parsed_at":"2022-09-13T03:33:08.292Z","dependency_job_id":"ac97d8c1-8b68-4b06-b3be-a65790f83c00","html_url":"https://github.com/kvark/choir","commit_stats":{"total_commits":59,"total_committers":1,"mean_commits":59.0,"dds":0.0,"last_synced_commit":"f9daef1c4511224d4d21640a8bfb49c2c2a98ab5"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kvark%2Fchoir","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kvark%2Fchoir/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kvark%2Fchoir/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kvark%2Fchoir/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kvark","download_url":"https://codeload.github.com/kvark/choir/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243817303,"owners_count":20352535,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["task-scheduler"],"created_at":"2024-10-10T20:46:40.906Z","updated_at":"2025-03-16T13:31:43.408Z","avatar_url":"https://github.com/kvark.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Choir\n\n[![Crates.io](https://img.shields.io/crates/v/choir.svg?label=choir)](https://crates.io/crates/choir)\n[![Docs.rs](https://docs.rs/choir/badge.svg)](https://docs.rs/choir)\n[![Build Status](https://github.com/kvark/choir/workflows/Check/badge.svg)](https://github.com/kvark/choir/actions)\n![MSRV](https://img.shields.io/badge/rustc-1.56+-blue.svg)\n[![codecov.io](https://codecov.io/gh/kvark/choir/branch/main/graph/badge.svg)](https://codecov.io/gh/kvark/choir)\n\nChoir is a task orchestration framework. It helps you to organize all the CPU workflow in terms of tasks.\n\n### Example:\n```rust\nlet mut choir = choir::Choir::new();\nlet _worker = choir.add_worker(\"worker\");\nlet task1 = choir.spawn(\"foo\").init_dummy().run();\nlet mut task2 = choir.spawn(\"bar\").init(|_| { println!(\"bar\"); });\ntask2.depend_on(\u0026task1);\ntask2.run().join();\n```\n\n### Selling Pitch\n\nWhat makes Choir _elegant_? Generally when we need to encode the semantics of \"wait for dependencies\", we think of some sort of a counter. Maybe an atomic, for the dependency number. When it reaches zero (or one), we schedule a task for execution. In _Choir_, the internal data for a task (i.e. the functor itself!) is placed in an `Arc`. Whenever we are able to extract it from the `Arc` (which means there are no other dependencies), we move it to a scheduling queue. I think Rust type system shows its best here.\n\nNote: it turns out `Arc` doesn't fully support such a \"linear\" usage as required here, and it's impossible to control where the last reference gets destructed (without logic in `drop()`). For this reason, we introduce our own `Linearc` to be used internally.\n\nYou can also add or remove workers at any time to balance the system load, which may be running other applications at the same time.\n\n## API\n\nGeneral workflow is about creating tasks and setting up dependencies between them. There is a few different kinds of tasks:\n  - single-run tasks, initialized with `init()` and represented as `FnOnce()`\n  - dummy tasks, initialized with `init_dummy()`, and having no function body\n  - multi-run tasks, executed for every index in a range, represented as `Fn(SubIndex)`, and initialized with `init_multi()`\n  - iteration tasks, executed for every item produced by an iterator, represented as `Fn(T)`, and initialized with `init_iter()`\n\nJust calling `run()` is done automatically on `IdleTask::drop()` if not called explicitly.\nThis object also allows adding dependencies before scheduling the task. The running task can be also used as a dependency for others.\n\nNote that all tasks are pre-empted at the `Fn()` execution boundary. Thus, for example, a long-running multi task will be pre-empted by any incoming single-run tasks.\n\n## Users\n\n[Blade](https://github.com/kvark/blade) heavily relies on Choir for parallelizing the asset loading. See [blade-asset talk](https://youtu.be/1DiA3OYqvqU) at Rust Gamedev meetup for details.\n\n### TODO:\n  - detect when dependencies aren't set up correctly\n  - test with [Loom](https://github.com/tokio-rs/loom): blocked by https://github.com/crossbeam-rs/crossbeam/pull/849\n\n## Overhead\n\nMachine: MBP 2016, 3.3 GHz Dual-Core Intel Core i7\n\n- functions `spawn()+init()` (optimized): 237ns\n- \"steal\" task: 61ns\n- empty \"execute\": 37ns\n- dummy \"unblock\": 78ns\n\nExecuting 100k empty tasks:\n- individually: 28ms\n- as a multi-task: 6ms\n\n### Profiling workflow example\n\nWith Tracy:\nAdd this line to the start of the benchmark:\n```rust\nlet _ = profiling::tracy_client::Client::start();\n```\nThen run in command prompt:\n```bash\ncargo bench --features \"profiling/profile-with-tracy\"\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkvark%2Fchoir","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkvark%2Fchoir","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkvark%2Fchoir/lists"}