{"id":13533607,"url":"https://github.com/bloomen/transwarp","last_synced_at":"2025-04-01T22:30:26.298Z","repository":{"id":39498742,"uuid":"79993446","full_name":"bloomen/transwarp","owner":"bloomen","description":"A header-only C++ library for task concurrency","archived":false,"fork":false,"pushed_at":"2023-05-18T09:33:16.000Z","size":6589,"stargazers_count":621,"open_issues_count":2,"forks_count":45,"subscribers_count":37,"default_branch":"master","last_synced_at":"2024-08-02T07:22:57.360Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bloomen.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2017-01-25T07:43:33.000Z","updated_at":"2024-07-30T09:14:16.000Z","dependencies_parsed_at":"2024-01-31T09:15:25.246Z","dependency_job_id":null,"html_url":"https://github.com/bloomen/transwarp","commit_stats":null,"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomen%2Ftranswarp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomen%2Ftranswarp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomen%2Ftranswarp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomen%2Ftranswarp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bloomen","download_url":"https://codeload.github.com/bloomen/transwarp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222774628,"owners_count":17035752,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T07:01:21.413Z","updated_at":"2024-11-02T20:31:47.359Z","avatar_url":"https://github.com/bloomen.png","language":"C++","readme":"# transwarp \n\n[![Actions](https://github.com/bloomen/transwarp/actions/workflows/transwarp-tests.yml/badge.svg?branch=master)](https://github.com/bloomen/transwarp/actions/workflows/transwarp-tests.yml?query=branch%3Amaster)\n\n\u003ca href=\"https://bloomen.github.io/transwarp\"\u003eDoxygen documentation\u003c/a\u003e\n\ntranswarp is a header-only C++ library for task concurrency. It\nallows you to easily create a graph of tasks where every task can be executed\nasynchronously. transwarp is written in C++17 and only depends on the standard\nlibrary. Just copy `include/transwarp.h` to your project and off you go!\nTested with GCC, Clang, ICC, and Visual Studio.\n\nC++11 support can be enabled by defining `TRANSWARP_CPP11` at compile time.\n\n**Important:** Only use tagged releases of transwarp in production code!\n\n**Table of contents**\n\n  * [Example](#example)\n  * [API doc](#api-doc)\n     * [Creating tasks](#creating-tasks)\n     * [Scheduling tasks](#scheduling-tasks)\n     * [Executors](#executors)\n     * [Range functions](#range-functions)\n     * [Canceling tasks](#canceling-tasks)\n     * [Event system](#event-system)\n     * [Task pool](#task-pool)\n     * [Timing tasks](#timing-tasks)\n     * [Optimizing efficiency](#optimizing-efficiency)\n  * [Feedback](#feedback)\n  * [Contributors](#contributors)\n\n## Example\n\nThis example creates three tasks and connects them with each other to form\na two-level graph. The tasks are then scheduled twice for computation \nwhile using 4 threads.\n```cpp\n#include \u003cfstream\u003e\n#include \u003ciostream\u003e\n#include \"transwarp.h\"\n\nnamespace tw = transwarp;\n\nint main() {\n    double x = 0;\n    int y = 0;\n\n    // Building the task graph\n    auto parent1 = tw::make_task(tw::root, [\u0026x]{ return 13.3 + x; })-\u003enamed(\"something\");\n    auto parent2 = tw::make_task(tw::root, [\u0026y]{ return 42 + y; })-\u003enamed(\"something else\");\n    auto child = tw::make_task(tw::consume, [](double a, int b) { return a + b;\n                                            }, parent1, parent2)-\u003enamed(\"adder\");\n\n    tw::parallel executor{4};  // Parallel execution with 4 threads\n\n    child-\u003eschedule_all(executor);  // Schedules all tasks for execution\n    std::cout \u003c\u003c \"result = \" \u003c\u003c child-\u003eget() \u003c\u003c std::endl;  // result = 55.3\n\n    // Modifying data input\n    x += 2.5;\n    y += 1;\n\n    child-\u003eschedule_all(executor);  // Re-schedules all tasks for execution\n    std::cout \u003c\u003c \"result = \" \u003c\u003c child-\u003eget() \u003c\u003c std::endl;  // result = 58.8\n\n    // Creating a dot-style graph for visualization\n    std::ofstream{\"basic_with_three_tasks.dot\"} \u003c\u003c tw::to_string(child-\u003eedges());\n}\n```\n\nThe resulting graph of this example looks like this:\n\n![graph](https://raw.githubusercontent.com/bloomen/transwarp/master/examples/basic_with_three_tasks.png)\n\nEvery bubble represents a task and every arrow an edge between two tasks. \nThe first line within a bubble is the task name. The second line denotes the task\ntype followed by the task id and the task level in the graph.\n\n## API doc\n\nThis is a brief API doc of transwarp. \nFor more details check out the \u003ca href=\"https://bloomen.github.io/transwarp\"\u003edoxygen documentation\u003c/a\u003e\nand the \u003ca href=\"https://github.com/bloomen/transwarp/tree/master/examples\"\u003etranswarp examples\u003c/a\u003e.\n\nIn the following we will use `tw` as a namespace alias for `transwarp`.\n\n### Creating tasks\n\ntranswarp supports seven different task types:\n```cpp\nroot, // The task has no parents\naccept, // The task's functor accepts all parent futures\naccept_any, // The task's functor accepts the first parent future that becomes ready\nconsume, // The task's functor consumes all parent results\nconsume_any, // The task's functor consumes the first parent result that becomes ready\nwait, // The task's functor takes no arguments but waits for all parents to finish\nwait_any, // The task's functor takes no arguments but waits for the first parent to finish\n```\nThe task type is passed as the first parameter to `make_task`, e.g., to create \na `consume` task simply do this:\n```cpp\nauto task = tw::make_task(tw::consume, functor, parent1, parent2);\n```\nwhere `functor` denotes some callable and `parent1/2` the parent tasks. \n\nThe functor as passed to `make_task` needs to fulfill certain requirements based\non the task type and the given parents:\n\n**_root_**: A task at the root (top) of the graph. This task gets executed first.\nA functor to a `root` task cannot have any parameters since this task does not\nhave parent tasks, e.g.:\n```cpp\nauto task = tw::make_task(tw::root, []{ return 42; });\n```\nAnother way of defining a`root` task is a _value task_ which can be created as:\n```cpp\nauto task = tw::make_value_task(42);  \n```\nA value task doesn't require scheduling and always returns the same value or exception.\n\n**_accept_**: This task is required to have at least one parent. It _accepts_\nthe resulting parent futures as they are without unwrapping. Hence, the child\ncan decide how to proceed since a call to `get()` can potentially throw an\nexception. Here's an example:\n```cpp\nauto task = tw::make_task(tw::accept, [](auto f1, auto f2) { return f1.get() + f2.get(); }, parent1, parent2);\n```\n\n**_accept_any_**: This task is required to have at least one parent but its\nfunctor takes exactly one future, namely the future of the parent that\nfirst finishes. All other parents are abandoned and canceled. Here's an example:\n```cpp\nauto task = tw::make_task(tw::accept_any, [](auto f1) { return f1.get(); }, parent1, parent2);\n```\nNote that canceling only works for already running tasks when the functor is \nsub-classed from `transwarp::functor`.\n\n**_consume_**: This task follows the same rules as `accept` with the difference\nthat the resulting parent futures are unwrapped (have `get()` called on them).\nThe results are then passed to the child, hence, consumed by the child task.\nThe child task will not be invoked if any parent throws an exception.\nFor example:\n```cpp\nauto task = tw::make_task(tw::consume, [](int x, int y) { return x + y; }, parent1, parent2);\n```\n\n**_consume_any_**: This task follows the same rules as `accept_any` with the difference\nthat the resulting parent futures are unwrapped (have `get()` called on them).\nFor example:\n```cpp\nauto task = tw::make_task(tw::consume_any, [](int x) { return x; }, parent1, parent2);\n``` \n\n**_wait_**: This task's functor does not take any parameters but the task\nmust have at least one parent. It simply waits for completion of all parents\nwhile unwrapping futures before calling the child's functor. For example:\n```cpp\nauto task = tw::make_task(tw::wait, []{ return 42; }, parent1, parent2);\n``` \n\n**_wait_any_**: This task works similar to the `wait` task but calls its functor\nas soon as the first parent completes. It abandons and cancels all remaining\nparent tasks. For example:\n```cpp\nauto task = tw::make_task(tw::wait_any, []{ return 42; }, parent1, parent2);\n``` \n\nGenerally, tasks are created using `make_task` which allows for any number \nof parents. However, it is a common use case for a child to only have one parent.\nFor this, `then()` can be directly called on the parent object to create a _continuation_:\n```cpp\nauto child = tw::make_task(tw::root, []{ return 42; })-\u003ethen(tw::consume, functor);\n```\n`child` is now a single-parent task whose functor consumes an integer.\n\n### Scheduling tasks\n\nOnce a task is created it can be scheduled just by itself:\n```cpp\nauto task = tw::make_task(tw::root, functor);\ntask-\u003eschedule();\n```\nwhich, if nothing else is specified, will run the task on the current thread. \nHowever, using the built-in `parallel` executor the task can be pushed into a \nthread pool and executed asynchronously:\n```cpp\ntw::parallel executor{4};  // Thread pool with 4 threads\nauto task = tw::make_task(tw::root, functor);\ntask-\u003eschedule(executor);\n```\nRegardless of how you schedule, the task result can be retrieved through:\n```cpp\nstd::cout \u003c\u003c task-\u003eget() \u003c\u003c std::endl;\n```  \nWhen chaining multiple tasks together a directed acyclic graph is built in which\nevery task can be scheduled individually. Though, in many scenarios it is useful\nto compute all tasks in the right order with a single call:\n```cpp\nauto parent1 = tw::make_task(tw::root, foo);  // foo is a functor\nauto parent2 = tw::make_task(tw::root, bar);  // bar is a functor\nauto task = tw::make_task(tw::consume, functor, parent1, parent2);\ntask-\u003eschedule_all();  // Schedules all parents and itself\n```\nwhich can also be scheduled using an executor, for instance:\n```cpp\ntw::parallel executor{4};\ntask-\u003eschedule_all(executor);\n```\nwhich will run those tasks in parallel that do not depend on each other.\n\n### Executors\n\nWe have seen that we can pass executors to `schedule()` and `schedule_all()`.\nAdditionally, they can be assigned to a task directly:\n```cpp\nauto exec1 = std::make_shared\u003ctw::parallel\u003e(2);\ntask-\u003eset_executor(exec1);\ntw::sequential exec2;\ntask-\u003eschedule(exec2);  // exec1 will be used to schedule the task\n``` \nThe task-specific executor will always be preferred over other executors when\nscheduling tasks.\n\ntranswarp defines an executor interface which can be implemented to perform custom \nbehavior when scheduling tasks. The interface looks like this:\n```cpp\nclass executor {\npublic:\n    virtual ~executor() = default;\n    \n    // The name of the executor\n    virtual std::string name() const = 0;\n    \n    // Only ever called on the thread of the caller to schedule()\n    virtual void execute(const std::function\u003cvoid()\u003e\u0026 functor, tw::itask\u0026 task) = 0;\n};\n\n``` \nwhere `functor` denotes the function to be run and `task` the task the functor belongs to.\n\n### Range functions\n\nThere are convenience functions that can be applied to an iterator range:\n* `tw::for_each`\n* `tw::transform`\n\nThese are very similar to their standard library counterparts except that they \nreturn a task for deferred, possibly asynchronous execution. Here's an example:\n\n```cpp\nstd::vector\u003cint\u003e vec = {1, 2, 3, 4, 5, 6, 7};\ntw::parallel exec{4};\nauto task = tw::for_each(exec, vec.begin(), vec.end(), [](int\u0026 x){ x *= 2; });\ntask-\u003ewait();  // all values in vec will have doubled\n```\n\n### Canceling tasks\n\nA task can be canceled by calling `task-\u003ecancel(true)` which will, by default, \nonly affect tasks that are not currently running yet. However, if you create a functor\nthat inherits from `transwarp::functor` you can terminate tasks while they're\nrunning. `transwarp::functor` looks like this:\n```cpp\n\nclass functor {\npublic:\n    virtual ~functor() = default;\n\nprotected:\n    // The associated task (only to be called after the task was constructed)\n    const tw::itask\u0026 transwarp_task() const noexcept;\n\n    // The associated task (only to be called after the task was constructed)\n    tw::itask\u0026 transwarp_task() noexcept;\n\n    // If the associated task is canceled then this will throw transwarp::task_canceled\n    // which will stop the task while it's running (only to be called after the task was constructed)\n    void transwarp_cancel_point() const;\n\nprivate:\n    ...\n};\n```\nBy placing calls to `transwarp_cancel_point()` in strategic places of your functor\nyou can denote well defined points where the functor will exit when the associated task is canceled.\nA task can also be canceled by throwing `transwarp::task_canceled` directly.\n\nAs mentioned above, tasks can be explicitly canceled on client request. In addition,\nall tasks considered abandoned by `accept_any`, `consume_any`, or `wait_any`\noperations are also canceled in order to terminate them as soon as their computations\nbecome superfluous. \n\n### Event system\n\nTranswarp provides an event system that allows you to subscribe to all or specific\nevents of a task, such as, before started or after finished events. The task events\nare enumerated in the `event_type` enum:\n```cpp\nenum class event_type {\n    before_scheduled, // Just before a task is scheduled\n    after_future_changed, // Just after the task's future was changed\n    before_started, // Just before a task starts running\n    before_invoked, // Just before a task's functor is invoked\n    after_finished, // Just after a task has finished running\n    after_canceled, // Just after a task was canceled\n    after_satisfied, ///\u003c Just after a task has satisfied all its children with results\n    after_custom_data_set, // Just after custom data was assigned\n}\n```\nListeners are created by sub-classing from the `listener` interface:\n```cpp\nclass listener {\npublic:\n    virtual ~listener() = default;\n\n    // This may be called from arbitrary threads depending on the event type\n    virtual void handle_event(tw::event_type event, tw::itask\u0026 task) = 0;\n};\n```\nA listener can then be passed to the `add_listener` functions of a task\nto add a new listener or to the `remove_listener` functions to remove\nan existing listener.\n\n### Task pool\n\nA task pool is useful when one wants to run the same graph in parallel. For this purpose,\ntranswarp provides a `task_pool` which manages a pool of tasks from which\none can request an idle task for parallel graph execution. For example:\n```cpp\ntw::parallel exec{4};\n\nauto my_task = make_graph();\ntw::task_pool\u003cdouble\u003e pool{my_task};\n\nfor (;;) {\n    auto task = pool.next_task(); // task may be null if the pool size is exhausted\n    if (task) {\n        task-\u003eschedule_all(exec);\n    }\n}\n```\n\n### Timing tasks\n\nIn order to identify bottlenecks it's often useful to know how much time is spent\nin which task. transwarp provides a `timer` listener that will automatically\ntime the tasks it listens to:\n```cpp\nauto task = make_graph();\ntask-\u003eadd_listener_all(std::make_shared\u003ctw::timer\u003e()); // assigns the timer listener to all tasks\ntask-\u003eschedule_all();\nstd::ofstream{\"graph.dot\"} \u003c\u003c tw::to_string(task-\u003eedges()); // the dot file now contains timing info\n```\n\n### Optimizing efficiency\n\n**Compile time switches**\n\nBy default, transwarp provides its full functionality to its client. However,\nin many cases not all of that is actually required and so transwarp provides\na few compile time switches to reduce the task size.\nThese switches are:\n```\nTRANSWARP_DISABLE_TASK_CUSTOM_DATA\nTRANSWARP_DISABLE_TASK_NAME\nTRANSWARP_DISABLE_TASK_PRIORITY\nTRANSWARP_DISABLE_TASK_REFCOUNT\nTRANSWARP_DISABLE_TASK_TIME\n```\n\nTo get the minimal task size with a single switch one can define\n```\nTRANSWARP_MINIMUM_TASK_SIZE\n```\nat build time.\n\n**Releasing unused memory**\n\nBy default, every task in a graph will keep its result until rescheduling or\na manual task reset. The `releaser` listener allows you to automatically\nrelease a task result after that task's children have consumed the result.\nFor example:\n```cpp\nauto task = make_graph();\ntask-\u003eadd_listener_all(std::make_shared\u003ctw::releaser\u003e()); // assigns the releaser listener to all tasks\ntask-\u003eschedule_all();\n// All intermediate task results are now released (i.e. futures are invalid)\nauto result = task-\u003eget(); // The final task's result remains valid\n```\nThe `releaser` also accepts an executor that gives control over _where_ a task's\nresult is released.\n\n## Using transwarp with tipi.build\n\n`transwarp` can be easily used in [tipi.build](https://tipi.build) projects simply by adding the following entry to your `.tipi/deps`:\n\n```json\n{\n    \"bloomen/transwarp\": { }\n}\n```\n\n## Feedback\n\nGet in touch if you have any questions or suggestions to make this a better library!\nYou can post on [gitter](https://gitter.im/bloomen/transwarp), submit a pull request,\ncreate a Github issue, or simply email one of the contributors.\n\nIf you're serious about contributing code to transwarp (which would be awesome!) then \nplease submit a pull request and keep in mind that:\n- unit tests should be added for all new code by extending the existing unit test suite\n- C++ code uses spaces throughout \n","funding_links":[],"categories":["Concurrency","Software"],"sub_categories":["Trends"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbloomen%2Ftranswarp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbloomen%2Ftranswarp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbloomen%2Ftranswarp/lists"}