{"id":13436513,"url":"https://github.com/CodingHanYa/workspace","last_synced_at":"2025-03-18T21:30:37.778Z","repository":{"id":65437914,"uuid":"590075665","full_name":"CodingHanYa/workspace","owner":"CodingHanYa","description":"workspace是基于C++11的轻量级异步执行框架，支持：通用任务异步并发执行、优先级任务调度、自适应动态线程池、高效静态线程池、异常处理机制等。","archived":false,"fork":false,"pushed_at":"2024-10-24T08:25:42.000Z","size":3970,"stargazers_count":1104,"open_issues_count":17,"forks_count":167,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-03-16T07:20:34.057Z","etag":null,"topics":["concise","concurrency","cpp11","easy-to-use","framework","high-performance","sfinae","stable","threadpool"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CodingHanYa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-17T15:53:22.000Z","updated_at":"2025-03-15T08:07:54.000Z","dependencies_parsed_at":"2024-09-29T00:40:37.513Z","dependency_job_id":"67d11405-ee8d-457a-9873-915ba70c42c0","html_url":"https://github.com/CodingHanYa/workspace","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodingHanYa%2Fworkspace","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodingHanYa%2Fworkspace/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodingHanYa%2Fworkspace/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodingHanYa%2Fworkspace/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CodingHanYa","download_url":"https://codeload.github.com/CodingHanYa/workspace/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244310344,"owners_count":20432521,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concise","concurrency","cpp11","easy-to-use","framework","high-performance","sfinae","stable","threadpool"],"created_at":"2024-07-31T03:00:49.387Z","updated_at":"2025-03-18T21:30:37.034Z","avatar_url":"https://github.com/CodingHanYa.png","language":"C++","readme":"# workspace\n\n**workspace**是基于C++11的轻量级异步执行框架，支持：通用任务异步执行、优先级任务调度、自适应动态线程池、高效静态线程池、异常处理机制等。\n\n## 目录\n\n- [特点](#特点)\n- [模块简介](#主要模块)\n\t- [workbranch](#workbranch)\n\t- [supervisor](#supervisor)\n\t- [workspace](#workspace)\n- [辅助模块](#辅助模块)\n    - [futures](#futures)   \n- [benchmark](#benchmark)\n- [如何使用](#如何使用)\n- [注意事项](#注意事项)\n- [其它](#其它)\n\n## 特点\n\n- 轻量的：Header-Only \u0026 代码量 \u003c= 1000行 \u0026 接口简单。\n- 高效的：超轻量级任务支持异步顺序执行，提高了框架的并发性能。\n- 灵活的：支持多种任务类型、动态线程调整、可通过workspace构建不同的池模型。\n- 稳定的：利用`std::function`的小任务优化减少内存碎片、拥有良好的异步线程异常处理机制。\n- 兼容性：纯C++11实现，跨平台，且兼容C++11以上版本。\n\n\n## 主要模块\n\n### **workbranch**\n\n**workbranch**（工作分支）是动态线程池的抽象，内置了一条线程安全的**任务队列**用于同步任务。其管理的每一条异步工作线程被称为**worker**，负责从任务队列不断获取任务并执行。（以下示例按顺序置于`workspace/example/`）\n\u003cbr\u003e\n\n让我们先简单地提交一点任务，当你的任务带有返回值时，workbranch会返回一个std::future，否则返回void。\n\n```c++\n#include \u003cworkspace/workspace.hpp\u003e\n\nint main() {\n    // 2 threads\n    wsp::workbranch br(2);\n    // return void\n    br.submit([]{ std::cout\u003c\u003c\"hello world\"\u003c\u003cstd::endl; });  \n    // return std::future\u003cint\u003e\n    auto result = br.submit([]{ return 2023; });  \n    std::cout\u003c\u003c\"Got \"\u003c\u003cresult.get()\u003c\u003cstd::endl;   \n    // wait for tasks done (timeout: 1000 milliseconds)\n    br.wait_tasks(1000); \n}\n```\n\n由于返回一个std::future会带来一定的开销，如果你不需要返回值并且希望程序跑得更快，那么你的任务应该是`void()`类型的。\n\u003cbr\u003e\n\n当你有一个任务并且你希望它能尽快被执行时，你可以指定该任务的类型为`urgent`，如下：\n\n```C++\n#include \u003cworkspace/workspace.hpp\u003e\n\nint main() {\n    // 1 threads\n    wsp::workbranch br;\n    br.submit\u003cwsp::task::nor\u003e([]{ std::cout\u003c\u003c\"task B done\\n\";}); // normal task \n    br.submit\u003cwsp::task::urg\u003e([]{ std::cout\u003c\u003c\"task A done\\n\";}); // urgent task\n    br.wait_tasks(); // wait for tasks done (timeout: no limit)\n}\n```\n在这里我们通过指定任务类型为`wsp::task::urg`，来提高任务的优先级。最终\n在我的机器上：\n\n```shell\njack@xxx:~/workspace/example/build$ ./e2\ntask A done\ntask B done\n```\n在这里我们不能保证`task A`一定会被先执行，因为当我们提交`task A`的时候，`task B`可能已经在执行中了。`urgent`标签可以让任务被插入到队列头部，但无法改变已经在执行的任务。\n\u003cbr\u003e\n\n假如你有几个轻量异步任务，执行他们只需要**非常短暂**的时间。同时，按照**顺序执行**它们对你来说没有影响，甚至正中你下怀。那么你可以把任务类型指定为`sequence`，以便提交一个**任务序列**。这个任务序列会被单个线程顺序执行：\n\n```c++\n#include \u003cworkspace/workspace.hpp\u003e\n\nint main() {\n    wsp::workbranch br;\n    // sequence tasks\n    br.submit\u003cwsp::task::seq\u003e([]{std::cout\u003c\u003c\"task 1 done\\n\";},\n                              []{std::cout\u003c\u003c\"task 2 done\\n\";},\n                              []{std::cout\u003c\u003c\"task 3 done\\n\";},\n                              []{std::cout\u003c\u003c\"task 4 done\\n\";});\n    // wait for tasks done (timeout: no limit)\n    br.wait_tasks();\n}\n```\n任务序列会被打包成一个较大的任务，以此来减轻框架同步任务的负担，提高整体的并发性能。\n\u003cbr\u003e\n\n当任务中抛出了一个异常，workbranch有两种处理方式：A-将其捕获并输出到终端 B-将其捕获并通过std::future传递到主线程。第二种需要你提交一个**带返回值**的任务。\n```C++\n#include \u003cworkspace/workspace.hpp\u003e\n// self-defined\nclass excep: public std::exception {\n    const char* err;\npublic:\n    excep(const char* err): err(err) {}\n    const char* what() const noexcept override {\n        return err;\n    }\n}; \nint main() {\n    wsp::workbranch wbr;\n    wbr.submit([]{ throw std::logic_error(\"A logic error\"); });     // log error\n    wbr.submit([]{ throw std::runtime_error(\"A runtime error\"); }); // log error\n    wbr.submit([]{ throw excep(\"XXXX\");});                          // log error\n\n    auto future1 =  wbr.submit([]{ throw std::bad_alloc(); return 1; }); // catch error\n    auto future2 =  wbr.submit([]{ throw excep(\"YYYY\"); return 2; });    // catch error\n    try {\n        future1.get();\n    } catch (std::exception\u0026 e) {\n        std::cerr\u003c\u003c\"Caught error: \"\u003c\u003ce.what()\u003c\u003cstd::endl;\n    }\n    try {\n        future2.get();\n    } catch (std::exception\u0026 e) {\n        std::cerr\u003c\u003c\"Caught error: \"\u003c\u003ce.what()\u003c\u003cstd::endl;\n    }\n}\n```\n在我的机器上：\n```\njack@xxx:~/workspace/test/build$ ./test_exception \nworkspace: worker[140509071521536] caught exception:\n  what(): A logic error\nworkspace: worker[140509071521536] caught exception:\n  what(): A runtime error\nworkspace: worker[140509071521536] caught exception:\n  what(): XXXX\nCaught error: std::bad_alloc\nCaught error: YYYY\n```\n\n\n---\n\n### **supervisor**\n\nsupervisor是异步管理者线程的抽象，负责监控workbranch的负载情况并进行动态调整。它允许你在每一次调控workbranch之后执行一个小任务，你可以用来**写日志**或者做一些其它调控等。\n\u003cbr\u003e\n\n每一个supervisor可以管理多个workbranch。此时workbranch之间共享supervisor的所有设定。\n\n```c++\n#include \u003cworkspace/workspace.hpp\u003e\n\nint main() {\n    wsp::workbranch br1(2);\n    wsp::workbranch br2(2);\n\n    // 2 \u003c= thread number \u003c= 4 \n    // time interval: 1000 ms \n    wsp::supervisor sp(2, 4, 1000);\n\n    sp.set_tick_cb([\u0026br1, \u0026br2]{\n        auto now = std::chrono::system_clock::now();\n        std::time_t timestamp = std::chrono::system_clock::to_time_t(now);\n        std::tm local_time = *std::localtime(\u0026timestamp);\n        static char buffer[40];\n        std::strftime(buffer, sizeof(buffer), \"%Y-%m-%d %H:%M:%S\", \u0026local_time);\n        std::cout\u003c\u003c\"[\"\u003c\u003cbuffer\u003c\u003c\"] \"\u003c\u003c\"br1: [workers] \"\u003c\u003cbr1.num_workers()\u003c\u003c\" | [blocking-tasks] \"\u003c\u003cbr1.num_tasks()\u003c\u003c'\\n';\n        std::cout\u003c\u003c\"[\"\u003c\u003cbuffer\u003c\u003c\"] \"\u003c\u003c\"br2: [workers] \"\u003c\u003cbr2.num_workers()\u003c\u003c\" | [blocking-tasks] \"\u003c\u003cbr2.num_tasks()\u003c\u003c'\\n';\n    });\n\n    sp.supervise(br1);  // start supervising\n    sp.supervise(br2);  // start supervising\n\n    for (int i = 0; i \u003c 1000; ++i) {\n        br1.submit([]{std::this_thread::sleep_for(std::chrono::milliseconds(10));});\n        br2.submit([]{std::this_thread::sleep_for(std::chrono::milliseconds(20));});\n    }\n\n    br1.wait_tasks();\n    br2.wait_tasks();\n}\n```\n\n在我的机器上，输出如下：\n\n```\njack@xxx:~/workspace/example/build$ ./e4\n[2023-06-13 12:24:31] br1: [workers] 4 | [blocking-tasks] 606\n[2023-06-13 12:24:31] br2: [workers] 4 | [blocking-tasks] 800\n[2023-06-13 12:24:32] br1: [workers] 4 | [blocking-tasks] 213\n[2023-06-13 12:24:32] br2: [workers] 4 | [blocking-tasks] 600\n[2023-06-13 12:24:33] br1: [workers] 4 | [blocking-tasks] 0\n[2023-06-13 12:24:33] br2: [workers] 4 | [blocking-tasks] 404\n[2023-06-13 12:24:34] br1: [workers] 3 | [blocking-tasks] 0\n[2023-06-13 12:24:34] br2: [workers] 4 | [blocking-tasks] 204\n[2023-06-13 12:24:35] br1: [workers] 2 | [blocking-tasks] 0\n[2023-06-13 12:24:35] br2: [workers] 4 | [blocking-tasks] 4\n[2023-06-13 12:24:35] br1: [workers] 2 | [blocking-tasks] 0\n[2023-06-13 12:24:35] br2: [workers] 4 | [blocking-tasks] 0\n```\n---\n\n### **workspace**\n\nworkspace是一个**托管器**/**任务分发器**，你可以将workbranch和supervisor托管给它，并用workspace分配的**组件专属ID**来访问它们。将组件托管至workspace至少有以下几点好处：\n\n- 堆内存正确释放：workspace在内部用unique指针来管理组件，确保没有内存泄漏\n- 分支间任务负载均衡：workspace支持任务分发，在workbranch之间实现了简单高效的**负载均衡**。\n- 避免空悬指针问题：当workbranch先于supervisor析构会造成**空悬指针**的问题，使用workspace可以避免这种情况\n- 更低的框架开销：workspace的任务分发机制能减少与工作线程的竞争，提高性能（见下Benchmark）。\n\n我们可以通过workspace自带的任务分发机制来异步执行任务（调用`submit`）。\n\n```C++\n#include \u003cworkspace/workspace.hpp\u003e\n\nint main() {\n    wsp::workspace spc;\n    auto bid1 = spc.attach(new wsp::workbranch);\n    auto bid2 = spc.attach(new wsp::workbranch);\n    auto sid1 = spc.attach(new wsp::supervisor(2, 4));\n    auto sid2 = spc.attach(new wsp::supervisor(2, 4));\n    spc[sid1].supervise(spc[bid1]);  // start supervising\n    spc[sid2].supervise(spc[bid2]);  // start supervising\n\n    // Automatic assignment\n    spc.submit([]{std::cout\u003c\u003cstd::this_thread::get_id()\u003c\u003c\" executed task\"\u003c\u003cstd::endl;});\n    spc.submit([]{std::cout\u003c\u003cstd::this_thread::get_id()\u003c\u003c\" executed task\"\u003c\u003cstd::endl;});\n\n    spc.for_each([](wsp::workbranch\u0026 each){each.wait_tasks();});\n}\n```\n\n当我们需要等待任务执行完毕的时候，我们可以调用`for_each`+`wait_tasks`，并为每一个workbranch指定等待时间，单位是毫秒。\n\n（更多详细接口见`workspace/test/`）\n\n## 辅助模块\n### futures \nwsp::futures是一个std::future收集器(collector)，可以缓存同类型的std::future，并进行批量操作。一个简单的操作如下:\n```C++\n#include \u003cworkspace/workspace.hpp\u003e\n\nint main() {\n    wsp::futures\u003cint\u003e futures;\n    wsp::workspace spc;\n    spc.attach(new wsp::workbranch(\"br\", 2));\n    \n    futures.add_back(spc.submit([]{return 1;}));\n    futures.add_back(spc.submit([]{return 2;}));\n\n    futures.wait();\n    auto res = futures.get();\n    for (auto\u0026 each: res) {\n        std::cout\u003c\u003c\"got \"\u003c\u003ceach\u003c\u003cstd::endl;\n    }\n}\n```\n这里`futures.get()`返回的是一个`std::vector\u003cint\u003e`，里面保存了所有任务的返回值。\n\n\n## benchmark\n\n### 空跑测试\n\n测试原理：通过快速提交大量的空任务以考察框架同步任务的开销。\u003cbr\u003e\n测试环境：Ubuntu20.04 : 16核 : AMD Ryzen 7 5800H with Radeon Graphics 3.20 GHz\n\n\u003c**测试1**\u003e\u003cbr\u003e 在测试1中我们调用了`submit\u003cwsp::task::seq\u003e`，每次打包10个空任务并提交到**workbranch**中执行。结果如下：（代码见`workspace/benchmark/bench1.cc`）\n\n```\nthreads: 1  |  tasks: 100000000  |  time-cost: 2.68801 (s)\nthreads: 2  |  tasks: 100000000  |  time-cost: 3.53964 (s)\nthreads: 3  |  tasks: 100000000  |  time-cost: 3.99903 (s)\nthreads: 4  |  tasks: 100000000  |  time-cost: 5.26045 (s)\nthreads: 5  |  tasks: 100000000  |  time-cost: 6.65157 (s)\nthreads: 6  |  tasks: 100000000  |  time-cost: 8.40907 (s)\nthreads: 7  |  tasks: 100000000  |  time-cost: 10.5967 (s)\nthreads: 8  |  tasks: 100000000  |  time-cost: 13.2523 (s)\n```\n\n\u003c**测试2**\u003e\u003cbr\u003e 在测试2中我们同样将10个任务打成一包，但是是将任务提交到**workspace**中，利用workspace进行任务分发，且在workspace托管的workbranch只拥有 **1条** 线程。结果如下：（代码见`workspace/benchmark/bench2.cc`）\n\n```\nthreads: 1  |  tasks: 100000000  |  time-cost: 4.38221 (s)\nthreads: 2  |  tasks: 100000000  |  time-cost: 4.01103 (s)\nthreads: 3  |  tasks: 100000000  |  time-cost: 3.6797 (s)\nthreads: 4  |  tasks: 100000000  |  time-cost: 3.39314 (s)\nthreads: 5  |  tasks: 100000000  |  time-cost: 3.03324 (s)\nthreads: 6  |  tasks: 100000000  |  time-cost: 3.16079 (s)\nthreads: 7  |  tasks: 100000000  |  time-cost: 3.04612 (s)\nthreads: 8  |  tasks: 100000000  |  time-cost: 3.11893 (s)\n```\n\n\u003c**测试3**\u003e\u003cbr\u003e 在测试3中我们同样将10个任务打成一包，并且将任务提交到**workspace**中，但是workspace管理的每个**workbranch**中都拥有 **2条** 线程。结果如下：（代码见`workspace/benchmark/bench3.cc`）\n\n```\nthreads: 2  |  tasks: 100000000  |  time-cost: 4.53911 (s)\nthreads: 4  |  tasks: 100000000  |  time-cost: 7.0178 (s)\nthreads: 6  |  tasks: 100000000  |  time-cost: 6.00101 (s)\nthreads: 8  |  tasks: 100000000  |  time-cost: 5.97501 (s)\nthreads: 10 |  tasks: 100000000  |  time-cost: 5.63834 (s)\nthreads: 12 |  tasks: 100000000  |  time-cost: 5.17316 (s)\n```\n\n**总结**：利用workspace进行任务分发，且**workbranch**线程数为1的情况下，整个任务同步框架是静态的，任务同步开销最小。当**workbranch**内的线程数越多，面对大量空任务时对任务队列的竞争越激烈，框架开销越大。（更加详尽的测试结果见`bench.md`，测试代码于`workspace/bench`）\n\n\n## 如何使用\n\n#### 生成doxygen文档\n请提前安装doxygen\n``` shell\n# 与workspace同级目录中（Linux）\ndoxygen ./doxygen.conf\n```\n生成的文档在`workspace/docs/`中，可以在浏览器中打开`workspace/docs/html/index.html`并查看接口。\n\n\n#### 简单使用\n\n```shell\n# 项目代码与workspace同级（Linux）\ng++ -I workspace/include xxx.cc -lpthread \u0026\u0026 ./a.out\n```\n其它平台可能需要链接不同的线程库，且可执行文件后缀不同。\n\n#### 运行已有实例（以example为例）\n\n```shell\n# 在\"workspace/example\"中\ncmake -B build \ncd build\nmake\n./e1\n```\n\n#### 安装到系统（支持Win/Linux/Mac）\n```shell\n# 在\"workspace/\"中\ncmake -B build \ncd build\nsudo make install\n```\n\n\n## 注意事项\n\n#### 雷区\n1. 不要在任务中操纵组件，如：`submit([\u0026br]{br.wait_tasks();});` 会阻塞线程 \u003cbr\u003e\n2. 不要在回调中操纵组件，如：`set_tick_cb([\u0026sp]{sp.suspend();});` \u003cbr\u003e\n3. 不要让workbranch先于supervisor析构（空悬指针问题）。\n\n#### 接口安全性\n\n|组件接口|是否线程安全|\n| :-- | :--: |\n|workspace|否|\n|workbranch|是|\n|supervisor|是|\n|futures|否|\n\n#### 时间单位\nworkspace有关时间的接口单位都是 -\u003e 毫秒（ms）\n\n## 其它\n#### 参考书目\n《C++并发编程》\n\n#### 联系我\n邮箱: 1848395727@qq.com\n","funding_links":[],"categories":["C++"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCodingHanYa%2Fworkspace","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FCodingHanYa%2Fworkspace","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCodingHanYa%2Fworkspace/lists"}