{"id":17318057,"url":"https://github.com/yukunj/zorro","last_synced_at":"2026-03-10T07:04:22.101Z","repository":{"id":156607662,"uuid":"621481567","full_name":"YukunJ/Zorro","owner":"YukunJ","description":"This is Zorro, the implementation of a distributed threadpool with work stealing enabled","archived":false,"fork":false,"pushed_at":"2023-05-05T01:23:48.000Z","size":4415,"stargazers_count":6,"open_issues_count":2,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-07T09:39:57.155Z","etag":null,"topics":["cpp17","distributed","shared-memory","threadpool","work-stealing-algorithm"],"latest_commit_sha":null,"homepage":"https://github.com/YukunJ/Zorro","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/YukunJ.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-30T18:49:16.000Z","updated_at":"2024-03-09T16:36:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"7ccf1c4f-363f-422b-a556-59e731269012","html_url":"https://github.com/YukunJ/Zorro","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/YukunJ/Zorro","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YukunJ%2FZorro","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YukunJ%2FZorro/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YukunJ%2FZorro/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YukunJ%2FZorro/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/YukunJ","download_url":"https://codeload.github.com/YukunJ/Zorro/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YukunJ%2FZorro/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30326893,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T05:25:20.737Z","status":"ssl_error","status_checked_at":"2026-03-10T05:25:17.430Z","response_time":106,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp17","distributed","shared-memory","threadpool","work-stealing-algorithm"],"created_at":"2024-10-15T13:18:42.742Z","updated_at":"2026-03-10T07:04:22.093Z","avatar_url":"https://github.com/YukunJ.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"img/zorro.png\" alt=\"Zorro Logo\" height=\"250\"\u003e\n\n\n-----------------\n\u003ca href=\"https://github.com/YukunJ/Zorro/blob/main/LICENSE\"\u003e\u003cimg src=\"https://badgen.net/github/license/YukunJ/Zorro?color=yellow\" alt=\"license\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/YukunJ/Zorro\"\u003e\u003cimg src=\"https://img.shields.io/badge/Language-C++-red.svg\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/YukunJ/Zorro\"\u003e\u003cimg src=\"https://badgen.net/badge/OS Support/Linux/cyan?list=1\" alt=\"os\"\u003e\u003c/a\u003e\n\n\n### Zorro\nThis is Zorro, the implementation of a distributed threadpool with work stealing enabled. It builds on Linux OS and use C++17 as the programming language.\n\nIt aims to separate parallel task definition and execution. This should ease users' experience of executing multiple tasks in parallel by automatically managing the scheduling and work balancing while squeezing all the parallel performance out of the underlying hardware.\n\n##### Authors\nYukun Jiang (yukunj) \u0026 Leo Guo (jiongtig)\n\n---\n\n#### Key Challenge\n\nFirstly, load balancing is hard. Without knowing the exact workload of each task, the workload balancing has to been dynamically on the fly. This is why we need the work-stealing policy.\n\nSecondly, with a distributed work queue, synchronization is challenging. The more workers in the threadpool, the higher the contention for critical data structure.\n\n--- \n\n#### System Diagram\n\nThe final version, which is thread-local fine-grained locking queue with work stealing policy enabled, is illustrated in the diagram below for reference.\n\n\u003cimg src=\"img/system.png\" alt=\"System Architecture\" height=\"450\"\u003e\n\n---\n\n#### Performance Benchmark\nWe benchmark our systems on PSC super computer cluster ranging thread from 2, 4, 8, ... to 128, and on various workload.\n\n\u003cimg src=\"img/light_task.png\" alt=\"System Architecture\" height=\"250\"\u003e\n\u003cimg src=\"img/computation_task.png\" alt=\"System Architecture\" height=\"250\"\u003e\n\u003cimg src=\"img/imbalanced_task.png\" alt=\"System Architecture\" height=\"250\"\u003e\n\u003cimg src=\"img/quick_task.png\" alt=\"System Architecture\" height=\"250\"\u003e\n\u003cimg src=\"img/merge_task.png\" alt=\"System Architecture\" height=\"250\"\u003e\n\n\n---\n\n#### Schedule\n\n+ [x] **Week1**: Read related literature and setup repo build with base interface\n+ [x] **Week2**: Build the global work queue version\n+ [x] **Week3**: Build the distributed thread-local work queue version\n+ [x] **Week4**: Enable work-stealing policy to enhance work balance\n+ [x] **Week5**: Wrap up the project and prepare report \u0026 poster\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyukunj%2Fzorro","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyukunj%2Fzorro","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyukunj%2Fzorro/lists"}