{"id":30288398,"url":"https://github.com/li-giegie/workerpool","last_synced_at":"2025-08-16T22:36:40.782Z","repository":{"id":308686199,"uuid":"1033664844","full_name":"Li-giegie/workerpool","owner":"Li-giegie","description":"一个灵活高效的 Go 任务调度池","archived":false,"fork":false,"pushed_at":"2025-08-07T08:25:36.000Z","size":13,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-07T10:15:45.781Z","etag":null,"topics":["go","goroutine-pool"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Li-giegie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-07T06:56:19.000Z","updated_at":"2025-08-07T08:25:39.000Z","dependencies_parsed_at":"2025-08-07T10:17:06.185Z","dependency_job_id":"9ba9cb29-4bee-4d6f-a1ce-923e997a45ea","html_url":"https://github.com/Li-giegie/workerpool","commit_stats":null,"previous_names":["li-giegie/workerpool"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Li-giegie/workerpool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Li-giegie%2Fworkerpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Li-giegie%2Fworkerpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Li-giegie%2Fworkerpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Li-giegie%2Fworkerpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Li-giegie","download_url":"https://codeload.github.com/Li-giegie/workerpool/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Li-giegie%2Fworkerpool/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270781211,"owners_count":24643807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-16T02:00:11.002Z","response_time":91,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","goroutine-pool"],"created_at":"2025-08-16T22:36:34.333Z","updated_at":"2025-08-16T22:36:40.767Z","avatar_url":"https://github.com/Li-giegie.png","language":"Go","readme":"# workerpool\r\n\r\nworkerpool 一个灵活高效的 Go 任务调度池\r\n\r\n## 什么是 workerpool？\r\nworkerpool 是一个轻量级的 Go 任务调度池实现，它允许你：\r\n- 控制并发执行的工作协程数量\r\n- 灵活处理不同签名的函数和参数\r\n- 通过事件机制监控任务生命周期\r\n- 管理任务队列和等待队列，避免资源耗尽\r\n## 快速开始\r\n使用 go get 命令快速安装：\r\n```go\r\ngo get -u  github.com/Li-giegie/workerpool\r\n```\r\n\r\n## 基础用法\r\n```go\r\npackage main\r\n\r\nimport (\r\n\t\"fmt\"\r\n\t\"time\"\r\n\t\"github.com/Li-giegie/workerpool\"\r\n)\r\n\r\nfunc main() {\r\n\t// 创建工作池：1个工作协程，任务队列容量12\r\n\twp := workerpool.NewWorkerPool(1, 12)\r\n\r\n\t// 异步推送任务\r\n\tgo func() {\r\n\t\ttime.Sleep(time.Second)\r\n\t\twp.Push(func() {\r\n\t\t\tfmt.Println(\"任务执行中...\")\r\n\t\t\t// 任务完成后停止工作池\r\n\t\t\tgo wp.Stop()\r\n\t\t})\r\n\t}()\r\n\r\n\t// 启动工作池（阻塞）\r\n\tif err := wp.Run(); err != nil {\r\n\t\tfmt.Println(\"工作池启动失败:\", err)\r\n\t}\r\n}\r\n```\r\n\r\n## 核心功能详解\r\n\r\n### 1. 支持任意函数签名和参数\r\nworkerpool 最强大的特性之一是支持任意函数签名和参数，无需提前定义特定类型：\r\n\r\n```go\r\n// 带参数和返回值的任务示例\r\nfunc main() {\r\n    wp := workerpool.NewWorkerPool(1, 12)\r\n    \r\n    // 添加任务完成事件监听\r\n    wp.AddListener(workerpool.EventEndTask, func(e workerpool.Event, a ...any) {\r\n        task := a[0].(workerpool.Task)\r\n        fmt.Printf(\"任务ID: %v, 错误: %v, 结果: %v\\n\", \r\n            task.Id(), task.Err(), task.Out())\r\n    })\r\n    \r\n    go func() {\r\n        time.Sleep(time.Second)\r\n        // 推送带参数的函数\r\n        wp.Push(func(a, b int) int {\r\n            fmt.Println(\"执行加法任务\")\r\n            go wp.Stop()\r\n            return a + b\r\n        }, 10, 20) // 传递参数 10 和 20\r\n    }()\r\n    \r\n    wp.Run()\r\n}\r\n```\r\n### 2. 灵活的队列配置\r\n创建工作池时可以配置三个关键参数：\r\n- 创建工作池时可以配置三个关键参数：\r\n- 任务队列容量\r\n- 等待队列容量\r\n```go\r\n// 示例：2个工作协程，任务队列容量50，等待队列容量100\r\nwp := workerpool.NewWorkerPool(2, 50, 100)\r\n```\r\n队列机制说明：\r\n- 新任务优先放入任务队列（缓冲区）\r\n- 任务队列满时，放入等待队列\r\n- 等待队列也满时，返回 ErrOverflow 错误\r\n- 工作协程空闲时会自动从等待队列获取任务\r\n\r\n### 3. 事件监听机制\r\nworkerpool 提供了完整的事件机制，让你可以监控整个任务生命周期：\r\n```go\r\n// 支持的事件类型\r\nconst (\r\n    EventPushTask    // 任务被推送时\r\n    EventStartTask   // 任务开始执行时\r\n    EventEndTask     // 任务执行结束时\r\n    EventStartWorker // 工作协程启动时\r\n)\r\n\r\n// 示例：添加多个事件监听\r\nwp.AddListener(workerpool.EventStartTask, func(e workerpool.Event, a ...any) {\r\n    task := a[0].(workerpool.Task)\r\n    fmt.Printf(\"任务 %v 开始执行\\n\", task.Id())\r\n})\r\n\r\nwp.AddListener(workerpool.EventEndTask, func(e workerpool.Event, a ...any) {\r\n    task := a[0].(workerpool.Task)\r\n    fmt.Printf(\"任务 %v 执行结束\\n\", task.Id())\r\n}, true) // 最后一个参数为true表示异步执行回调\r\n```\r\n### 4. 工作池管理\r\nworkerpool 提供了丰富的管理方法：\r\n```go\r\n// 重启工作池\r\nerr := wp.Reboot()\r\n\r\n// 停止工作池\r\nwp.Stop()\r\n\r\n// 获取待执行任务数量\r\nwaitNum := wp.WaitTaskNum()\r\n\r\n// 获取可用任务队列容量\r\nfree := wp.Free()\r\n\r\n// 获取当前工作池状态\r\nstate := wp.State()\r\n```\r\n\r\n## 实现原理简析\r\n### 核心数据结构\r\nworkerpool 的核心是 workerPool 结构体，主要包含：\r\n```go\r\ntype workerPool struct {\r\n    numWorker    int           // 工作协程数量\r\n    numTaskQueue int           // 任务队列容量\r\n    numWaitQueue int           // 等待队列容量\r\n    lock         sync.Mutex    // 同步锁\r\n    wg           sync.WaitGroup // 用于等待所有工作协程完成\r\n    taskQueue    chan Task     // 任务队列（缓冲区）\r\n    waitQueue    []Task        // 等待队列\r\n    eventMap     sync.Map      // 事件处理器映射\r\n    state        WorkerPoolState // 工作池状态\r\n}\r\n```\r\n### 任务执行流程\r\n\r\n1. 调用 Push 方法添加任务时，会先尝试放入 taskQueue\r\n2. 若 taskQueue 已满，则放入 waitQueue（如果有容量）\r\n3. 工作协程从 taskQueue 中获取任务并执行\r\n4. 任务执行完成后，工作协程会检查 waitQueue，并将任务移至 taskQueue\r\n\r\n### 反射调用机制\r\nworkerpool 能支持任意函数签名，核心在于使用了 Go 的反射机制：\r\n```go\r\n// call 函数通过反射动态调用任意函数\r\nfunc call(fn any, args ...any) ([]any, error) {\r\n    rv := reflect.ValueOf(fn)\r\n    // ... 反射处理逻辑 ...\r\n    out = rv.Call(in) // 或 rv.CallSlice(in) 处理可变参数\r\n    // ... 结果转换 ...\r\n}\r\n```\r\n\r\n## 性能考量\r\n为了验证 workerpool 的性能，我们可以对比直接使用 goroutine 和使用工作池的性能差异：\r\n```go\r\n// 工作池性能测试\r\nfunc BenchmarkWorkerPool(b *testing.B) {\r\n    wp := NewWorkerPool(12, 12)\r\n    go wp.Run()\r\n    time.Sleep(time.Millisecond * 100)\r\n    b.ResetTimer()\r\n    for i := 0; i \u003c b.N; i++ {\r\n        err := wp.Push(func() {\r\n            _ = i\r\n        })\r\n        if err != nil {\r\n            b.Error(err)\r\n            return\r\n        }\r\n    }\r\n}\r\n\r\n// 直接使用goroutine的性能测试\r\nfunc BenchmarkGoroutine(b *testing.B) {\r\n    var wg sync.WaitGroup\r\n    b.ResetTimer()\r\n    for i := 0; i \u003c b.N; i++ {\r\n        wg.Add(1)\r\n        go func() {\r\n            defer wg.Done()\r\n            _ = i\r\n        }()\r\n    }\r\n    wg.Wait()\r\n}\r\n```\r\n在大量短任务场景下，workerpool 可以避免频繁创建和销毁 goroutine 的开销，从而提升性能。\r\n\r\n## 总结\r\nworkerpool 是一个简单而强大的任务调度池，它的主要优势包括：\r\n- 接口简洁易用，学习成本低\r\n- 支持任意函数签名和参数，灵活性高\r\n- 完善的事件机制，便于监控和扩展\r\n- 合理的队列管理，避免资源耗尽\r\n- 良好的性能表现，适合处理大量并发任务\r\n\r\n如果你正在寻找一个轻量级的 Go 任务调度解决方案，workerpool 值得一试。它的源码简洁明了，也适合作为学习 Go 并发编程的参考示例。","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fli-giegie%2Fworkerpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fli-giegie%2Fworkerpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fli-giegie%2Fworkerpool/lists"}