{"id":16511321,"url":"https://github.com/pbenner/threadpool","last_synced_at":"2025-07-17T22:34:37.029Z","repository":{"id":57497401,"uuid":"87962211","full_name":"pbenner/threadpool","owner":"pbenner","description":"Go Thread Pool","archived":false,"fork":false,"pushed_at":"2023-09-25T11:13:03.000Z","size":47,"stargazers_count":18,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-03T18:52:57.598Z","etag":null,"topics":["go","golang","golang-library","thread-pool"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pbenner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-11T17:37:37.000Z","updated_at":"2024-08-10T09:37:13.000Z","dependencies_parsed_at":"2022-08-28T19:50:38.956Z","dependency_job_id":"1e3cedd6-2a40-44ea-8bb9-13eb795219f3","html_url":"https://github.com/pbenner/threadpool","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pbenner/threadpool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbenner%2Fthreadpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbenner%2Fthreadpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbenner%2Fthreadpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbenner%2Fthreadpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pbenner","download_url":"https://codeload.github.com/pbenner/threadpool/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbenner%2Fthreadpool/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265672369,"owners_count":23808845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","golang-library","thread-pool"],"created_at":"2024-10-11T15:59:50.315Z","updated_at":"2025-07-17T22:34:37.013Z","avatar_url":"https://github.com/pbenner.png","language":"Go","readme":"## ThreadPool\n\nGo / Golang thread-pool library that supports nested job queuing. The general programming pattern is described in the following schematic:\n\n```go\n  // create a new thread pool with 5 working threads and\n  // a queue buffer of 100 (in addition to this thread, 4\n  // more threads will be launched that start reading\n  // from the job queue)\n  pool := threadpool.New(5, 100)\n\n  // allocate some memory for each thread\n  data := make([]ThreadData, pool.NumberOfThreads())\n\n  // jobs are always grouped, get a new group index\n  g1 := pool.NewJobGroup()\n\n  // add a new job to group g1, if there is only one thread in the pool\n  // (i.e. if pool == threadpool.ThreadPool{}), the main thread will process this\n  // job immediately\n  if err := pool.AddJob(g1, func(pool threadpool.ThreadPool, erf func() error) error {\n    // check if there was an error in one of the other tasks\n    if erf() != nil {\n      return nil\n    }\n    // notice that this function receives as argument another\n    // ThreadPool structure, which hides the old pool variable\n    // in the surrounding scope and carries the thread id of\n    // the current process\n\n    // access global data in a thread-safe way\n    data := data[pool.GetThreadId()]\n\n    // do some work here...\n\n    // a job may also add other jobs to the queue...\n    g2 := pool.NewJobGroup()\n    if err := pool.AddJob(g2, func(pool threadpool.ThreadPool, erf func() error) error {\n      // do some work here...\n    }); err != nil {\n      // some task returned an error\n    }\n    if err := pool.Wait(g2); err != nil {\n      // some task returned an error\n    }\n\n    // return nil if task is done\n    return nil\n  }); err != nil {\n    // some task returned an error\n  }\n  // add other tasks to g1...\n\n  // wait until all tasks are done\n  if err := pool.Wait(g1); err != nil {\n    // some task returned an error\n  }\n  // stop all threads\n  pool.Stop()\n```\n\nAny of the following functions can be used to add jobs to the queue:\n\n| Function    | Description                                           |\n| ----------- | --------------------------------------------------------------------------- |\n| AddJob      | add a single job to the queue                                               |\n| AddRangeJob | add a range job to the queue (replaces for-loops)                           |\n| Job         | create a job group, add a single job to the queue and wait until it is done |\n| RangeJob    | create a job group, add a range job to the queue and wait until it is done  |\n\n## Examples\n\n### Example 1: Simple job queuing\n```go\n  // create a new thread pool with 5 working threads and\n  // a queue buffer of 100 (in addition to this thread, 4\n  // more threads will be launched that start reading\n  // from the job queue)\n  pool := threadpool.New(5, 100)\n\n  // jobs are always grouped, get a new group index\n  g := pool.NewJobGroup()\n  // slice carrying the results\n  r := make([]int, 20)\n\n  // add jobs to the thread pool, where the i'th job sets\n  // r[i] to the thread index\n  for i_, _ := range r {\n    i := i_\n    pool.AddJob(g, func(pool threadpool.ThreadPool, erf func() error) error {\n      time.Sleep(10 * time.Millisecond)\n      r[i] = pool.GetThreadId()+1\n      return nil\n    })\n  }\n  // wait until all jobs in group g are done, meanwhile, this thread\n  // is also used as a worker\n  pool.Wait(g)\n  fmt.Println(\"result:\", r)\n```\n\n### Example 2: Distribute range equally among threads\n```go\n  pool := threadpool.New(5, 100)\n\n  g := pool.NewJobGroup()\n  r := make([]int, 20)\n\n  // instead of creating len(r) jobs, this method splits\n  // r into #threads pieces and adds one job for each piece\n  // to increase efficiency\n  pool.AddRangeJob(0, len(r), g, func(i int, pool threadpool.ThreadPool, erf func() error) error {\n    time.Sleep(10 * time.Millisecond)\n    r[i] = pool.GetThreadId()+1\n    return nil\n  })\n  pool.Wait(g)\n  fmt.Println(\"result:\", r)\n```\n\n### Example 3: Error handling\n```go\n  pool := threadpool.New(5, 100)\n\n  g := pool.NewJobGroup()\n  r := make([]int, 20)\n\n  if err := pool.AddRangeJob(0, len(r), g, func(i int, pool threadpool.ThreadPool, erf func() error) error {\n    time.Sleep(10 * time.Millisecond)\n    // stop if there was an error in one of the\n    // previous jobs\n    if erf() != nil {\n      // stop if there was an error\n      return nil\n    }\n    if i == 2 {\n      r[i] = -1\n      return fmt.Errorf(\"error in thread %d\", pool.GetThreadId())\n    } else {\n      r[i] = pool.GetThreadId()+1\n      return nil\n    }\n  }); err != nil {\n    fmt.Println(err)\n  }\n  if err := pool.Wait(g); err != nil {\n    fmt.Println(err)\n  }\n  fmt.Println(\"result:\", r)\n```\n\n### Example 4: Nested job queuing\n```go\n  pool := threadpool.New(5, 100)\n\n  g0 := pool.NewJobGroup()\n  r  := make([][]int, 5)\n\n  pool.AddRangeJob(0, len(r), g0, func(i int, pool threadpool.ThreadPool, erf func() error) error {\n    r[i] = make([]int, 5)\n\n    // get a new job group for filling the i'th sub-slice, which allows\n    // us to wait until the sub-slice is filled\n    gi := pool.NewJobGroup()\n\n    for j_, _ := range r[i] {\n      j := j_\n      pool.AddJob(gi, func(pool threadpool.ThreadPool, erf func() error) error {\n        time.Sleep(10 * time.Millisecond)\n        r[i][j] = pool.GetThreadId()+1\n        return nil\n      })\n    }\n    // wait until sub-slice i is filled\n    pool.Wait(gi)\n    return nil\n  })\n  // wait until the whole slice is filled\n  pool.Wait(g0)\n  fmt.Println(\"result:\", r)\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpbenner%2Fthreadpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpbenner%2Fthreadpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpbenner%2Fthreadpool/lists"}