{"id":13562831,"url":"https://github.com/contribsys/faktory_worker_go","last_synced_at":"2025-04-03T19:31:35.492Z","repository":{"id":26325285,"uuid":"105295196","full_name":"contribsys/faktory_worker_go","owner":"contribsys","description":"Faktory workers for Go","archived":false,"fork":false,"pushed_at":"2025-03-19T17:54:59.000Z","size":152,"stargazers_count":242,"open_issues_count":12,"forks_count":41,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-19T18:38:49.575Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/contribsys.png","metadata":{"files":{"readme":"README.md","changelog":"Changes.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-29T16:40:23.000Z","updated_at":"2025-03-19T17:55:03.000Z","dependencies_parsed_at":"2024-01-23T20:57:14.283Z","dependency_job_id":"62a25d34-c58e-4eef-81ce-bee0473cabd5","html_url":"https://github.com/contribsys/faktory_worker_go","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/contribsys%2Ffaktory_worker_go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/contribsys%2Ffaktory_worker_go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/contribsys%2Ffaktory_worker_go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/contribsys%2Ffaktory_worker_go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/contribsys","download_url":"https://codeload.github.com/contribsys/faktory_worker_go/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247065255,"owners_count":20877743,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T13:01:12.657Z","updated_at":"2025-04-03T19:31:35.485Z","avatar_url":"https://github.com/contribsys.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# faktory_worker_go\n\n![travis](https://travis-ci.org/contribsys/faktory_worker_go.svg?branch=master)\n\nThis repository provides a Faktory worker process for Go apps.\nThis worker process fetches background jobs from the Faktory server and processes them.\n\nHow is this different from all the other Go background worker libraries?\nThey all use Redis or another \"dumb\" datastore.\nThis library is far simpler because the Faktory server implements most of the data storage, retry logic, Web UI, etc.\n\n# Installation\n\nYou must install [Faktory](https://github.com/contribsys/faktory) first.\nThen:\n\n```\ngo get -u github.com/contribsys/faktory_worker_go\n```\n\n# Usage\n\nTo process background jobs, follow these steps:\n\n1. Register your job types and their associated funcs\n2. Set a few optional parameters\n3. Start the processing\n\nThere are a couple ways to stop the process.\nIn this example, send the TERM or INT signal.\n\n```go\npackage main\n\nimport (\n  \"context\"\n  \"log\"\n\n  worker \"github.com/contribsys/faktory_worker_go\"\n)\n\nfunc someFunc(ctx context.Context, args ...interface{}) error {\n  help := worker.HelperFor(ctx)\n  log.Printf(\"Working on job %s\\n\", help.Jid())\n  return nil\n}\n\nfunc main() {\n  mgr := worker.NewManager()\n\n  // register job types and the function to execute them\n  mgr.Register(\"SomeJob\", someFunc)\n  //mgr.Register(\"AnotherJob\", anotherFunc)\n\n  // use up to N goroutines to execute jobs\n  mgr.Concurrency = 20\n  // wait up to 25 seconds to let jobs in progress finish\n  mgr.ShutdownTimeout = 25 * time.Second\n\n  // pull jobs from these queues, in this order of precedence\n  mgr.ProcessStrictPriorityQueues(\"critical\", \"default\", \"bulk\")\n\n  // alternatively you can use weights to avoid starvation\n  //mgr.ProcessWeightedPriorityQueues(map[string]int{\"critical\":3, \"default\":2, \"bulk\":1})\n\n  // Start processing jobs, this method does not return.\n  mgr.Run()\n}\n```\n\nAlternatively you can control the stopping of the Manager using\n`RunWithContext`. **You must process any signals yourself.**\n\n```go\npackage main\n\nimport (\n  \"context\"\n  \"log\"\n  \"os\"\n  \"os/signal\"\n  \"syscall\"\n\n  worker \"github.com/contribsys/faktory_worker_go\"\n)\n\nfunc someFunc(ctx context.Context, args ...interface{}) error {\n  help := worker.HelperFor(ctx)\n  log.Printf(\"Working on job %s\\n\", help.Jid())\n  return nil\n}\n\nfunc main() {\n  ctx, cancel := context.WithCancel(context.Background())\n  mgr := worker.NewManager()\n\n  // register job types and the function to execute them\n  mgr.Register(\"SomeJob\", someFunc)\n  //mgr.Register(\"AnotherJob\", anotherFunc)\n\n  // use up to N goroutines to execute jobs\n  mgr.Concurrency = 20\n  // wait up to 25 seconds to let jobs in progress finish\n  mgr.ShutdownTimeout = 25 * time.Second\n\n  // pull jobs from these queues, in this order of precedence\n  mgr.ProcessStrictPriorityQueues(\"critical\", \"default\", \"bulk\")\n\n  // alternatively you can use weights to avoid starvation\n  //mgr.ProcessWeightedPriorityQueues(map[string]int{\"critical\":3, \"default\":2, \"bulk\":1})\n\n  go func(){\n    // Start processing jobs in background routine, this method does not return\n    // unless an error is returned or cancel() is called\n    mgr.RunWithContext(ctx)\n  }()\n\n  go func() {\n    stopSignals := []os.Signal{\n      syscall.SIGTERM,\n      syscall.SIGINT,\n      // TODO Implement the TSTP signal to call mgr.Quiet()\n    }\n    stop := make(chan os.Signal, len(stopSignals))\n    for _, s := range stopSignals {\n       signal.Notify(stop, s)\n    }\n\n    for {\n      select {\n      case \u003c-ctx.Done():\n        return\n      case \u003c-stop:\n        break\n      }\n    }\n\n    _ = time.AfterFunc(mgr.ShutdownTimeout, cancel)\n    _ = mgr.Terminate(true) // never returns\n  }()\n\n  \u003c-ctx.Done()\n}\n```\n\n# Middleware\n\nAttach middleware if you want to run code around every job execution with full access to the Job object. Returning an error will FAIL the job, returning nil will ACK it.\n\n```go\nmgr := worker.NewManager()\nmgr.Use(func(ctx context.Context, job *faktory.Job, next func(context.Context) error) error {\n  modctx := context.WithValue(ctx, EXAMPLE, 4.0)\n  return next(modctx)\n})\n```\n\n# Testing\n\n`faktory_worker_go` provides helpers that allow you to configure tests to execute jobs inline if you prefer. In this example, the application has defined its own wrapper function for `client.Push`.\n\n```go\nimport (\n\tfaktory \"github.com/contribsys/faktory/client\"\n\tworker \"github.com/contribsys/faktory_worker_go\"\n)\n\nfunc Push(mgr worker.Manager, job *faktory.Job) error {\n\tif viper.GetBool(\"faktory_inline\") {\n\t\treturn syntheticPush(mgr worker.Manager, job)\n\t}\n\treturn realPush(job)\n}\n\nfunc syntheticPush(mgr worker.Manager, job *faktory.Job) error {\n\terr := mgr.InlineDispatch(job)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"syntheticPush failed\")\n\t}\n\n  return nil\n}\n\nfunc realPush(job *faktory.Job) error {\n\tclient, err := faktory.Open()\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"failed to open Faktory client connection\")\n\t}\n\n\terr = client.Push(job)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"failed to enqueue Faktory job\")\n\t}\n\n\treturn nil\n}\n```\n\n# FAQ\n\n* How do I specify the Faktory server location?\n\nBy default, it will use localhost:7419 which is sufficient for local development.\nUse FAKTORY\\_URL to specify the URL, e.g. `tcp://faktory.example.com:12345` or\nuse FAKTORY\\_PROVIDER to specify the environment variable which does\ncontain the URL: FAKTORY\\_PROVIDER=FAKTORYTOGO\\_URL.  This level of\nindirection is useful for SaaSes, Heroku Addons, etc.\n\n* How do I push new jobs to Faktory?\n\n1. Inside a job, you can check out a connection from the Pool of Faktory\n   connections using the job helper's `With` method:\n```go\nfunc someFunc(ctx context.Context, args ...interface{}) error {\n  help := worker.HelperFor(ctx)\n  return help.With(func(cl *faktory.Client) error {\n    job := faktory.NewJob(\"SomeJob\", 1, 2, 3)\n    return cl.Push(job)\n  })\n}\n```\n2. You can always open a client connection to Faktory directly but this\n   won't perform as well:\n```go\nimport (\n  faktory \"github.com/contribsys/faktory/client\"\n)\n\nclient, err := faktory.Open()\njob := faktory.NewJob(\"SomeJob\", 1, 2, 3)\nerr = client.Push(job)\n```\n\n**NB:** Client instances are **not safe to share**, you can use a Pool of Clients\nwhich is thread-safe.\n\nSee the Faktory Client API for\n[Go](https://github.com/contribsys/faktory/blob/main/client) or\n[Ruby](https://github.com/contribsys/faktory_worker_ruby/blob/main/lib/faktory/client.rb).\nYou can implement a Faktory client in any programming language.\nSee [the wiki](https://github.com/contribsys/faktory/wiki) for details.\n\n# Author\n\nMike Perham, https://ruby.social/@getajobmike\n\n# License\n\nThis codebase is licensed via the Mozilla Public License, v2.0. https://choosealicense.com/licenses/mpl-2.0/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontribsys%2Ffaktory_worker_go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcontribsys%2Ffaktory_worker_go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontribsys%2Ffaktory_worker_go/lists"}