{"id":13694483,"url":"https://github.com/mcmathja/curlyq","last_synced_at":"2025-05-03T03:31:43.592Z","repository":{"id":57508955,"uuid":"235888596","full_name":"mcmathja/curlyq","owner":"mcmathja","description":"Efficient and reliable background processing for Go","archived":false,"fork":false,"pushed_at":"2020-05-01T00:29:33.000Z","size":111,"stargazers_count":131,"open_issues_count":1,"forks_count":8,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-11-12T21:38:42.273Z","etag":null,"topics":["background-jobs","background-worker","go","golang","redis"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mcmathja.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-23T21:19:21.000Z","updated_at":"2024-11-02T14:13:11.000Z","dependencies_parsed_at":"2022-08-29T20:00:15.746Z","dependency_job_id":null,"html_url":"https://github.com/mcmathja/curlyq","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcmathja%2Fcurlyq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcmathja%2Fcurlyq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcmathja%2Fcurlyq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcmathja%2Fcurlyq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mcmathja","download_url":"https://codeload.github.com/mcmathja/curlyq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252137580,"owners_count":21700243,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["background-jobs","background-worker","go","golang","redis"],"created_at":"2024-08-02T17:01:33.361Z","updated_at":"2025-05-03T03:31:43.339Z","avatar_url":"https://github.com/mcmathja.png","language":"Go","funding_links":[],"categories":["开源类库","Open source library"],"sub_categories":["任务/定时器","Task/Timer"],"readme":"# CurlyQ\n\n[![GoDoc](https://godoc.org/github.com/mcmathja/curlyq?status.svg)](https://godoc.org/github.com/mcmathja/curlyq)\n[![Build Status](https://api.travis-ci.org/mcmathja/curlyq.svg?branch=master)](https://travis-ci.org/mcmathja/curlyq)\n[![GoCover](https://gocover.io/_badge/github.com/mcmathja/curlyq)](https://gocover.io/github.com/mcmathja/curlyq)\n[![Go Report Card](https://goreportcard.com/badge/github.com/mcmathja/curlyq)](https://goreportcard.com/report/github.com/mcmathja/curlyq)\n[![License](https://img.shields.io/github/license/mcmathja/curlyq.svg)](https://github.com/mcmathja/curlyq/blob/master/LICENSE)\n\nCurlyQ provides a simple, easy-to-use interface for performing background processing in Go. It supports scheduled jobs, job deduplication, and configurable concurrent execution out of the box.\n\n## Quickstart\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"log\"\n\n\tcq \"github.com/mcmathja/curlyq\"\n)\n\nfunc main() {\n\t// Create a new producer\n\tproducer := cq.NewProducer(\u0026cq.ProducerOpts{\n\t\tAddress: \"localhost:6379\",\n\t\tQueue: \"testq\",\n\t})\n\n\t// Use the producer to push a job to the queue\n\tproducer.Perform(cq.Job{\n\t\tData: []byte(\"Some data!\"),\n\t})\n\n\t// Create a new consumer\n\tconsumer := cq.NewConsumer(\u0026cq.ConsumerOpts{\n\t\tAddress: \"localhost:6379\",\n\t\tQueue: \"testq\",\n\t})\n\n\t// Consume jobs from the queue with the consumer\n\tconsumer.Consume(func(ctx context.Context, job cq.Job) error {\n\t\tlog.Println(string(job.Data))\n\t\treturn nil\n\t})\n}\n```\n\n## The Basics\n\nCurlyQ exposes three key types: Jobs, Producers, and Consumers.\n\n\n### Jobs\n\nA Job wraps your data. In most cases, that's all you'll ever need to know about it:\n\n```go\njob := cq.Job{\n\tData: []byte(\"Some data.\"),\n}\n```\n\nEvery Job also exposes an `ID` field that uniquely identifies it among all jobs in the queue, and an `Attempt` field representing how many times it has been attempted so far.\n\n### Producers\n\nA Producer pushes jobs on to the queue. Create one by providing it with the address of your Redis instance and a queue name:\n\n```go\nproducer := cq.NewProducer(\u0026cq.ProducerOpts{\n\tAddress: \"my.redis.addr:6379\",\n\tQueue: \"queue_name\",\n})\n```\n\nYou can also provide an existing [go-redis](https://github.com/go-redis/redis) instance if you would like to configure the queue to run on a more advanced Redis configuration or set up your own retry and timeout logic for network calls:\n\n```go\nimport \"github.com/go-redis/redis/v7\"\n\nclient := redis.NewClient(\u0026redis.Options{\n\tPassword: \"p@55vvoRd\",\n\tDB: 3,\n\tMaxRetries: 2,\n})\n\nproducer := cq.NewProducer(\u0026cq.ProducerOpts{\n\tClient: client,\n\tQueue: \"queue_name\",\n})\n```\n\nRunning `producer.Perform(job)` will add a job to the queue to be run asynchronously. You can also schedule a job to be enqueued at a particular time by running `producer.PerformAt(time, job)`, or after a certain wait period by running `producer.PerformAfter(duration, job)`. All of the `Perform` methods return the ID assigned to the job and an error if one occurred.\n\nYou can deduplicate jobs by pre-assigning them IDs:\n\n```go\njob := cq.Job{\n\tID: \"todays_job\",\n}\n\n// Enqueue the job\nproducer.PerformAfter(10 * time.Second, job)\n\n// Does nothing, because a job with the same ID is already on the queue\nproducer.Perform(job)\n```\n\nOnce a job has been acknowledged, its ID becomes available for reuse.\n\nSee the documentation for [ProducerOpts](https://godoc.org/github.com/mcmathja/curlyq#ProducerOpts) for more details about available configuration options.\n\n### Consumers\n\nA Consumer pulls jobs off the queue and executes them using a provided handler function. Create one with the same basic options as a Producer:\n\n```go\nconsumer := cq.NewConsumer(\u0026cq.ConsumerOpts{\n\tQueue: \"queue_name\",\n\n\t// With an address:\n\tAddress: \"my.redis.addr:6379\",\n\t// With a preconfigured go-redis client:\n\tClient: redisClient,\n})\n```\n\nYou start a consumer by running its `Consume` method with a handler function:\n\n```go\nconsumer.Consume(func(ctx context.Context, job cq.Job) error {\n\tlog.Println(\"Job %s has been processed!\")\n\treturn nil\n})\n```\n\nIf the provided handler function returns `nil`, the job is considered to have been processed successfully and is removed from the queue. If the handler returns an error or panics, the job is considered to have failed and will be retried or killed based on how many times it has been attempted.\n\n`Consume` will continue to process jobs until your application receives an interrupt signal or the consumer encounters a fatal error. Fatal errors only occur when the consumer is unable to communicate with Redis for an essential operation, such as updating the status of a job in flight.\n\nSee the documentation for [ConsumerOpts](https://godoc.org/github.com/mcmathja/curlyq#ConsumerOpts) for more details about available configuration options.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcmathja%2Fcurlyq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmcmathja%2Fcurlyq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcmathja%2Fcurlyq/lists"}