{"id":48367133,"url":"https://github.com/not-empty/omniq-go","last_synced_at":"2026-05-03T16:01:57.649Z","repository":{"id":337654815,"uuid":"1154575518","full_name":"not-empty/omniq-go","owner":"not-empty","description":"Golang library for OmniQ, the language agnostic queue ecosystem with parallel limits, draining and much more. Publish and consume in your favorite language like Python, Go, Node (TS or Vanilla), PHP and any other you want to.","archived":false,"fork":false,"pushed_at":"2026-04-05T13:51:42.000Z","size":88,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-05T15:26:09.981Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/not-empty.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-10T14:45:55.000Z","updated_at":"2026-04-05T13:51:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/not-empty/omniq-go","commit_stats":null,"previous_names":["not-empty/omniq-go"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/not-empty/omniq-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Fomniq-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Fomniq-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Fomniq-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Fomniq-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/not-empty","download_url":"https://codeload.github.com/not-empty/omniq-go/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/not-empty%2Fomniq-go/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32575115,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-03T06:36:36.687Z","status":"ssl_error","status_checked_at":"2026-05-03T06:36:09.306Z","response_time":103,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-05T15:02:11.131Z","updated_at":"2026-05-03T16:01:57.642Z","avatar_url":"https://github.com/not-empty.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OmniQ - GO\n\nGo client for OmniQ, a Redis-based distributed job queue designed for \ndeterministic, consumer-driven job execution and coordination.\n\n**OmniQ Go** executes queue logic directly inside Redis using Lua scripts,\nensuring atomicity, consistency and predictable behavior across distributed\nsystems.\n\nInstead of treating jobs as transient messages, Omniq maintains explicit\nexecution state and coordination primitives Redis, allowing consumers to\nsafely manage retries, concurrency, ordering, and distributed processing.\n\nThe system is language-agnostic, anabling producers and consumers written in diferrent languages to share the same execution model.\n\nCore project and protocol documentation:\nhttps://github.com/not-empty/omniq\n\n------------------------------------------------------------------------\n\n## Versioning\n\nOmniQ is a multi-language system (Go, Node.js, Python), so versions are aligned across SDKs.\n\nThe Go module stays on `v1`:\n\n### Convention\n\nWe use minor versions in multiples of 10 to represent ecosystem milestones:\n\n* `v1.x.x` → OmniQ v1\n* `v1.20.x` → OmniQ v2\n* `v1.30.x` → OmniQ v3\n\n### Recommendation\n\nWe strongly advise locking your minor version to avoid unexpected behavior changes:\n\ngo get github.com/not-empty/omniq-go@v1.30.0\n\n------------------------------------------------------------------------\n\n## Installation\nIn your Go project:\n``` bash\ngo get github.com/not-empty/omniq-go\n```\n\nThe main package will be imported as:\n``` bash\nimport \"github.com/not-empty/omniq-go\n```\n\n------------------------------------------------------------------------\n\n## Connection Model\n\n`ClientOpts` uses `Host` and `Port` for Redis connections.\n\n```go\nclient, err := omniq.NewClient(omniq.ClientOpts{\n\tHost: \"127.0.0.1\",\n\tPort: 6379,\n})\n```\n\nIf the target is a Redis Cluster, the Go transport will detect it and use\ncluster mode automatically.\n\n------------------------------------------------------------------------\n\n## Queue Names\n\nQueue names are validated in Go before they reach Redis.\n\nAllowed:\n- letters\n- numbers\n- `.`\n- `_`\n- `-`\n\nRejected:\n- empty names\n- leading or trailing whitespace\n- `{` or `}`\n- `:`\n- spaces and unsupported characters\n- names longer than `128` chars\n\n------------------------------------------------------------------------\n\n## Features\n\n- Redis-native execution model:\n  - Queue operations are executed atomically inside Redis using Lua\n- Consumer-driven processing:  \n  - Workers control job reservation and execution lifecycle\n- Deterministc job state:\n  - Explicit handling of job states such as wait, active, failed, and completed\n- Grouped jobs with concurrency control:\n  - FIFO ordering within groups and parallel execution across groups\n- Atomic administrative operations:\n  -  Retry, removal, pause, and batch operations with strong consistency\n- Parent/Child workflow primitive:\n  - Fan-out execution with atomic completion tracking\n- Cross-language compatibility:\n  - Same execution model across different runtimes\n   \n------------------------------------------------------------------------\n\n## Main Concepts\n\n### Execution Model\n- **Jobs** are sent to a queue with data (payload) and a maximum number\nof attempts.\n- Workers reserve job using a **lease** (temporary lock).\n- Execution confirmation (ack) or failure happens based on the handler \nresult.\n- Falied jobs are retried until the configured number of attempts is\nreached.\n\n------------------------------------------------------------------------\n\n## Usage Example\n\n### Initialize Client\n\n``` go\nclient, err := omniq.NewClient(omniq.ClientOpts{\n\tHost: \"localhost\",\n\tPort: 6379,\n})\nif err != nil {\n\tlog.Fatalf(\"error creating client: %v\", err)\n}\n```\n\n------------------------------------------------------------------------\n\n### Publish Jobs\n\n``` go\njobId, err := client.Publish(omniq.PublishOpts {\n\tQueue: \"demo\",\n\tPayload: map[string]any{\"hello\": \"world\"},\n\tMaxAttempts: 3,\n})\nif err != nil {\n\tlog.Fatalf(\"failed to publish job: %v\", err)\n}\nfmt.Println(\"Published JOb ID: \", jobId)\t\n```\n\n------------------------------------------------------------------------\n\n## Consume Jobs\n\n``` go\nerr := client.Cosume(omniq.CosumeOpts {\n\tQueue: \"demo\",\n\tHandler: func(stx omniq.JObCtx) {\n\t\tvar payload struct {\n\t\t\tHello string `json:\"hello\"`\n\t\t}\n\t\tif err := ctx.DecodePayload(\u0026payload); err != nill {\n\t\t\tpanic(err)\n\t\t}\n\t\tlog.Println(\"Processing: \", payload.Hello)\n\t},\n})\t\nif err != nil {\n\tlog.Fatalf(\"consumer error: %v\", err)\n}\n```\n**Handler behavior**\n- If the handler finishes normally, the jobs is considered **successfully executed**.\n- If a `panic(...)` accors, the job will be marked as **failed** and may \nbe retried.\n\nHandler context includes:\n- `Attempt`\n- `MaxAttempts`\n- `Exec`\n\nExample:\n\n``` go\nfunc handler(ctx omniq.JobCtx) {\n\tisLastAttempt := ctx.Attempt \u003e= ctx.MaxAttempts\n\tlog.Println(\"Last attempt?\", isLastAttempt)\n}\n```\n\nSee `examples/max_attempts` for a complete retry-until-last-attempt flow.\n\n------------------------------------------------------------------------\n\n## Monitoring Discovery\n\nQueue discovery uses `ScanQueues()`, which scans Redis for `*:stats` keys and\nreturns normalized queue names.\n\n```go\nmonitor, err := omniq.NewMonitor(client)\nif err != nil {\n\tlog.Fatalf(\"monitor error: %v\", err)\n}\n\nqueues := monitor.ScanQueues()\nfmt.Println(queues)\n```\n\n`ScanQueues()` is intended for admin/bootstrap discovery, not for hot-path UI\npolling.\n\n------------------------------------------------------------------------\n\n# Administrative Operations !!!\nThe following queue operations are supported atomically:\n\n## Retry a Falied Job\n\n``` go\nerr := client.RetryFailed(\"demo\", \"jobId\")\n```\n\n------------------------------------------------------------------------\n\n## Retry in Batch\n\n``` go\nresults, err := client. RetryFailedBatch(\"demo\", []string{\"id1\", \"id2\"})\n```\n\n------------------------------------------------------------------------\n\n## Remove Job\n\n``` go\nerr := client.RemoveJob(\"demo\", \"jobId\", \"failed\")\n```\n\n------------------------------------------------------------------------\n\n## Remove Jobs in Batch\n\n``` go\nresults, err := client.RemoveJobsBatch(\"demo\", \"failed\", []string{\"id1\", \"id2\"})\n```\n\n------------------------------------------------------------------------\n\n## Pause and Resume / IsPaused Queue\n\n``` go\nerr := client.Pause(\"demo\")\npaused, _ := client.IsPaused(\"demo\")\nerr = client.Resume(\"demo\")\n```\nPausing prevents new reservations; jobs already running continue until\ncompletion.\n\n------------------------------------------------------------------------\n\n# Grouping (Group IDs - GID)\nJobs can be published with a **GID** in order to:\n- Maintein FIFO ordering within a group\n- Limit concurrency per group\n- Allow safe parallel execution between groups\nJobs without a GID are executed fairly across queues.\n\n------------------------------------------------------------------------\n\n## Parent/Child Workflows\nThis primitive enables fan-out workflows, where a parent job distributes work across multiple child jobs and tracks completion using an atomic counter stored in Redis.\n\nEach child job acknowledges completion using a shared completion key. The system guarantees that retries or duplicate executions do not corrupt the counter.\n\nWhen all child jobs complete, the counter reaches zero.\n\n#### Parent Example\n``` go\nfunc parent(ctx omniq.JobCtx) {\n\n    completionKey := \"doc-123\"\n\n    ctx.Exec.ChildsInit(completionKey, 5)\n\n    for i := 0; i \u003c 5; i++ {\n        ctx.Exec.Publish(omniq.PublishOpts{\n            Queue: \"pages\",\n            Payload: map[string]any{\n                \"page\":           i,\n                \"completion_key\": completionKey,\n            },\n        })\n    }\n}\n```\n\n#### Child Example\n\n``` go\nfunc pageWorker(ctx omniq.JobCtx) {\n\n    type PageJob struct {\n        Page          int    `json:\"page\"`\n        CompletionKey string `json:\"completion_key\"`\n    }\n\n    var p PageJob\n    if err := ctx.DecodePayload(\u0026p); err != nil {\n        panic(err)\n    }\n\n    remaining, err := ctx.Exec.ChildAck(p.CompletionKey)\n    if err != nil {\n        panic(err)\n    }\n\n    if remaining == 0 {\n        println(\"Last page finished.\")\n    }\n}\n```\n\n**Properties:**\n-   Idempotent decrement\n-   Safe under retries\n-   Cross-queue safe\n-   Fully business-logic driven\n\n------------------------------------------------------------------------\n\n## Grouped Jobs\n\n``` go\nclient.Publish(omniq.PublishOpts{\n    Queue:      \"demo\",\n    Payload:    map[string]any{\"i\": 1},\n    GID:        \"company:acme\",\n    GroupLimit: 1,\n})\n\nclient.Publish(omniq.PublishOpts{\n    Queue:   \"demo\",\n    Payload: map[string]any{\"i\": 2},\n})\n```\n\n------------------------------------------------------------------------\n\n## Pause and Resume Inside a Handler\n\n``` go\nfunc pauseExample(ctx omniq.JobCtx) {\n\n    paused, _ := ctx.Exec.IsPaused(\"test\")\n    println(\"Is paused:\", paused)\n\n    ctx.Exec.Pause(\"test\")\n\n    paused, _ = ctx.Exec.IsPaused(\"test\")\n    println(\"Is paused:\", paused)\n\n    ctx.Exec.Resume(\"test\")\n}\n```\n\n------------------------------------------------------------------------\n\n## Best Practices\n1. Idempotent handlers: always consider unexpected re-executions.\n2. Monitoring leases and execution: poorly consigured lease durations may\ncause duplication or timeouts.\n3. Redis sizing: adjust memory and persistence settings according to workload.\n\n------------------------------------------------------------------------\n\n## Versioning and Compatibility\nChanges to the contract (OmniQ protocol) follow **Semantic Versioning**.\nVersions that intruduce incompatible contract changes require a major \nversion increment, and clients must align with that version.\n\n------------------------------------------------------------------------\n\n## Examples\n\nAll examples can be found in the `./examples` folder.\n\n------------------------------------------------------------------------\n\n## License\n\nThis project is licensed under **GPL-3.0**. \nSee the `LICENSE` file for the complete terms.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnot-empty%2Fomniq-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnot-empty%2Fomniq-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnot-empty%2Fomniq-go/lists"}