{"id":37125271,"url":"https://github.com/arquivei/goduck","last_synced_at":"2026-01-22T19:30:47.801Z","repository":{"id":37251856,"uuid":"214006776","full_name":"arquivei/goduck","owner":"arquivei","description":"Simple and extensible stream/pool consumer","archived":false,"fork":false,"pushed_at":"2026-01-12T15:57:17.000Z","size":413,"stargazers_count":6,"open_issues_count":14,"forks_count":4,"subscribers_count":16,"default_branch":"main","last_synced_at":"2026-01-12T20:50:12.489Z","etag":null,"topics":["hacktoberfest","kafka","serverless"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arquivei.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-10-09T19:33:05.000Z","updated_at":"2026-01-12T15:48:20.000Z","dependencies_parsed_at":"2023-02-19T08:30:52.333Z","dependency_job_id":"ad222fc8-cc0b-4c9c-8b72-a37e7255130d","html_url":"https://github.com/arquivei/goduck","commit_stats":null,"previous_names":[],"tags_count":38,"template":false,"template_full_name":null,"purl":"pkg:github/arquivei/goduck","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arquivei%2Fgoduck","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arquivei%2Fgoduck/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arquivei%2Fgoduck/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arquivei%2Fgoduck/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arquivei","download_url":"https://codeload.github.com/arquivei/goduck/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arquivei%2Fgoduck/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28423163,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T13:30:50.153Z","status":"ssl_error","status_checked_at":"2026-01-14T13:29:08.907Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","kafka","serverless"],"created_at":"2026-01-14T14:29:12.670Z","updated_at":"2026-01-14T14:29:13.448Z","avatar_url":"https://github.com/arquivei.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"## goDuck\n\nThis project's purpose is to be an engine that abstract message dispatching for workers\nthat deals with the concept of either streams or pools. \nIn other words, reading a message from a stream or a pool, and delivering that message\nthrough the **Processor** Interface and interpreting its return value.\n\n\u003eIt is important to note that, if the Process function returns an error, the engine wont \nAck the message, thus, not removing it from the queue or stream. The main idea for this, \nis that the engine guarantees that every message will be processed at least once, without errors.  \n\nSample of a stream processor\n```go\nimport(\n\t\"github.com/arquivei/goduck\"\n\t\"github.com/arquivei/goduck/engine/streamengine\"\n)\n// The engine requires a type that implements the Process function\ntype processor struct{}\n\n// Process func will receive the pulled message from the engine.\nfunc (p processor) Process(ctx context.Context, message []byte) error {\n\t...\n    err := serviceCall(args)\n    ...\n\treturn err\n}\nfunc main {\n    // call below returns a kafka abstraction (interface)\n    kafka := NewKafkaStream(\u003cyour-config\u003e)\n    engine := streamengine.New(processor{}, []goduck.Stream{kafka})\n    engine.Run(context.Background())\n}\n```\n\nSample of a pool processor\n```go\nimport(\n\t\"github.com/arquivei/goduck\"\n\t\"github.com/arquivei/goduck/engine/streamengine\"\n)\n// The engine requires a type that implements the Process function\ntype processor struct{}\n\n// Process func will receive the pulled message from the engine.\nfunc (p processor) Process(ctx context.Context, message []byte) error {\n\t...\n    err := serviceCall(args)\n    ...\n\treturn err\n}\n\n\nfunc main {\n    // call below returns a pubsub abstraction (interface)\n    pubsub, err := NewPubsubQueue(\u003cyour-config\u003e) \n    if err != nil {\n        \u003chandle err\u003e\n    }\n    engine := jobpoolengine.New(pubsub, processor{}, 1)\n    engine.Run(context.Background())\n}\n```\n\n\n## Important configuration\n### Kafka\n* Commit interval:\nSet this to allow asynchronous message processing between commits. \nWithout a value, defaults to every message having to be acknowledged before a new one\nis retrieved.\nThis is bad to have when you should avoid message reprocessing. Suppose there is a failure\nand the engine stops executing while processing. The larger the commit interval is, higher\nis the chance of duplicating messages \n\nTo terminate the engine execution, a simple context cancellation will perform a shutdown\nof the application.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farquivei%2Fgoduck","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farquivei%2Fgoduck","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farquivei%2Fgoduck/lists"}