{"id":43188598,"url":"https://github.com/honestbank/kp","last_synced_at":"2026-02-19T08:10:10.424Z","repository":{"id":40633689,"uuid":"506914105","full_name":"honestbank/kp","owner":"honestbank","description":"Library for handing Kafka messages with retries","archived":false,"fork":false,"pushed_at":"2026-01-22T03:55:31.000Z","size":3489,"stargazers_count":6,"open_issues_count":16,"forks_count":2,"subscribers_count":15,"default_branch":"main","last_synced_at":"2026-02-01T13:45:00.123Z","etag":null,"topics":["backend","backoff","deadletters","golang","hacktoberfest","kafka","retries","workspace-code-infrastructure-prod"],"latest_commit_sha":null,"homepage":"honestbank.github.io/kp","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/honestbank.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"contributing.md","funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-06-24T07:12:08.000Z","updated_at":"2025-12-26T08:12:48.000Z","dependencies_parsed_at":"2024-04-30T05:30:47.110Z","dependency_job_id":"449df23e-d6b2-4a20-982f-2641ca250914","html_url":"https://github.com/honestbank/kp","commit_stats":{"total_commits":50,"total_committers":9,"mean_commits":5.555555555555555,"dds":0.4,"last_synced_commit":"1809ff7d71f84e4eb94ddede3f42a963a5304793"},"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"purl":"pkg:github/honestbank/kp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/honestbank%2Fkp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/honestbank%2Fkp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/honestbank%2Fkp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/honestbank%2Fkp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/honestbank","download_url":"https://codeload.github.com/honestbank/kp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/honestbank%2Fkp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29608154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T06:47:36.664Z","status":"ssl_error","status_checked_at":"2026-02-19T06:45:47.551Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backend","backoff","deadletters","golang","hacktoberfest","kafka","retries","workspace-code-infrastructure-prod"],"created_at":"2026-02-01T04:35:43.843Z","updated_at":"2026-02-19T08:10:10.406Z","avatar_url":"https://github.com/honestbank.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kp\nLibrary for handing Kafka messages with retries\n\n## How it works\nEach KP instance create a kafka sarama client which will listen to 2 topics: main topic and retry topic. The Retry\ntopic will be prepended with the consumer group name such that each instance could have its own retry and dead letter\ntopics. This will enable a case such that you have two services that listen to the same topic but one service might\nfail the topic, with this it would only retry on that service and not the other one.\n\nCode Example: (see [examples](https://github.com/honestbank/kp/tree/main/examples))\n\n```golang\nfunc main() {\n\tcfg, err := config.LoadConfig()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tretryCount := 10\n\tconsumerGroup := \"simple-service\" // usually the name of your service\n\tbackoffDuration := time.Second * 1 // set to 0 for no backoff\n\n\tprocessor := kp.NewKafkaProcessor(\"main-topic\", \"retry-main-topic\", \"dead-main-topic\", retryCount, consumerGroup, kp.KafkaConfig{KafkaBootstrapServers: strings.Split(cfg.KafkaConfig.KafkaBootstrapServers, \",\")}, backoffDuration)\n\t// retry topic becomes \"simple-service-retry-main-topic\"\n\t// dead letter topic becomes \"simple-service-dead-main-topic\"\n\tprocessor.Process(func(key string, message string, retries int, rawMessage *sarama.ConsumerMessage) error {\n\t\terr := processMessage(message)\n\t\tif err != nil {\n\t\t\tlog.Printf(\"Error processing message: %s\", err)\n\t\t\treturn err\n\t\t}\n\t\tlog.Println(\"successfully processed message\")\n\t\treturn nil\n\t})\n\n\tprocessor.Start()\n\n}\n```\n\nTo disable retries, just set retries to 0\n\n## Exponential backoff\n\nThis library supports exponential backoff. To use a backoff policy, set the backoffDuration to anything above 1. To\ndisable the backoff policy, set the backoffDuration to 0. Backoff happens on the whole client which will slow down all\nmesaages. When a message is successful, the backoffDuration is reduced.\n\n## Topics\n\nKP Uses 3 topics: main topic, retry topic and dead letter topic.\nYou are able to modify the retry and dead letter topics like so:\n\n```golang\n\nprocessor := kp.NewKafkaProcessor(\n\tcfg.Kafka.CardReplacementTopic,\n\t\"retry-\"+cfg.Kafka.CardReplacementTopic,\n\t\"deadletter-\"+cfg.Kafka.CardReplacementTopic,\n\t10,\n\t\"card-delivery-service\",\n\tkp.KafkaConfig{KafkaBootstrapServers: strings.Split(cfg.Kafka.KafkaBootstrap, \",\")},\n\ttime.Second*5,\n\t)\n```\n\nThis is useful when you have a worker that listens to multiple topics and each topic needs to have its own retry and\ndead letter topic.\n\nFor retry and deadletter topics they are generated with the consumer group.\nex:\n```\n  topic: test\n  retrytopic: retry\n  deadlettertopic: deadletter\n  consumergroup: group\n\nresulting topics:\n    retrytopic: group-retry\n    deadlettertopic: group-deadletter\n\n\nresulting topic format: \u003cconsumer group\u003e-\u003cretry|deadletter topic\u003e\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhonestbank%2Fkp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhonestbank%2Fkp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhonestbank%2Fkp/lists"}