{"id":37224943,"url":"https://github.com/ouzi-dev/kinesis-producer","last_synced_at":"2026-01-15T01:44:38.859Z","repository":{"id":57517306,"uuid":"246571057","full_name":"ouzi-dev/kinesis-producer","owner":"ouzi-dev","description":"An aggregated records producer for Amazon Kinesis","archived":false,"fork":true,"pushed_at":"2021-02-25T11:43:49.000Z","size":71,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-06-20T16:48:17.062Z","etag":null,"topics":["aws","kinesis","kinesis-producer","kpl-aggregation"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"a8m/kinesis-producer","license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ouzi-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-11T12:56:03.000Z","updated_at":"2022-12-16T16:09:50.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ouzi-dev/kinesis-producer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/ouzi-dev/kinesis-producer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ouzi-dev%2Fkinesis-producer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ouzi-dev%2Fkinesis-producer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ouzi-dev%2Fkinesis-producer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ouzi-dev%2Fkinesis-producer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ouzi-dev","download_url":"https://codeload.github.com/ouzi-dev/kinesis-producer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ouzi-dev%2Fkinesis-producer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28441031,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-15T00:55:22.719Z","status":"ssl_error","status_checked_at":"2026-01-15T00:55:20.945Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","kinesis","kinesis-producer","kpl-aggregation"],"created_at":"2026-01-15T01:44:38.268Z","updated_at":"2026-01-15T01:44:38.851Z","avatar_url":"https://github.com/ouzi-dev.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Amazon kinesis producer [![Build status][travis-image]][travis-url] [![License][license-image]][license-url] [![GoDoc][godoc-img]][godoc-url]\n\u003e A KPL-like batch producer for Amazon Kinesis built on top of the official Go AWS SDK  \nand using the same aggregation format that [KPL][kpl-url] use.  \n\n### Useful links\n- [Documentation][godoc-url]\n- [Aggregation format][aggregation-format-url]\n- [Considerations When Using KPL Aggregation][kpl-aggregation]\n- [Consumer De-aggregation][de-aggregation]\n\n### Example\n```go\npackage main\n\nimport (\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/a8m/kinesis-producer\"\n\t\"github.com/aws/aws-sdk-go/aws\"\n\t\"github.com/aws/aws-sdk-go/aws/session\"\n\t\"github.com/aws/aws-sdk-go/service/kinesis\"\n)\n\nfunc main() {\n\tclient := kinesis.New(session.New(aws.NewConfig()))\n\tpr := producer.New(\u0026producer.Config{\n\t\tStreamName:   \"test\",\n\t\tBacklogCount: 2000,\n\t\tClient:       client\n\t})\n\n\tpr.Start()\n\n\t// Handle failures\n\tgo func() {\n\t\tfor r := range pr.NotifyFailures() {\n\t\t\t// r contains `Data`, `PartitionKey` and `Error()`\n\t\t\tlog.Error(r)\n\t\t}\n\t}()\n\n\tgo func() {\n\t\tfor i := 0; i \u003c 5000; i++ {\n\t\t\terr := pr.Put([]byte(\"foo\"), \"bar\")\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(err).Fatal(\"error producing\")\n\t\t\t}\n\t\t}\n\t}()\n\n\ttime.Sleep(3 * time.Second)\n\tpr.Stop()\n}\n```\n\n#### Specifying logger implementation\n`producer.Config` takes an optional `logging.Logger` implementation.\n\n##### Using a custom logger\n```go\ncustomLogger := \u0026CustomLogger{}\n\n\u0026producer.Config{\n  StreamName:   \"test\",\n  BacklogCount: 2000,\n  Client:       client,\n  Logger:       customLogger,\n}\n```\n\n#### Using logrus\n\n```go\nimport (\n\t\"github.com/sirupsen/logrus\"\n\tproducer \"github.com/a8m/kinesis-producer\"\n\t\"github.com/a8m/kinesis-producer/loggers\"\n)\n\nlog := logrus.New()\n\n\u0026producer.Config{\n  StreamName:   \"test\",\n  BacklogCount: 2000,\n  Client:       client,\n  Logger:       loggers.Logrus(log),\n}\n```\n\nkinesis-producer ships with three logger implementations.\n\n- `producer.Standard` used the standard library logger\n- `loggers.Logrus` uses logrus logger\n- `loggers.Zap` uses zap logger\n\n### License\nMIT\n\n[godoc-url]: https://godoc.org/github.com/a8m/kinesis-producer\n[godoc-img]: https://img.shields.io/badge/godoc-reference-blue.svg?style=flat-square\n[kpl-url]: https://github.com/awslabs/amazon-kinesis-producer\n[de-aggregation]: http://docs.aws.amazon.com/kinesis/latest/dev/kinesis-kpl-consumer-deaggregation.html\n[kpl-aggregation]: http://docs.aws.amazon.com/kinesis/latest/dev/kinesis-producer-adv-aggregation.html\n[aggregation-format-url]: https://github.com/a8m/kinesis-producer/blob/master/aggregation-format.md\n[license-image]: https://img.shields.io/badge/license-MIT-blue.svg?style=flat-square\n[license-url]: LICENSE\n[travis-image]: https://img.shields.io/travis/a8m/kinesis-producer.svg?style=flat-square\n[travis-url]: https://travis-ci.org/a8m/kinesis-producer\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fouzi-dev%2Fkinesis-producer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fouzi-dev%2Fkinesis-producer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fouzi-dev%2Fkinesis-producer/lists"}