{"id":20617101,"url":"https://github.com/k-yomo/pm","last_synced_at":"2025-04-15T08:57:01.411Z","repository":{"id":37483552,"uuid":"354424148","full_name":"k-yomo/pm","owner":"k-yomo","description":"Thin Cloud Pub/Sub client wrapper which lets you manage publishing / subscribing with pluggable middleware.","archived":false,"fork":false,"pushed_at":"2024-03-29T17:40:31.000Z","size":313,"stargazers_count":6,"open_issues_count":6,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T08:56:55.481Z","etag":null,"topics":["batch","cloudpubsub","gcp","interceptor","middleware","pluggable","pubsub"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/k-yomo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-04T00:36:40.000Z","updated_at":"2023-08-09T17:19:52.000Z","dependencies_parsed_at":"2024-04-01T13:15:15.594Z","dependency_job_id":null,"html_url":"https://github.com/k-yomo/pm","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k-yomo%2Fpm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k-yomo%2Fpm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k-yomo%2Fpm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k-yomo%2Fpm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/k-yomo","download_url":"https://codeload.github.com/k-yomo/pm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249040047,"owners_count":21202813,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batch","cloudpubsub","gcp","interceptor","middleware","pluggable","pubsub"],"created_at":"2024-11-16T11:22:11.187Z","updated_at":"2025-04-15T08:57:01.389Z","avatar_url":"https://github.com/k-yomo.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pm\n\n![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)\n![Test Workflow](https://github.com/k-yomo/pm/workflows/Test/badge.svg)\n[![codecov](https://codecov.io/gh/k-yomo/pm/branch/main/graph/badge.svg)](https://codecov.io/gh/k-yomo/pm)\n[![Go Report Card](https://goreportcard.com/badge/k-yomo/pm)](https://goreportcard.com/report/k-yomo/pm)\n\npm is a thin Cloud Pub/Sub client wrapper which lets you manage publishing / subscribing with pluggable middleware.\n\n## Installation\n\n```sh\ngo get -u github.com/k-yomo/pm\n```\n\n## Example\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"os/signal\"\n\t\"syscall\"\n\t\"time\"\n\n\t\"cloud.google.com/go/pubsub\"\n\t\"github.com/k-yomo/pm\"\n\t\"github.com/k-yomo/pm/middleware/logging/pm_zap\"\n\t\"github.com/k-yomo/pm/middleware/pm_attributes\"\n\t\"github.com/k-yomo/pm/middleware/pm_autoack\"\n\t\"github.com/k-yomo/pm/middleware/pm_recovery\"\n\t\"go.uber.org/zap\"\n)\n\nfunc main() {\n\tlogger, _ := zap.NewProduction()\n\tctx := context.Background()\n\tpubsubClient, err := pubsub.NewClient(ctx, \"pm-example\")\n\tif err != nil {\n\t\tlogger.Fatal(\"initialize pubsub client failed\", zap.Error(err))\n\t}\n\tdefer pubsubClient.Close()\n\n\tpubsubPublisher := pm.NewPublisher(\n\t\tpubsubClient,\n\t\tpm.WithPublishInterceptor(\n\t\t\tpm_attributes.PublishInterceptor(map[string]string{\"key\": \"value\"}),\n\t\t),\n\t)\n\n\tpubsubSubscriber := pm.NewSubscriber(\n\t\tpubsubClient,\n\t\tpm.WithSubscriptionInterceptor(\n\t\t\tpm_zap.SubscriptionInterceptor(logger),\n\t\t\tpm_autoack.SubscriptionInterceptor(),\n\t\t\tpm_recovery.SubscriptionInterceptor(pm_recovery.WithDebugRecoveryHandler()),\n\t\t),\n\t)\n\tdefer pubsubSubscriber.Close()\n\n\tsub := pubsubClient.Subscription(\"example-topic-sub\")\n\tbatchSub := pubsubClient.Subscription(\"example-topic-batch-sub\")\n\terr = pubsubSubscriber.HandleSubscriptionFuncMap(map[*pubsub.Subscription]pm.MessageHandler{\n\t\tsub: exampleSubscriptionHandler,\n\t\tbatchSub: pm.NewBatchMessageHandler(exampleSubscriptionBatchHandler, pm.BatchMessageHandlerConfig{\n\t\t\tDelayThreshold:    100 * time.Millisecond,\n\t\t\tCountThreshold:    1000,\n\t\t\tByteThreshold:     1e6,\n\t\t\tBufferedByteLimit: 1e8,\n\t\t}),\n\t})\n\tif err != nil {\n\t\tlogger.Fatal(\"register subscription failed\", zap.Error(err))\n\t}\n\n\tpubsubSubscriber.Run(ctx)\n\tdefer pubsubSubscriber.Close()\n\n\tpubsubPublisher.Publish(\n\t\tctx,\n\t\tpubsubPublisher.Topic(\"example-topic\"),\n\t\t\u0026pubsub.Message{\n\t\t\tData: []byte(\"test\"),\n\t\t},\n\t)\n\n\tc := make(chan os.Signal, 1)\n\tsignal.Notify(c, os.Interrupt, syscall.SIGTERM)\n\t\u003c-c\n}\n\nfunc exampleSubscriptionHandler(ctx context.Context, m *pubsub.Message) error {\n\tdataStr := string(m.Data)\n\tif dataStr == \"panic\" {\n\t\tpanic(\"panic\")\n\t}\n\n\tif dataStr == \"error\" {\n\t\treturn errors.New(\"error\")\n\t}\n\n\tfmt.Println(dataStr)\n\treturn nil\n}\n\nfunc exampleSubscriptionBatchHandler(messages []*pubsub.Message) error {\n\tbatchErr := make(pm.BatchError)\n\tfor _, m := range messages {\n\t\tdataStr := string(m.Data)\n\t\tif dataStr == \"error\" {\n\t\t\tbatchErr[m.ID] = errors.New(\"error\")\n\t\t} else {\n\t\t\tfmt.Println(dataStr)\n\t\t}\n\t}\n\n\treturn batchErr\n}\n```\n\n## Middlewares\n\n### Core Middleware\n\npm comes equipped with an optional middleware packages named `pm_*`.\n\n#### Publish interceptor\n\n| interceptor                                                                                                | description                                                              |\n|------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------|\n| [Attributes](https://pkg.go.dev/github.com/k-yomo/pm/middleware/pm_attributes#PublishInterceptor)          | Set custom attributes to all outgoing messages when publish              |\n\n#### Subscription interceptor\n\n| interceptor                                                                                                        | description                                                              |\n|--------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------|\n| [Auto Ack](https://pkg.go.dev/github.com/k-yomo/pm/middleware/pm_autoack#SubscriptionInterceptor)                 | Ack automatically depending on if error is returned when subscribe       |\n| [Effectively Once](https://pkg.go.dev/github.com/k-yomo/pm/middleware/pm_effectively_once#SubscriptionInterceptor)| De-duplicate messages with the same de-duplicate key                     |\n| [Logging - Zap](https://pkg.go.dev/github.com/k-yomo/pm/middleware/logging/pm_zap#SubscriptionInterceptor)        | Emit an informative zap log when subscription processing finish          |\n| [Logging - Logrus](https://pkg.go.dev/github.com/k-yomo/pm/middleware/logging/pm_logrus#SubscriptionInterceptor) | Emit an informative logrus log when subscription processing finish       |\n| [Recovery](https://pkg.go.dev/github.com/k-yomo/pm/middleware#SubscriptionInterceptor)                | Gracefully recover from panics and prints the stack trace when subscribe |\n\n#### Custom Middleware\n\npm middleware is just wrapping publishing / subscribing process which means you can define your custom middleware as well.\n- publish interceptor\n```go\nfunc MyPublishInterceptor(attrs map[string]string) pm.PublishInterceptor {\n\treturn func (next pm.MessagePublisher) pm.MessagePublisher {\n\t\treturn func (ctx context.Context, topic *pubsub.Topic, m *pubsub.Message) *pubsub.PublishResult {\n\t\t\t// do something before publishing \n\t\t\tresult := next(ctx, topic, m)\n\t\t\t// do something after publishing \n\t\t\treturn result\n\t\t}\n\t}\n}\n```\n\n- subscription interceptor\n```go\nfunc MySubscriptionInterceptor() pm.SubscriptionInterceptor {\n\treturn func(_ *pm.SubscriptionInfo, next pm.MessageHandler) pm.MessageHandler {\n\t\treturn func(ctx context.Context, m *pubsub.Message) error {\n\t\t\t// do something before subscribing \n\t\t\terr := next(ctx, m) \n\t\t\t// do something after subscribing \n\t\t\treturn err\n\t\t}\n\t}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk-yomo%2Fpm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fk-yomo%2Fpm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk-yomo%2Fpm/lists"}