{"id":18039505,"url":"https://github.com/adamluzsi/googlecloudpubsub","last_synced_at":"2025-04-05T01:13:44.892Z","repository":{"id":57529048,"uuid":"83032661","full_name":"adamluzsi/GoogleCloudPubsub","owner":"adamluzsi","description":"Golang Pubsub pipeline consumer for rapid develeopment and testing when building google pubsub pipeline enhancers","archived":false,"fork":false,"pushed_at":"2017-07-24T07:29:30.000Z","size":34,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-10T09:28:54.547Z","etag":null,"topics":["bdd","cloud","gcloud-pubsub","golang","google","pubsub","tdd"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adamluzsi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-02-24T10:59:57.000Z","updated_at":"2018-07-23T09:58:56.000Z","dependencies_parsed_at":"2022-09-26T18:10:58.033Z","dependency_job_id":null,"html_url":"https://github.com/adamluzsi/GoogleCloudPubsub","commit_stats":null,"previous_names":["adamluzsi/gcloud_pubsub"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamluzsi%2FGoogleCloudPubsub","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamluzsi%2FGoogleCloudPubsub/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamluzsi%2FGoogleCloudPubsub/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamluzsi%2FGoogleCloudPubsub/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adamluzsi","download_url":"https://codeload.github.com/adamluzsi/GoogleCloudPubsub/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247271527,"owners_count":20911587,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bdd","cloud","gcloud-pubsub","golang","google","pubsub","tdd"],"created_at":"2024-10-30T14:09:35.537Z","updated_at":"2025-04-05T01:13:44.876Z","avatar_url":"https://github.com/adamluzsi.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"Google Cloud Pubsub\n[![GoDoc](https://godoc.org/github.com/adamluzsi/GoogleCloudPubsub?status.svg)](https://godoc.org/github.com/adamluzsi/GoogleCloudPubsub)\n===================\n\nThis package wraps the google cloud pubsub logic into two seperate part.\nA consumer package and a publisher package.\n\nThe Package provides testing support, so all you have to do is focuse on the business values,\nwithout any fuss of emulated environment and other stuffs like that.\n\n## Dependencies\n\n    go get -u cloud.google.com/go/pubsub\n\n## Install\n\n    go get -u github.com/adamluzsi/GoogleCloudPubsub\n\n# Consumer\n\n## Usage\n\nFirst of all, you need a Handler struct that will implement the business logic for the Gcloud pubsub subscription consuming:\n\n```go\n\ntype ExampleHandler struct {\n    // you can use use fields to store messages if you want for example bulk processing\n\t// messages []consumer.Message\n}\n\n// HandleMessage method will be called after a message had beed fetched from the pubsub.\n// return error will Nack the message\nfunc (eh *ExampleHandler) HandleMessage(msg consumer.Message) error {\n\t// eh.messages = append(eh.messages, msg)\n    // single element processing can be implemented here\n    return nil\n}\n\n// Finish method will be called before all messages should be acked\n// With this method, you can do bulk actions after the HandleMessage Collected all the elements\nfunc (eh *ExampleHandler) Finish() error {\n\tdefer eh.wg.Done()\n    // heavy bulk actions can be implemented here\n\treturn nil\n}\n\n// The return value should be the consumer.Handler interface,\n// not the actual struct pointer\nfunc NewExampleHandler() consumer.Handler {\n    return \u0026ExampleHandler{}\n}\n\n```\n\nNow with your new fancy struct and with it's constructor function, you can begin to use the Consumer\n\n```go\nctx := context.Background()\nc := consumer.New(ctx, \"example-subscription-name\", NewExampleHandler)\n```\n\nIf you want to specify further options for the consumer, you can do so with option setters.\n\n```go\nctx := context.Background()\ncons := consumer.New(ctx, \"example\", NewExampleHandler,\n        // you can configure the new consumer to use given amount of BatchSize\n        // This is the amount that will be passed for the HandleMessage method for a single Handler object\n\t\tconsumer.SetBatchSizeTo(amount),\n\n        // This will set the Google Pubsub Message Iterators MaxExtensionDuration\n\t\tconsumer.SetMaxExtensionDurationTo(10*time.Minute),\n\n        // This will configure the consumer to how manny parallel worker should pull from the subscription\n\t\tconsumer.SetWorkersCountTo(runtime.NumCPU()))\n\n```\n\n## Testing\n\nWhen You test your application, Before the Consumer is being initialized, you should turn on Mock mod.\nWhen Mock mod enabled, not the original but a Mock consumer will be created when the New method called.\nIt's behavior is alike, but remove the Dependency to use Google Pubsub Emulated Host,\nand increase the speed for your tests.\n\nMake even the Benchmarking more valuable\n\n```go\n\nfunc TestConsumerMockingAllPerfect(t *testing.T) {\n\tconsumer.TurnMockModOn()\n\tdefer consumer.TurnMockModOff()\n\n    // consumer creation is the same, and not required to be happen here,\n    // this is just an example , that it should be created after the mock mod enabled\n    ctx := context.Background()\n    c := consumer.New(ctx, \"example-subscription-name\", NewExampleHandler)\n    c.Start()\n    defer c.Stop()\n\n    // And this is how you Send Messages to the Mock Consumer\n    consumer.MockMessageFeeder[\"example-subscription-name\"] \u003c- []byte(`Hello World!`)\n\n    // super complex business logic testing here\n\n}\n\n```\n\n# Publisher\n\n## Usage\n\nNow with your new fancy struct and with it's constructor function, you can begin to use the Consumer\n\n```go\nimport \"github.com/adamluzsi/GoogleCloudPubsub/publisher\"\n//...\n\nctx := context.Background()\np := publisher.New(ctx, \"my-example-topic-name\")\np.Publish([]byte(`hello world!`))\np.Publish([]byte(`hello`), []byte(`world`))\n```\n\n## Testing\n\nPublisher also support mock mod\n\n```go\n\nfunc TestMockedPublishing(t *testing.T) {\n\n    publisher.TurnMockModOn()\n\tdefer publisher.TurnMockModOff()\n\tctx := context.Background()\n\n    p := publisher.New(ctx, TopicName)\n\tp.Publish([]byte(`hello world!`))\n\tp.Publish([]byte(`hello`), []byte(`world`))\n\n\tdatas := make([][]byte, 0, 3)\n\n\tfor i := 0; i \u003c 3; i++ {\n\t\tdata := \u003c- publisher.MockMessageReceiver[TopicName]\n\t\tdatas = append(datas, data)\n\t}\n\n}\n\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamluzsi%2Fgooglecloudpubsub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadamluzsi%2Fgooglecloudpubsub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamluzsi%2Fgooglecloudpubsub/lists"}