{"id":17739025,"url":"https://github.com/pierrekieffer/pubsubtoolbox","last_synced_at":"2025-07-21T07:02:17.762Z","repository":{"id":57562542,"uuid":"257851071","full_name":"PierreKieffer/pubsubToolBox","owner":"PierreKieffer","description":"Google Cloud Platform Pub/Sub toolbox ","archived":false,"fork":false,"pushed_at":"2022-03-31T13:45:35.000Z","size":47,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-21T08:44:52.339Z","etag":null,"topics":["gcp","gcp-pubsub","pubsub","pubsub-interface","pubsub-publisher","pubsub-subscriber"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PierreKieffer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-22T09:17:04.000Z","updated_at":"2023-11-12T00:15:54.000Z","dependencies_parsed_at":"2022-09-17T14:03:54.379Z","dependency_job_id":null,"html_url":"https://github.com/PierreKieffer/pubsubToolBox","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/PierreKieffer/pubsubToolBox","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PierreKieffer%2FpubsubToolBox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PierreKieffer%2FpubsubToolBox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PierreKieffer%2FpubsubToolBox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PierreKieffer%2FpubsubToolBox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PierreKieffer","download_url":"https://codeload.github.com/PierreKieffer/pubsubToolBox/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PierreKieffer%2FpubsubToolBox/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266255243,"owners_count":23900097,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gcp","gcp-pubsub","pubsub","pubsub-interface","pubsub-publisher","pubsub-subscriber"],"created_at":"2024-10-26T02:06:59.410Z","updated_at":"2025-07-21T07:02:17.746Z","avatar_url":"https://github.com/PierreKieffer.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pubsubToolBox \nGoogle Cloud Platform Pub/Sub toolbox \n\n* [Prerequisities](#prerequisities)\n* [Usage](#usage)\n\t* [Publisher](#publisher)\n\t* [Consumer](#consumer)\n\n\n## Prerequisities \n- GCP project with a Pub/Sub up \u0026 running. \n- GCP service account and the credential file _private_key.json_ associated.\n- GCP root package \"cloud.google.com/go\" (```go get -u cloud.google.com/go/pubsub```) \n\n\n## Usage\n \n```bash\ngo get github.com/PierreKieffer/pubsubToolBox\n```\n\n### Publisher \nTo publish messages : \n\n```go\nimport (\n        \"context\"\n        \"github.com/PierreKieffer/pubsubToolBox/client\"\n        \"github.com/PierreKieffer/pubsubToolBox/publisher\"\n        \"log\"\n)\n\nfunc main() {\n\n        projectID := \"PROJECT_ID\"\n\n        // If GOOGLE_APPLICATION_CREDENTIALS is set :\n        pubsubClient, err := client.InitPubSubClient(ctx, projectID)\n\n        // Else you need to pass the json key                                                                         \n        pubsubClient, err := client.InitPubSubClient(ctx, projectID, \"private_key.json\")\n        if err != nil {\n                log.Println(err)\n        }   \n\n\tp := publisher.Publisher{\n\t\tContext : context.Background(),\n\t\tTopicID : \"TOPIC_ID\",\n\t\tPubSubClient : pubsubClient\n\t}\n\n        message := `{\"Message\" : \"Hello world\"}`\n\tattributes := map[string]string{\"foo\" : \"bar\"}\n        p.Publish(message, attributes)\n}\n\n\n```\n\n\n### Consumer \nTo consume messages : \n\nA message buffer must be instantiated in order to store messages at the application level.\n\nThe consumer.Pull method receives messages from the Pub/Sub broker and adds them to the message buffer.\n\nThe message structure is : \n```go\ntype Message struct {\n\tData       string\n\tAttributes map[string]string\n}\n```\n\nThis allows consumption of the Pub/Sub broker and processing of messages in parallel.\n\nThe initialization takes the size of the buffer as a parameter :\n\n```go \nvar buffer = consumer.InitBuffer(10)\n```\n\nOnce a message is consumed, it is dropped from the buffer. \n\nConsume messages from Pub/Sub broker : \n\nIf the subscriber doesn't exist, it will be created. \n\nInstanciate first a consumer.Consumer : \n```go\n\tc := \u0026consumer.Consumer{\n\t\tContext:        context.Background(),\n\t\tPubSubClient:   pubsubClient,\n\t\tSubscriberName: \"subscriber-name\",\n\t\tTopicID:        \"topic-id\",\n\t\tBuffer:         buffer,\n\t}\n```\n\nStart message consumption : \n```go \n\tgo c.Pull()\n```\nThe message buffer is a classic channel, It can be consumed through a goroutine : \n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"log\"\n\n\t\"github.com/PierreKieffer/pubsubToolBox/client\"\n\t\"github.com/PierreKieffer/pubsubToolBox/consumer\"\n)\n\nvar exit = make(chan bool)\n\nfunc main() {\n\n\tprojectID := \"test-project-name\"\n\n\tpubsubClient, _ := client.InitPubSubClient(ctx, projectID, \"private_key.json\")\n\n\t// Init message buffer to receive pulled messages\n\tvar buffer = consumer.InitBuffer(10)\n\n\tc := \u0026consumer.Consumer{\n\t\tContext:        context.Background(),\n\t\tPubSubClient:   pubsubClient,\n\t\tSubscriberName: \"subscriber-name\",\n\t\tTopicID:        \"topic-id\",\n\t\tBuffer:         buffer,\n\t}\n\n\t// Launch local buffer consumer to process messages\n\tgo ProcessBuffer(buffer)\n\n\t// Launch the pubsub consumer to pull messages\n\tgo c.Pull()\n\n\t\u003c-exit\n}\n\nfunc ProcessBuffer(messageBuffer chan consumer.Message) {\n\tfor {\n\t\t// ... Process received messages\n\t\tlog.Println(\"Message consumed : \", \u003c-messageBuffer)\n\t}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpierrekieffer%2Fpubsubtoolbox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpierrekieffer%2Fpubsubtoolbox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpierrekieffer%2Fpubsubtoolbox/lists"}