https://github.com/core-go/pubsub
https://github.com/core-go/pubsub
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/core-go/pubsub
- Owner: core-go
- Created: 2020-06-23T08:17:59.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2024-07-21T14:35:25.000Z (12 months ago)
- Last Synced: 2025-01-10T04:15:05.306Z (6 months ago)
- Language: Go
- Size: 21.5 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# pubsub
A fully managed messaging service that allows for event-driven systems and real-time analytics on Google Cloud Platform. Key features include:
- Scalability: Automatically scales to handle high-throughput workloads.
- Durability: Ensures message delivery with at-least-once delivery guarantees.
- Flexibility: Supports both push and pull delivery models.
- Integration: Easily integrates with other Google Cloud services.
### Libraries for Google Pub/Sub
- GO: [pubsub](https://github.com/core-go/pubsub), to wrap and simplify [pubsub](https://pkg.go.dev/cloud.google.com/go/pubsub). Example is at [go-pubsub-sample](https://github.com/project-samples/go-pubsub-sample)
- nodejs: [google-pubsub](https://www.npmjs.com/package/google-pubsub), to wrap and simplify [@google-cloud/pubsub](https://www.npmjs.com/package/@google-cloud/pubsub). Example is at [pubsub-sample](https://github.com/typescript-tutorial/pubsub-sample)
#### A common flow to consume a message from a message queue
- The libraries to implement this flow are:
- [mq](https://github.com/core-go/mq) for GOLANG. Example is at [go-pubsub-sample](https://github.com/project-samples/go-pubsub-sample)
- [mq-one](https://www.npmjs.com/package/mq-one) for nodejs. Example is at [pubsub-sample](https://github.com/typescript-tutorial/pubsub-sample)
### Use Cases of Google Pub/Sub
Common use cases include event-driven architectures, log collection, and streaming analytics.
#### Event-Driven Architectures
- Scenario: Building applications where different components communicate via events (e.g., microservices)
- Benefit: Decouples components, allowing independent scaling and development
#### Log Collection and Monitoring
- Scenario: Aggregating logs from multiple applications and systems.
- Benefit: Centralized logging and monitoring, improving visibility and debugging capabilities.
#### Streaming analytics
- Scenario: Collecting and analyzing data streams from various sources like IoT devices, social media, or user activity.
- Benefit: Enables real-time data processing and analytics, providing timely insights and actions.
### Comparison of Google Pub/Sub, Amazon SQS, and Apache Kafka
#### Google Pub/Sub:
- Type: Managed real-time messaging service.
- Use Case: Event-driven architectures, real-time analytics.
- Scalability: Automatically scales.
- Delivery Guarantees: At-least-once delivery.
- Integration: Tight with Google Cloud services.
- Delivery Models: Push and pull.
#### Amazon SQS
- Type: Managed message queuing service.
- Use Case: Decoupling and scaling microservices, asynchronous tasks.
- Scalability: Automatically scales.
- Delivery Guarantees: At-least-once, FIFO (exactly-once).
- Integration: Deep integration with AWS services.
- Delivery Models: Primarily pull, with long polling.
#### Apache Kafka
- Type: Open-source event streaming platform.
- Use Case: High-throughput messaging, event sourcing, log aggregation.
- Scalability: High with partitioned topics.
- Delivery Guarantees: Configurable (at-least-once, exactly-once).
- Integration: Broad ecosystem with various connectors.
- Delivery Models: Pull-based consumer groups.
### Key Differences
- Management: Pub/Sub and SQS are managed services, while Kafka is typically self-managed or via managed services like Confluent.
- Use Case Focus: Pub/Sub and Kafka are ideal for real-time processing, whereas SQS is great for decoupling microservices and handling asynchronous tasks.
- Delivery Models: Pub/Sub supports push and pull, SQS supports pull with long polling, and Kafka primarily uses pull with consumer groups.
- Scalability: All three are highly scalable, but Kafka offers the most control over performance tuning.
- Integration: Pub/Sub integrates well with Google Cloud, SQS with AWS, and Kafka has a broad integration ecosystem.
### When to Use
- Google Pub/Sub: If you're using Google Cloud and need a managed, real-time messaging solution.
- Amazon SQS: For reliable, scalable message queuing in AWS environments.
- Apache Kafka: For complex event streaming and log aggregation, with a need for fine-tuned control and a broad integration ecosystem.
## Installation
Please make sure to initialize a Go module before installing core-go/pubsub:
```shell
go get -u github.com/core-go/pubsub
```
Import:
```go
import "github.com/core-go/pubsub"
```