https://github.com/core-go/sqs
https://github.com/core-go/sqs
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/core-go/sqs
- Owner: core-go
- Created: 2020-06-20T14:07:37.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2024-07-21T10:14:08.000Z (11 months ago)
- Last Synced: 2025-01-10T04:15:05.005Z (6 months ago)
- Language: Go
- Size: 15.6 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# sqs
A fully managed message queue service offered by AWS. It provides a reliable, scalable, and cost-effective way to decouple and coordinate distributed software systems and microservices.
### Libraries for Amazon SQS (Simple Queue Service)
- GO: [sqs](https://github.com/core-go/sqs), to wrap and simplify [aws-sdk-go/service/sqs](https://github.com/aws/aws-sdk-go/tree/main/service/sqs). Example is at [go-amazon-sqs-sample](https://github.com/project-samples/go-amazon-sqs-sample)
#### A common flow to consume a message from a message queue
- The libraries to implement this flow are:
- [mq](https://github.com/core-go/mq) for GOLANG. Example is at [go-amazon-sqs-sample](https://github.com/project-samples/go-amazon-sqs-sample)
### Use Cases of Amazon SQS (Simple Queue Service)
#### Decoupling Microservices
- Scenario: Separating different parts of an application to ensure that a failure in one part does not affect others.
- Benefit: Enhances fault tolerance and scalability by allowing asynchronous communication between services.
#### Asynchronous Processing
- Scenario: Handling tasks that do not need immediate processing, such as batch processing or background tasks.
- Benefit: Improves system efficiency and response times for end-users.
#### Job Queuing
- Scenario: Managing and distributing jobs to worker processes.
- Benefit: Balances load and ensures all tasks are completed without overloading any single worker.
#### Order Processing Systems
- Scenario: Processing customer orders, where each order can be handled as a separate task.
- Benefit: Ensures reliable and scalable processing of orders, even during high demand.
#### Message Buffering
- Scenario: Smoothing out bursty traffic in applications to prevent overload.
- Benefit: Protects the system from spikes in traffic by buffering messages.
#### Workflow Orchestration
- Scenario: Orchestrating steps in a complex workflow, such as image processing pipelines.
- Benefit: Coordinates different stages of processing in a reliable and scalable manner.
## Comparison of Amazon SQS, Google Pub/Sub and Apache Kafka
#### Amazon SQS
- Type: Managed message queuing service.
- Use Case: Decoupling and scaling microservices, asynchronous tasks.
- Scalability: Automatically scales.
- Delivery Guarantees: At-least-once, FIFO (exactly-once).
- Integration: Deep integration with AWS services.
- Delivery Models: Primarily pull, with long polling.
#### Google Pub/Sub:
- Type: Managed real-time messaging service.
- Use Case: Event-driven architectures, real-time analytics.
- Scalability: Automatically scales.
- Delivery Guarantees: At-least-once delivery.
- Integration: Tight with Google Cloud services.
- Delivery Models: Push and pull.
#### Apache Kafka
- Type: Open-source event streaming platform.
- Use Case: High-throughput messaging, event sourcing, log aggregation.
- Scalability: High with partitioned topics.
- Delivery Guarantees: Configurable (at-least-once, exactly-once).
- Integration: Broad ecosystem with various connectors.
- Delivery Models: Pull-based consumer groups.
### Key Differences
- Management: Pub/Sub and SQS are managed services, while Kafka is typically self-managed or via managed services like Confluent.
- Use Case Focus: Pub/Sub and Kafka are ideal for real-time processing, whereas SQS is great for decoupling microservices and handling asynchronous tasks.
- Delivery Models: Pub/Sub supports push and pull, SQS supports pull with long polling, and Kafka primarily uses pull with consumer groups.
- Scalability: All three are highly scalable, but Kafka offers the most control over performance tuning.
- Integration: Pub/Sub integrates well with Google Cloud, SQS with AWS, and Kafka has a broad integration ecosystem.
## Installation
Please make sure to initialize a Go module before installing core-go/sqs:
```shell
go get -u github.com/core-go/sqs
```
Import:
```go
import "github.com/core-go/sqs"
```