{"id":17002271,"url":"https://github.com/bxcodec/goqueue","last_synced_at":"2026-02-22T22:08:30.046Z","repository":{"id":243549731,"uuid":"769027617","full_name":"bxcodec/goqueue","owner":"bxcodec","description":"GoQueue - One library to rule them all. A Golang wrapper that handles all the complexity of various Queue platforms. Extensible and easy to learn.","archived":false,"fork":false,"pushed_at":"2024-12-09T00:42:52.000Z","size":335,"stargazers_count":6,"open_issues_count":4,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-18T13:32:03.211Z","etag":null,"topics":["go-queue","golang","google-pubsub","googlepubsub","goqueue","rabbitmq","rabbitmq-consumer","sns","sns-producer","sqs","sqs-consumer"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bxcodec.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-08T07:36:14.000Z","updated_at":"2024-08-11T16:05:25.000Z","dependencies_parsed_at":"2024-06-19T02:59:05.214Z","dependency_job_id":"458dcd59-0200-4985-83ab-1788a7d93186","html_url":"https://github.com/bxcodec/goqueue","commit_stats":null,"previous_names":["bxcodec/goqueue"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bxcodec%2Fgoqueue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bxcodec%2Fgoqueue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bxcodec%2Fgoqueue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bxcodec%2Fgoqueue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bxcodec","download_url":"https://codeload.github.com/bxcodec/goqueue/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244986244,"owners_count":20542980,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go-queue","golang","google-pubsub","googlepubsub","goqueue","rabbitmq","rabbitmq-consumer","sns","sns-producer","sqs","sqs-consumer"],"created_at":"2024-10-14T04:27:31.105Z","updated_at":"2026-02-22T22:08:30.017Z","avatar_url":"https://github.com/bxcodec.png","language":"Go","readme":"# 🚀 GoQueue - Universal Go Message Queue Library\n\n[![Go Reference](https://pkg.go.dev/badge/github.com/bxcodec/goqueue.svg)](https://pkg.go.dev/github.com/bxcodec/goqueue)\n[![Go Report Card](https://goreportcard.com/badge/github.com/bxcodec/goqueue)](https://goreportcard.com/report/github.com/bxcodec/goqueue)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/bxcodec/goqueue/blob/main/LICENSE)\n[![GitHub stars](https://img.shields.io/github/stars/bxcodec/goqueue)](https://github.com/bxcodec/goqueue/stargazers)\n\n**One library to rule them all** - A powerful, extensible, and developer-friendly Go wrapper that simplifies message queue operations across multiple platforms. Build robust, scalable applications with consistent queue operations, regardless of your underlying message broker.\n\n## ✨ Why GoQueue?\n\n![Core Concept](misc/images/core-concept.png)\n\n🎯 **Universal Interface** - Write once, run anywhere. Switch between queue providers without changing your code  \n⚡ **Production Ready** - Built-in retry mechanisms, dead letter queues, and error handling  \n🛡️ **Type Safe** - Strongly typed interfaces with comprehensive error handling  \n🔧 **Extensible** - Plugin architecture for custom middleware and queue providers  \n📊 **Observable** - Built-in logging and middleware support for monitoring  \n🚀 **Developer Experience** - Intuitive API design with sensible defaults\n\n---\n\n## 📋 Table of Contents\n\n- [🚀 Quick Start](#-quick-start)\n- [💫 Features](#-features)\n- [🛠️ Installation](#️-installation)\n- [📖 Basic Usage](#-basic-usage)\n- [🔧 Advanced Features](#-advanced-features)\n- [🎮 Examples](#-examples)\n- [🏗️ Architecture](#️-architecture)\n- [📚 Documentation](#-documentation)\n- [🤝 Contributing](#-contributing)\n- [📄 License](#-license)\n\n---\n\n## 🚀 Quick Start\n\nGet up and running in less than 5 minutes:\n\n```bash\ngo get -u github.com/bxcodec/goqueue\n```\n\n```go\npackage main\n\nimport (\n\t\"context\"\n    \"log\"\n\n\t\"github.com/bxcodec/goqueue\"\n\t\"github.com/bxcodec/goqueue/consumer\"\n    \"github.com/bxcodec/goqueue/publisher\"\n\t\"github.com/bxcodec/goqueue/interfaces\"\n)\n\nfunc main() {\n\n\t// Create queue service\n    queueSvc := goqueue.NewQueueService(\n        options.WithConsumer(myConsumer),\n        options.WithPublisher(myPublisher),\n        options.WithMessageHandler(handleMessage),\n    )\n\n    // Publish a message\n    queueSvc.Publish(context.Background(), interfaces.Message{\n        Data:   map[string]interface{}{\"hello\": \"world\"},\n        Action: \"user.created\",\n        Topic:  \"users\",\n    })\n\n    // Start consuming\n    queueSvc.Start(context.Background())\n}\n\nfunc handleMessage(ctx context.Context, m interfaces.InboundMessage) error {\n    log.Printf(\"Received: %v\", m.Data)\n    return m.Ack(ctx) // Acknowledge successful processing\n}\n```\n\n---\n\n## 💫 Features\n\n### 🎯 **Core Features**\n\n- **Multi-Provider Support**: Currently supports RabbitMQ (more coming soon!)\n- **Unified API**: Consistent interface across all queue providers\n- **Type Safety**: Strongly typed message structures\n- **Context Support**: Full Go context integration for cancellation and timeouts\n\n### 🛡️ **Reliability \u0026 Resilience**\n\n- **Automatic Retries**: Configurable retry mechanisms with exponential backoff\n- **Dead Letter Queues**: Handle failed messages gracefully\n- **Circuit Breaker**: Built-in protection against cascading failures\n- **Graceful Shutdown**: Clean resource cleanup on application termination\n\n### 🔧 **Advanced Capabilities**\n\n- **Middleware System**: Extensible pipeline for message processing\n- **Custom Serialization**: Support for JSON, Protocol Buffers, and custom formats\n- **Message Routing**: Flexible topic and routing key patterns\n- **Batching**: Efficient batch message processing\n- **Connection Pooling**: Optimized connection management\n\n### 📊 **Observability**\n\n- **Structured Logging**: Built-in zerolog integration\n- **Metrics Ready**: Hooks for Prometheus, StatsD, and custom metrics\n- **Tracing Support**: OpenTelemetry compatible\n- **Health Checks**: Built-in health check endpoints\n\n---\n\n## 🛠️ Installation\n\n```bash\n# Install the core library\ngo get -u github.com/bxcodec/goqueue\n```\n\n### Requirements\n\n- Go 1.21 or higher\n- Message broker (RabbitMQ supported, more coming soon)\n\n---\n\n## 📖 Basic Usage\n\n### 🚀 Publisher Example\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"github.com/bxcodec/goqueue/publisher\"\n    publisherOpts \"github.com/bxcodec/goqueue/options/publisher\"\n    amqp \"github.com/rabbitmq/amqp091-go\"\n)\n\nfunc main() {\n    // Connect to RabbitMQ\n    conn, _ := amqp.Dial(\"amqp://localhost:5672/\")\n\n    // Create publisher\n    pub := publisher.NewPublisher(\n\t\tpublisherOpts.PublisherPlatformRabbitMQ,\n\t\tpublisherOpts.WithRabbitMQPublisherConfig(\u0026publisherOpts.RabbitMQPublisherConfig{\n            Conn:                     conn,\n\t\t\tPublisherChannelPoolSize: 5,\n\t\t}),\n        publisherOpts.WithPublisherID(\"my-service\"),\n    )\n\n    // Publish message\n    err := pub.Publish(context.Background(), interfaces.Message{\n        Data:   map[string]interface{}{\"user_id\": 123, \"action\": \"signup\"},\n        Action: \"user.created\",\n        Topic:  \"users\",\n    })\n\tif err != nil {\n        log.Fatal(err)\n    }\n}\n```\n\n### 📨 Consumer Example\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"github.com/bxcodec/goqueue/consumer\"\n    consumerOpts \"github.com/bxcodec/goqueue/options/consumer\"\n)\n\nfunc main() {\n    // Create consumer\n    cons := consumer.NewConsumer(\n\t\tconsumerOpts.ConsumerPlatformRabbitMQ,\n        consumerOpts.WithQueueName(\"user-events\"),\n\t\tconsumerOpts.WithMaxRetryFailedMessage(3),\n        consumerOpts.WithBatchMessageSize(10),\n    )\n\n    // Start consuming\n    cons.Consume(context.Background(), messageHandler, metadata)\n}\n\nfunc messageHandler(ctx context.Context, msg interfaces.InboundMessage) error {\n    // Process your message\n    userData := msg.Data.(map[string]interface{})\n\n    // Business logic here\n    if err := processUser(userData); err != nil {\n        // Retry with exponential backoff\n        return msg.RetryWithDelayFn(ctx, interfaces.ExponentialBackoffDelayFn)\n    }\n\n    // Acknowledge successful processing\n    return msg.Ack(ctx)\n}\n```\n\n---\n\n## 🔧 Advanced Features\n\n### 🔄 Retry Mechanisms\n\nGoQueue provides sophisticated retry mechanisms with multiple strategies:\n\n```go\n// Exponential backoff retry\nreturn msg.RetryWithDelayFn(ctx, interfaces.ExponentialBackoffDelayFn)\n\n// Custom retry logic\nreturn msg.RetryWithDelayFn(ctx, func(retryCount int64) int64 {\n    return retryCount * 2 // Custom delay calculation\n})\n\n// Move to dead letter queue after max retries\nreturn msg.MoveToDeadLetterQueue(ctx)\n```\n\n### 🔌 Middleware System\n\nExtend functionality with custom middleware:\n\n```go\n// Custom logging middleware\nfunc LoggingMiddleware() interfaces.InboundMessageHandlerMiddlewareFunc {\n    return func(next interfaces.InboundMessageHandlerFunc) interfaces.InboundMessageHandlerFunc {\n        return func(ctx context.Context, m interfaces.InboundMessage) error {\n            start := time.Now()\n            err := next(ctx, m)\n            log.Printf(\"Message processed in %v\", time.Since(start))\n            return err\n        }\n    }\n}\n\n// Apply middleware\ncons := consumer.NewConsumer(\n    consumerOpts.ConsumerPlatformRabbitMQ,\n    consumerOpts.WithMiddlewares(\n        LoggingMiddleware(),\n        MetricsMiddleware(),\n        AuthMiddleware(),\n    ),\n)\n```\n\n### 🎛️ Configuration Options\n\nFine-tune your queue behavior:\n\n```go\ncons := consumer.NewConsumer(\n    consumerOpts.ConsumerPlatformRabbitMQ,\n    consumerOpts.WithQueueName(\"high-priority-queue\"),\n    consumerOpts.WithMaxRetryFailedMessage(5),\n    consumerOpts.WithBatchMessageSize(50),\n    consumerOpts.WithConsumerID(\"worker-01\"),\n    consumerOpts.WithRabbitMQConsumerConfig(\u0026consumerOpts.RabbitMQConsumerConfig{\n        ConsumerChannel: channel,\n        ReQueueChannel:  requeueChannel,\n        QueueDeclareConfig: \u0026consumerOpts.RabbitMQQueueDeclareConfig{\n            Durable:    true,\n            AutoDelete: false,\n            Exclusive:  false,\n        },\n    }),\n)\n```\n\n---\n\n## 🎮 Examples\n\n### 📁 Complete Examples\n\nExplore our comprehensive examples:\n\n- **[Basic Usage](examples/rabbitmq/basic/)** - Simple publish/consume example\n- **[With Retries](examples/rabbitmq/withretries/)** - Advanced retry mechanisms\n\n### 🐰 RabbitMQ Quick Setup\n\nStart RabbitMQ with Docker:\n\n```bash\n# Clone the repository\ngit clone https://github.com/bxcodec/goqueue.git\ncd goqueue/examples/rabbitmq/basic\n\n# Start RabbitMQ\ndocker-compose up -d\n\n# Run the example\ngo run main.go\n```\n\n### 🔄 Retry Architecture\n\n![GoQueue Retry Architecture](misc/images/rabbitmq-retry.png)\n\n_Automatic retry mechanism with exponential backoff and dead letter queue_\n\n---\n\n## 🏗️ Architecture\n\n### 🎯 Design Principles\n\n- **Interface Segregation**: Clean, focused interfaces for different responsibilities\n- **Dependency Injection**: Easy testing and swappable implementations\n- **Error Handling**: Comprehensive error types and recovery mechanisms\n- **Performance**: Optimized for high-throughput scenarios\n- **Extensibility**: Plugin architecture for custom providers and middleware\n\n### 🧩 Core Components\n\n![Core Concept](misc/images/core-concept.png)\n\n### 📦 Provider Support\n\n| Provider       | Status          | Features             |\n| -------------- | --------------- | -------------------- |\n| RabbitMQ       | 🔄 Beta Version | Full feature support |\n| Google Pub/Sub | 📋 Planned      | Coming soon          |\n| AWS SQS + SNS  | 📋 Planned      | Coming soon          |\n\n---\n\n## 🔧 Configuration\n\n### 📝 Logging Setup\n\nGoQueue uses structured logging with zerolog:\n\n```go\nimport \"github.com/bxcodec/goqueue\"\n\n// Setup basic logging (automatic when importing consumer/publisher)\n// OR setup with custom configuration:\ngoqueue.SetupLoggingWithDefaults() // Pretty console output for development\n```\n\n---\n\n## 🧪 Testing\n\nRun the test suite:\n\n```bash\n# Unit tests\nmake test\n\n# Integration tests with RabbitMQ\nmake integration-test\n\n```\n\n---\n\n## 📚 Documentation\n\n### 📖 Component Documentation\n\nExplore our comprehensive guides for each system component:\n\n| Component             | Description                                 | Documentation                                   |\n| --------------------- | ------------------------------------------- | ----------------------------------------------- |\n| 🔌 **Middleware**     | Extend functionality with custom logic      | [📖 Middleware Guide](docs/MIDDLEWARE.md)       |\n| 📨 **Consumer**       | Reliable message consumption and processing | [📖 Consumer Guide](docs/CONSUMER.md)           |\n| 📤 **Publisher**      | High-performance message publishing         | [📖 Publisher Guide](docs/PUBLISHER.md)         |\n| 🔄 **RabbitMQ Retry** | Advanced retry mechanisms for RabbitMQ      | [📖 Retry Architecture](docs/RABBITMQ-RETRY.md) |\n\n### 🎯 Quick Links\n\n- **[📚 Full Documentation Index](docs/README.md)** - Complete documentation overview\n- **[🔧 API Reference](https://pkg.go.dev/github.com/bxcodec/goqueue)** - Go package documentation\n- **[🎮 Examples](examples/)** - Working code examples\n- **[🐛 Troubleshooting](docs/README.md#troubleshooting)** - Common issues and solutions\n\n---\n\n## 🤝 Contributing\n\nWe welcome contributions! Here's how to get started:\n\n### 🚀 Quick Contribution Guide\n\n1. **Fork** the repository\n2. **Create** a feature branch (`git checkout -b feature/amazing-feature`)\n3. **Commit** your changes (`git commit -m 'Add amazing feature'`)\n4. **Push** to the branch (`git push origin feature/amazing-feature`)\n5. **Open** a Pull Request\n\n### 📋 Development Setup\n\n```bash\n# Clone your fork\ngit clone https://github.com/yourusername/goqueue.git\ncd goqueue\n\n# Install dependencies\ngo mod download\n\n# Run tests\nmake test\n\n# Run linting\nmake lint\n\n```\n\n### 🎯 Contribution Areas\n\n- 🔌 **New Queue Providers** (Google Pub/Sub, SQS+SNS)\n- 🛠️ **Middleware Components** (Metrics, Tracing, Auth)\n- 📚 **Documentation \u0026 Examples**\n- 🧪 **Testing \u0026 Benchmarks**\n- 🐛 **Bug Fixes \u0026 Improvements**\n\n---\n\n## 📞 Support \u0026 Community\n\n- 📖 **Documentation**: [pkg.go.dev/github.com/bxcodec/goqueue](https://pkg.go.dev/github.com/bxcodec/goqueue)\n- 🐛 **Issues**: [GitHub Issues](https://github.com/bxcodec/goqueue/issues)\n- 📧 **Email**: [iman@tumorang.com](mailto:iman@tumorang.com)\n\n---\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n---\n\n## 🙏 Acknowledgments\n\n- Thanks to all [contributors](https://github.com/bxcodec/goqueue/contributors)\n- Inspired by the Go community's best practices\n- Built with ❤️ for the Go ecosystem\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n[🚀 Get Started](#-quick-start) • [📖 Documentation](https://pkg.go.dev/github.com/bxcodec/goqueue) • [🤝 Contribute](#-contributing)\n\n\u003c/div\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbxcodec%2Fgoqueue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbxcodec%2Fgoqueue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbxcodec%2Fgoqueue/lists"}