{"id":27135516,"url":"https://github.com/iamnilotpal/pubsub","last_synced_at":"2025-04-10T00:07:43.526Z","repository":{"id":285965584,"uuid":"959920795","full_name":"iamNilotpal/pubsub","owner":"iamNilotpal","description":"PubSub implementation using golang channels.","archived":false,"fork":false,"pushed_at":"2025-04-07T15:22:47.000Z","size":35,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-10T00:07:26.664Z","etag":null,"topics":["channels","go","golang","goroutine","publisher","pubsub","subscriber"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iamNilotpal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-04-03T15:07:43.000Z","updated_at":"2025-04-07T15:22:51.000Z","dependencies_parsed_at":"2025-04-03T16:25:21.321Z","dependency_job_id":"3f4e068d-055d-454a-803c-d338a924eee6","html_url":"https://github.com/iamNilotpal/pubsub","commit_stats":null,"previous_names":["iamnilotpal/pubsub"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamNilotpal%2Fpubsub","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamNilotpal%2Fpubsub/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamNilotpal%2Fpubsub/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamNilotpal%2Fpubsub/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iamNilotpal","download_url":"https://codeload.github.com/iamNilotpal/pubsub/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248131317,"owners_count":21052819,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["channels","go","golang","goroutine","publisher","pubsub","subscriber"],"created_at":"2025-04-08T01:48:26.424Z","updated_at":"2025-04-10T00:07:43.484Z","avatar_url":"https://github.com/iamNilotpal.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Go PubSub\n\nA lightweight, in-memory Publish-Subscribe (PubSub) system written in Go\nleveraging Go's powerful concurrency model with channels. This library enables\nmultiple subscribers to listen to topics and receive messages asynchronously.\n\n## Features\n\n- Simple and efficient PubSub implementation built on Go's concurrency\n  primitives.\n- Support for multiple subscribers per topic.\n- Structured message delivery with topic information included.\n- Thread-safe operations with proper locking mechanisms.\n- Graceful shutdown capabilities for safe channel closure.\n- Generic support for any payload type.\n\n## Installation\n\n```sh\ngo get github.com/iamNilotpal/pubsub\n```\n\n## Implementation Details\n\n### Message Struct\n\nThe `Message` struct provides a structured way to receive messages with context:\n\n```go\ntype Message[T any] struct {\n\tTopic   string // The topic this message belongs to.\n\tPayload T      // The actual message payload.\n}\n```\n\nThis allows subscribers to filter or route messages based on topic information,\neven when listening to multiple topics. The generic type parameter `T` enables\nyou to use any type as the payload.\n\n### PubSub Methods\n\n#### `New[T any]()`\n\nCreates a new PubSub instance with proper initialization for the specified\npayload type.\n\n#### `Subscribe(topic string) (\u003c-chan *Message[T], error)`\n\nSubscribes to a topic and returns a channel that will receive messages published\nto that topic.\n\n#### `Publish(topic string, msg T) error`\n\nPublishes a message to the specified topic. Returns an error if the topic\ndoesn't exist or if the PubSub system is closed.\n\n#### `Close() error`\n\nCloses the PubSub system, shutting down all subscription channels.\n\n### Configuration\n\nYou can configure the PubSub system using options:\n\n```go\nps := pubsub.New[string](pubsub.WithChannelSize(10))\n```\n\nThis sets the buffer size of subscriber channels to 10 and creates a PubSub\ninstance that works with string payloads.\n\n## Usage Examples\n\n### Basic Example\n\nThis example demonstrates the core functionality of subscribing to topics,\npublishing messages, and handling graceful shutdown:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\n\t\"github.com/iamNilotpal/pubsub\"\n)\n\nfunc main() {\n\t// Create a PubSub instance that works with string payloads\n\tps := pubsub.New[string](pubsub.WithChannelSize(5))\n\tvar wg sync.WaitGroup\n\n\t// Subscribe to different topics\n\tdevopsChan, err := ps.Subscribe(\"devops\")\n\tif err != nil {\n\t\tfmt.Println(\"Error subscribing to devops:\", err)\n\t\treturn\n\t}\n\n\tgolangChan, err := ps.Subscribe(\"golang\")\n\tif err != nil {\n\t\tfmt.Println(\"Error subscribing to golang:\", err)\n\t\treturn\n\t}\n\n\t// Create goroutines to handle messages\n\twg.Add(2)\n\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tfor msg := range devopsChan {\n\t\t\tfmt.Printf(\"[%s]: %s\\n\", msg.Topic, msg.Payload)\n\t\t}\n\t\tfmt.Println(\"DevOps channel closed\")\n\t}()\n\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tfor msg := range golangChan {\n\t\t\tfmt.Printf(\"[%s]: %s\\n\", msg.Topic, msg.Payload)\n\t\t}\n\t\tfmt.Println(\"Golang channel closed\")\n\t}()\n\n\t// Publish messages to topics\n\tif err := ps.Publish(\"golang\", \"Go is great for concurrency!\"); err != nil {\n\t\tfmt.Println(\"Error publishing to golang:\", err)\n\t}\n\tif err := ps.Publish(\"devops\", \"CI/CD pipelines automate deployments.\"); err != nil {\n\t\tfmt.Println(\"Error publishing to devops:\", err)\n\t}\n\n\t// Close PubSub system\n\tif err := ps.Close(); err != nil {\n\t\tfmt.Println(\"Error closing pubsub:\", err)\n\t}\n\n\twg.Wait()\n}\n```\n\n### Using Custom Types\n\nThis example demonstrates using a custom struct as the payload type:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/iamNilotpal/pubsub\"\n)\n\n// Define a custom message type\ntype EventData struct {\n\tTimestamp time.Time\n\tSeverity  string\n\tContent   string\n}\n\nfunc main() {\n\t// Create a PubSub instance for EventData type\n\tps := pubsub.New[EventData]()\n\tvar wg sync.WaitGroup\n\n\t// Subscribe to events topic\n\teventsChan, err := ps.Subscribe(\"events\")\n\tif err != nil {\n\t\tfmt.Println(\"Error subscribing to events:\", err)\n\t\treturn\n\t}\n\n\t// Process received events\n\twg.Add(1)\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tfor event := range eventsChan {\n\t\t\tevt := event.Payload\n\t\t\tfmt.Printf(\"[%s] %s: %s (at %s)\\n\",\n\t\t\t\tevent.Topic,\n\t\t\t\tevt.Severity,\n\t\t\t\tevt.Content,\n\t\t\t\tevt.Timestamp.Format(time.RFC3339),\n\t\t\t)\n\t\t}\n\t\tfmt.Println(\"Events channel closed\")\n\t}()\n\n\t// Publish events\n\tps.Publish(\"events\", EventData{\n\t\tTimestamp: time.Now(),\n\t\tSeverity:  \"INFO\",\n\t\tContent:   \"System startup completed\",\n\t})\n\n\tps.Publish(\"events\", EventData{\n\t\tTimestamp: time.Now(),\n\t\tSeverity:  \"WARNING\",\n\t\tContent:   \"High memory usage detected\",\n\t})\n\n\t// Close PubSub system after a short delay\n\ttime.Sleep(time.Second)\n\tps.Close()\n\twg.Wait()\n}\n```\n\n### HTTP Server Example\n\nThis example demonstrates how to use the PubSub system within an HTTP server to\npush real-time updates:\n\n```go\npackage main\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/iamNilotpal/pubsub\"\n)\n\n// Define a message structure for updates\ntype UpdateMessage struct {\n\tContent   string    `json:\"content\"`\n\tTimestamp time.Time `json:\"timestamp\"`\n}\n\nfunc main() {\n\tps := pubsub.New[UpdateMessage]()\n\tvar wg sync.WaitGroup\n\n\t// Handler function that subscribes clients to the \"updates\" topic\n\thttp.HandleFunc(\"/updates\", func(w http.ResponseWriter, r *http.Request) {\n\t\t// Set headers for server-sent events\n\t\tw.Header().Set(\"Content-Type\", \"text/event-stream\")\n\t\tw.Header().Set(\"Cache-Control\", \"no-cache\")\n\t\tw.Header().Set(\"Connection\", \"keep-alive\")\n\n\t\t// Subscribe to updates topic\n\t\tch, err := ps.Subscribe(\"updates\")\n\t\tif err != nil {\n\t\t\thttp.Error(w, \"Subscription error: \"+err.Error(), http.StatusInternalServerError)\n\t\t\treturn\n\t\t}\n\n\t\t// Detect client disconnection\n\t\tnotify := r.Context().Done()\n\t\tgo func() {\n\t\t\t\u003c-notify\n\t\t\tfmt.Println(\"Client disconnected\")\n\t\t}()\n\n\t\t// Stream messages as they arrive\n\t\tfor msg := range ch {\n\t\t\t// Create a response object\n\t\t\tresponse := struct {\n\t\t\t\tTopic     string       `json:\"topic\"`\n\t\t\t\tContent   string       `json:\"content\"`\n\t\t\t\tTimestamp time.Time    `json:\"timestamp\"`\n\t\t\t}{\n\t\t\t\tTopic:     msg.Topic,\n\t\t\t\tContent:   msg.Payload.Content,\n\t\t\t\tTimestamp: msg.Payload.Timestamp,\n\t\t\t}\n\n\t\t\t// Convert to JSON\n\t\t\teventData, _ := json.Marshal(response)\n\t\t\tfmt.Fprintf(w, \"data: %s\\n\\n\", eventData)\n\t\t\tw.(http.Flusher).Flush()\n\t\t}\n\t})\n\n\t// Start HTTP server\n\tgo func() {\n\t\tfmt.Println(\"Server started at http://localhost:8080/updates\")\n\t\thttp.ListenAndServe(\":8080\", nil)\n\t}()\n\n\t// Simulate a background process publishing updates\n\twg.Add(1)\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tfor i := 1; i \u003c= 5; i++ {\n\t\t\tps.Publish(\"updates\", UpdateMessage{\n\t\t\t\tContent:   fmt.Sprintf(\"System update %d: Processing completed\", i),\n\t\t\t\tTimestamp: time.Now(),\n\t\t\t})\n\t\t\ttime.Sleep(2 * time.Second)\n\t\t}\n\t\tps.Close()\n\t}()\n\n\twg.Wait()\n}\n```\n\n### Complex Example: Multi-Topic Monitoring System\n\nThis example showcases a more comprehensive implementation with multiple topics\nand different message types:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/iamNilotpal/pubsub\"\n)\n\n// Define different event types\ntype BaseEvent struct {\n\tTimestamp time.Time\n\tSource    string\n}\n\ntype SystemEvent struct {\n\tBaseEvent\n\tAction string\n\tStatus string\n}\n\ntype SecurityEvent struct {\n\tBaseEvent\n\tIPAddress string\n\tUsername  string\n\tAction    string\n}\n\ntype PerformanceMetric struct {\n\tBaseEvent\n\tMetric string\n\tValue  float64\n\tUnit   string\n}\n\ntype ErrorEvent struct {\n\tBaseEvent\n\tCode        int\n\tDescription string\n\tSeverity    string\n}\n\ntype Event any\n\nfunc main() {\n\t// Create PubSub with Event to handle different event types\n\tps := pubsub.New[Event]()\n\tvar wg sync.WaitGroup\n\n\t// Define topics\n\ttopics := []string{\"system\", \"security\", \"performance\", \"errors\"}\n\n\t// Subscribe to all topics\n\ttopicChannels := make(map[string]\u003c-chan *pubsub.Message[Event])\n\n\tfor _, topic := range topics {\n\t\tch, err := ps.Subscribe(topic)\n\t\tif err != nil {\n\t\t\tfmt.Printf(\"Failed to subscribe to %s: %v\\n\", topic, err)\n\t\t\tcontinue\n\t\t}\n\t\ttopicChannels[topic] = ch\n\t}\n\n\t// Process messages from each topic\n\tfor topic, ch := range topicChannels {\n\t\twg.Add(1)\n\t\tgo func(topic string, msgChan \u003c-chan *pubsub.Message[Event]) {\n\t\t\tdefer wg.Done()\n\n\t\t\tfor msg := range msgChan {\n\t\t\t\ttimestamp := time.Now().Format(\"15:04:05\")\n\n\t\t\t\tswitch topic {\n\t\t\t\tcase \"system\":\n\t\t\t\t\tif evt, ok := msg.Payload.(SystemEvent); ok {\n\t\t\t\t\t\tfmt.Printf(\"🖥️ [%s] SYSTEM: %s - %s (%s)\\n\",\n\t\t\t\t\t\t\ttimestamp, evt.Action, evt.Status, evt.Source)\n\t\t\t\t\t}\n\n\t\t\t\tcase \"security\":\n\t\t\t\t\tif evt, ok := msg.Payload.(SecurityEvent); ok {\n\t\t\t\t\t\tfmt.Printf(\"🔒 [%s] SECURITY: %s by %s from %s (%s)\\n\",\n\t\t\t\t\t\t\ttimestamp, evt.Action, evt.Username, evt.IPAddress, evt.Source)\n\t\t\t\t\t}\n\n\t\t\t\tcase \"performance\":\n\t\t\t\t\tif evt, ok := msg.Payload.(PerformanceMetric); ok {\n\t\t\t\t\t\tfmt.Printf(\"📊 [%s] METRIC: %s = %.2f%s (%s)\\n\",\n\t\t\t\t\t\t\ttimestamp, evt.Metric, evt.Value, evt.Unit, evt.Source)\n\t\t\t\t\t}\n\n\t\t\t\tcase \"errors\":\n\t\t\t\t\tif evt, ok := msg.Payload.(ErrorEvent); ok {\n\t\t\t\t\t\tfmt.Printf(\"❌ [%s] ERROR[%d]: %s - %s (%s)\\n\",\n\t\t\t\t\t\t\ttimestamp, evt.Code, evt.Description, evt.Severity, evt.Source)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tfmt.Printf(\"Channel for topic '%s' closed\\n\", topic)\n\t\t}(topic, ch)\n\t}\n\n\t// Publish different types of events\n\tnow := time.Now()\n\n\t// System events\n\tps.Publish(\"system\", SystemEvent{\n\t\tBaseEvent: BaseEvent{Timestamp: now, Source: \"kernel\"},\n\t\tAction:    \"System startup\",\n\t\tStatus:    \"completed\",\n\t})\n\n\tps.Publish(\"system\", SystemEvent{\n\t\tBaseEvent: BaseEvent{Timestamp: now, Source: \"scheduler\"},\n\t\tAction:    \"Job processing\",\n\t\tStatus:    \"in progress\",\n\t})\n\n\t// Security events\n\tps.Publish(\"security\", SecurityEvent{\n\t\tBaseEvent: BaseEvent{Timestamp: now, Source: \"auth-service\"},\n\t\tIPAddress: \"192.168.1.254\",\n\t\tUsername:  \"admin\",\n\t\tAction:    \"Login attempt\",\n\t})\n\n\tps.Publish(\"security\", SecurityEvent{\n\t\tBaseEvent: BaseEvent{Timestamp: now, Source: \"user-service\"},\n\t\tIPAddress: \"10.0.0.1\",\n\t\tUsername:  \"system\",\n\t\tAction:    \"Permission change\",\n\t})\n\n\t// Performance metrics\n\tps.Publish(\"performance\", PerformanceMetric{\n\t\tBaseEvent: BaseEvent{Timestamp: now, Source: \"monitor-service\"},\n\t\tMetric:    \"CPU Usage\",\n\t\tValue:     65.5,\n\t\tUnit:      \"%\",\n\t})\n\n\tps.Publish(\"performance\", PerformanceMetric{\n\t\tBaseEvent: BaseEvent{Timestamp: now, Source: \"monitor-service\"},\n\t\tMetric:    \"Memory\",\n\t\tValue:     2.45,\n\t\tUnit:      \"GB\",\n\t})\n\n\t// Error events\n\tps.Publish(\"errors\", ErrorEvent{\n\t\tBaseEvent:   BaseEvent{Timestamp: now, Source: \"database\"},\n\t\tCode:        5001,\n\t\tDescription: \"Connection timeout\",\n\t\tSeverity:    \"high\",\n\t})\n\n\tps.Publish(\"errors\", ErrorEvent{\n\t\tBaseEvent:   BaseEvent{Timestamp: now, Source: \"api-gateway\"},\n\t\tCode:        4029,\n\t\tDescription: \"Rate limit exceeded\",\n\t\tSeverity:    \"medium\",\n\t})\n\n\t// Wait briefly to ensure message processing\n\ttime.Sleep(time.Second)\n\n\t// Close the PubSub system\n\tif err := ps.Close(); err != nil {\n\t\tfmt.Printf(\"Error during shutdown: %v\\n\", err)\n\t}\n\n\t// Wait for all goroutines to finish\n\twg.Wait()\n\tfmt.Println(\"All subscribers have terminated successfully\")\n}\n```\n\n## Error Handling\n\nThe library provides specific error types to handle different scenarios:\n\n- `ErrTopicNotFound`: Returned when attempting to publish to a non-existent\n  topic\n- `ErrPubSubClosed`: Returned when attempting operations on a closed PubSub\n  instance\n\n## Best Practices\n\n- Create a new PubSub instance for each logical separation of concerns.\n- Always check for errors when subscribing or publishing.\n- Use `defer` to ensure proper closure of the PubSub system.\n- Implement proper context handling for HTTP-based implementations.\n- Consider using channel buffering for high-throughput scenarios.\n- Use a termination signal (like a `done` channel) to cleanly shut down\n  goroutines.\n- Choose appropriate generic types based on your use case:\n  - Use specific types like `string` or custom structs for type safety.\n  - Use `interface{}` or `any` when flexibility is needed.\n\n## Thread Safety\n\nAll operations on the PubSub instance are thread-safe, utilizing a read-write\nmutex to coordinate access to the subscription map.\n\n## Performance Considerations\n\nThe PubSub system is designed for in-memory operations within a single process.\nFor distributed systems requiring cross-service communication, consider using\nspecialized message brokers like RabbitMQ, Kafka, or NATS.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file\nfor details.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamnilotpal%2Fpubsub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiamnilotpal%2Fpubsub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamnilotpal%2Fpubsub/lists"}