Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/saumya40-codes/pubsub
A kafka-like functionality pubsub written in GO
https://github.com/saumya40-codes/pubsub
golang pubsub
Last synced: 26 days ago
JSON representation
A kafka-like functionality pubsub written in GO
- Host: GitHub
- URL: https://github.com/saumya40-codes/pubsub
- Owner: Saumya40-codes
- Created: 2024-06-04T08:54:16.000Z (8 months ago)
- Default Branch: master
- Last Pushed: 2024-06-05T07:04:27.000Z (8 months ago)
- Last Synced: 2024-11-08T16:49:56.379Z (3 months ago)
- Topics: golang, pubsub
- Language: Go
- Homepage:
- Size: 29.3 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# PUBSUB
pubsub is a communication/streaming service where exchange of messages happens without producer knowing who sender might be
The project tries to implement a kafka-like pubsub based model/arch functionality
Some of the main components involved:
- Broker: A single instance or a node which is responsible for accepting the incoming messages, storing them and directing them to specific client/consumers.
- Topics: Different categories in which messages might be organized, and can have more then one partition within it.
- Producers: Produces a message to a particular topic and to a particular partition.
- Consumers: Consumer subscribe to a particular topic, during this a partitions might be assigned to it to which it will be listening to incase on any message received there.
- The partition in this are *auto-balancing* that is all consumer takes the equal amount of load of listening to the partitions
### Some main functions
**Installation**
On terminal using go
```
go get github.com/Saumya40-codes/pubsub
```
import *core_pubsub "github.com/Saumya40-codes/pubsub/core"* [if it doesn't get automatically while going through below procedure]
*Some functions*:
Create a broker instance
```
broker := core_pubsub.GetorSetBrokerInstance()
```
Create new topic
```
newTopic, err := broker.CreateNewTopic(topicName, No_of_partitions)
```
Create a new consumer
```
consumer := core_pubsub.CreateConsumer(name, groupId)
```
Subscribe to a topic
```
err := consumer.Subscribe(consumer, topicName)
```
Listen for any change in subscribed topic
```
consumer.Run()
```
Create producer
```
producer := core_pubsub.CreateProducer(producerName)
```
Create a message to be sent
```
message := core_pubsub.CreateMessage(topicName, messageContent, partitionIndex)
```
Publish the message
```
err := producer.Publish(topicName, message)
```
You can refer more: [here](https://github.com/Saumya40-codes/pubsub/blob/master/main.go)
## Use case/example
Consider following simple stock prediction architecture
Now only those consumer will receive message who are subscribed to particular partition in a topic to which consumer is sending message to.
On new consumer, auto load-balancing occurs (if existing consumer has more then one partiton)
**Output/Execution**:
Creating topics, consumer and subscribing them to relavant topics
Publishing of messages by producer and consumer receiving it (here order might be deferring as processes are running concurrently)
