https://github.com/paularlott/gossip
Go library to implement gossip based membership and messaging, it supports TCP, UDP and WebSocket protocols with pluggable codecs.
https://github.com/paularlott/gossip
distributed-systems gossip-protocol websocket
Last synced: about 1 month ago
JSON representation
Go library to implement gossip based membership and messaging, it supports TCP, UDP and WebSocket protocols with pluggable codecs.
- Host: GitHub
- URL: https://github.com/paularlott/gossip
- Owner: paularlott
- License: mit
- Created: 2025-03-31T16:54:54.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2026-05-14T17:40:36.000Z (about 1 month ago)
- Last Synced: 2026-05-14T19:38:27.278Z (about 1 month ago)
- Topics: distributed-systems, gossip-protocol, websocket
- Language: Go
- Homepage:
- Size: 557 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.txt
- Code of conduct: CODE_OF_CONDUCT.md
- Cla: CONTRIBUTOR_LICENSE_AGREEMENT.md
Awesome Lists containing this project
README
# Gossip Protocol Library
A lightweight, Go-based library for implementing the gossip protocol in distributed systems. The library supports multiple transport mechanisms, including TCP, UDP, and HTTP, providing flexibility for a variety of use cases.
With its straightforward API, the library simplifies decentralized communication by enabling the creation and management of nodes, exchanging messages, and handling critical events like node joins and departures. Designed with reliability in mind, it continuously monitors node health within the cluster and seamlessly removes unreachable nodes, ensuring the system stays robust and adaptive.
The library leverages a hybrid approach to gossiping, combining **event-driven gossiping** and **periodic gossiping** to balance rapid updates and eventual consistency across the cluster:
- **Event-Driven Gossiping** swiftly propagates critical updates immediately after events occur, minimizing latency.
- **Periodic Gossiping** adds redundancy by disseminating updates at regular intervals, ensuring eventual consistency even if some nodes initially miss updates.
This flexible architecture supports the development of resilient distributed systems with efficient data sharing and robust fault tolerance.
## Features
- **Multiple Transport Support**: TCP, UDP, and HTTP connections
- **Message Security**: Optional encryption for message payloads
- **Compression**: Configurable message compression, support for Snappy is provide by default
- **Health Monitoring**: Automatic node health checking with direct and indirect pings
- **Flexible Codec Support**: Pluggable serialization with support for multiple msgpack implementations
- **Metadata Sharing**: Distribute custom node metadata across the cluster
- **Version Checking**: Application and protocol version compatibility verification
- **Automatic Transport Selection**: UDP will be used wherever possible, however if the packet exceeds the MTU size, TCP will be used instead
## Installation
To install the library, use the following command:
```shell
go get github.com/paularlott/gossip
```
## Basic Usage
```go
package main
import (
"fmt"
"time"
"github.com/paularlott/gossip"
"github.com/paularlott/gossip/codec"
"github.com/paularlott/gossip/compression"
)
func main() {
// Create configuration
config := gossip.DefaultConfig()
config.NodeID = "01960f9b-72ca-7a51-9efa-47c12f42a138" // Optional: auto-generated if not specified
config.BindAddr = "127.0.0.1:8000" // Listen on TCP and UDP
config.EncryptionKey = "your-32-byte-key" // Optional: enables encryption
config.Cipher = encryption.NewAESEncryptor() // Encryption algorithm
config.MsgCodec = codec.NewShamatonMsgpackCodec() // Message serialization
config.Compressor = compression.NewSnappyCompressor() // Optional: enables compression
// Create and start the cluster
cluster, err := gossip.NewCluster(config)
if err != nil {
panic(err)
}
cluster.Start()
defer cluster.Stop()
// Join existing cluster (if any)
err = cluster.Join([]string{"127.0.0.1:8001"})
if err != nil {
fmt.Println("Warning:", err)
}
// Register message handler
const CustomMsg gossip.MessageType = gossip.UserMsg + 1
cluster.HandleFunc(CustomMsg, func(sender *gossip.Node, packet *gossip.Packet) error {
var message string
if err := packet.UnmarshalPayload(&message); err != nil {
return err
}
fmt.Printf("Received message from %s: %s\n", sender.ID, message)
return nil
})
// Broadcast a message
message := "Hello cluster!"
cluster.Broadcast(CustomMsg, message)
// Keep the application running
select {}
}
```
## Address Formats
The gossip library supports multiple address formats for binding and connecting to the cluster:
- **IP:port** - Standard TCP/UDP address (e.g., 127.0.0.1:8000)
- **hostname:port** - DNS hostname with port, when multiple addresses are returned the node will attempt to connect to each address in turn assuming each is a node within the cluster
- **hostname** or **IP** - The default port will be used, for a hostname returning multiple addresses the node will attempt to connect to each address in turn assuming each is a node within the cluster
- **srv+service-name** - SRV DNS record lookup, when multiple addresses are returned the node will attempt to connect to each address in turn assuming each is a node within the cluster
- **https://hostname:port/endpoint** - HTTP connection
- **srv+https://hostname:port/endpoint** - Secure HTTP connection looking up the port number for the target
## Configuration Options
The `Config` struct provides extensive customization:
```go
config := gossip.DefaultConfig()
// Node identification
config.NodeID = "unique-node-id" // Optional: auto-generated if not provided
config.BindAddr = "0.0.0.0:3500" // Address to bind for listening
config.AdvertiseAddr = "192.168.1.1:3500" // Address to advertise to peers (optional)
// Communication
config.EncryptionKey = "your-32-byte-key" // Optional: enables encryption
config.Cipher = encryption.NewAESEncryptor() // Encryption algorithm
config.Compressor = compression.NewSnappyCompressor() // Enable payload compression using the provided compressor
config.CompressMinSize = 1024 // Minimum size of a packet that will be considered for compression
config.Transport = gossip.NewSocketTransport(config) // Socket based transport
// Custom dialer/listener (e.g. for overlay networks like Tailscale tsnet)
config.DialFunc = srv.Dial // Custom TCP dial function, defaults to net.DialTimeout
config.ListenFunc = srv.Listen // Custom TCP listen function, defaults to net.ListenTCP
// Setting ListenFunc automatically enables ForceReliableTransport
// Logging
config.Logger = logger.NewNullLogger() // Default: no logging
```
## Logging
The library uses the [github.com/paularlott/logger](https://github.com/paularlott/logger) interface for logging. By default, a null logger is used which discards all log output. You can provide your own logger implementation by implementing the `logger.Logger` interface:
```go
type Logger interface {
Trace(msg string, args ...any)
Debug(msg string, args ...any)
Info(msg string, args ...any)
Warn(msg string, args ...any)
Error(msg string, args ...any)
Fatal(msg string, args ...any)
With(key string, value any) Logger
WithError(err error) Logger
WithGroup(name string) Logger
}
```
The `Fatal` method logs the message and then calls `os.Exit(1)`, making it convenient for handling fatal errors without explicit exit calls.
### Using slog
The library provides examples using the slog implementation:
```go
import (
"os"
"github.com/paularlott/logger/slog"
)
// Create a console logger with colored output
logger := slog.New("debug", "console", os.Stderr)
// Configure the cluster with the logger
config := gossip.DefaultConfig()
config.Logger = logger
```
See the [examples/common/log.go](examples/common/log.go) file for a complete implementation using slog.
## Node States
Nodes in the cluster go through several states:
- **NodeUnknown** - Node state is unknown, nodes start in this state and change to NodeAlive when joining the cluster
- **NodeAlive** - Node is active and healthy
- **NodeSuspect** - Node might be unhealthy (pending confirmation)
- **NodeDead** - Node is confirmed dead
- **NodeLeaving** - Node is gracefully leaving the cluster
## Message Codecs
Multiple serialization options are available allowing you to choose the one that best fits your application:
```go
// Using Shamaton msgpack
config.MsgCodec = codec.NewShamatonMsgpackCodec()
// Using Vmihailenco msgpack
config.MsgCodec = codec.NewVmihailencoMsgpackCodec()
// Using JSON
config.MsgCodec = codec.NewJSONCodec()
```
### Examples
The `examples` directory contains various examples demonstrating the library's capabilities. Each example is self-contained and can be run independently.
- **[basic](examples/basic)**: A basic usage example that creates a cluster and joins nodes to it. Nodes can communicate over TCP/UDP or HTTP.
- **[events](examples/events)**: Example that installs an event handler to display cluster events.
- **[usermessages](examples/usermessages)**: Example that demonstrates user defined message handling.
- **[kv](examples/kv)**: Example Key Value store.
- **[leader](examples/leader)**: Example demonstrating leader election.