https://github.com/fortio/memstore
Distributed HA in memory store for Golang
https://github.com/fortio/memstore
Last synced: 4 months ago
JSON representation
Distributed HA in memory store for Golang
- Host: GitHub
- URL: https://github.com/fortio/memstore
- Owner: fortio
- License: apache-2.0
- Created: 2022-09-19T01:43:08.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2025-05-26T13:03:48.000Z (4 months ago)
- Last Synced: 2025-05-26T14:23:23.876Z (4 months ago)
- Language: Go
- Size: 120 KB
- Stars: 2
- Watchers: 3
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# memstore
Distributed HA in memory store for Golang
## Config Input
(all [dflags](https://github.com/fortio/dflag#fortio-dynamic-flags) so can be changed without restart)
- List of DNS names, IPs (in Kubernetes you'd pass just a headless service name)
- Refresh frequency for DNS to IP
### Prototype
```
go run . -config-port 7999
```
Then go change the `peers` on http://localhost:7999/?type=dynamic to see:
```
20:26:10 I mstore.go:17> Peer set changed from to a,b,c,z
20:26:10 I mstore.go:23> Connecting to added Peer : "a"
20:26:10 I mstore.go:23> Connecting to added Peer : "b"
20:26:10 I mstore.go:23> Connecting to added Peer : "c"
20:26:10 I mstore.go:23> Connecting to added Peer : "z"
```
and
```
20:26:31 I mstore.go:17> Peer set changed from a,b,c,z to d,a,b,z
20:26:31 I mstore.go:20> Disconnecting from removed peer : "c"
20:26:31 I mstore.go:23> Connecting to added Peer : "d"
```
or similar
```
make test
```
## Communication
Should we
- use some broadcasting/bus
- ring
- tcp or http or grpc
Let's use a fully mesh broadcast using point2point h2.
## Components
### Discovery Service
Need to get list of (possible) peers
Pluggable module
#### Kubernetes
Endpoint(s) - or rather to make it simpler and allow by name addressing - StatefulSet (would also work better with istio than trying direct pod to pod by IP)
#### DNS
###
## Embedded or separate
Why not both
## Protocol
- Zookeeper
- Raft
- Something wrong but simpler (*)
## Persistence
- Dump to disk (Persistent Volume in k8s) periodically
## CircularBuffer
Both pub/sub thread safe blocking version and pure FIFO queue with set capacity versions:
See [cb/cb.go](cb/cb.go)