https://github.com/ejunjsh/raftkv
🥒 distributed key/value storage with raft
https://github.com/ejunjsh/raftkv
go key-value-store raft
Last synced: 3 months ago
JSON representation
🥒 distributed key/value storage with raft
- Host: GitHub
- URL: https://github.com/ejunjsh/raftkv
- Owner: ejunjsh
- Created: 2019-04-05T05:42:19.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2019-07-23T06:12:22.000Z (almost 6 years ago)
- Last Synced: 2024-12-29T08:41:54.878Z (4 months ago)
- Topics: go, key-value-store, raft
- Language: Go
- Homepage:
- Size: 2.48 MB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# raftkv
[](https://travis-ci.org/ejunjsh/raftkv)
distributed key value storage with raft
most of the code is from [etcd](https://github.com/etcd-io/etcd) , just for reading and learning its source code
## Getting Started
### install from remote
go get github.com/ejunjsh/raftkv/cmd/raftkvd
### build and test locally
cd $GOPATH/src/github.com/ejunjsh/raftkv/
sh ci.sh
### Running single node
First start a single-member cluster of raftkvd:
```sh
raftkvd --id 1 --cluster http://127.0.0.1:12379 --port 12380
```
Each raftkvd process maintains a single raft instance and a key-value server.
The process's list of comma separated peers (--cluster), its raft ID index into the peer list (--id), and http key-value server port (--port) are passed through the command line.
Next, store a value ("hello") to a key ("my-key"):
```
curl -L http://127.0.0.1:12380/my-key -XPUT -d hello
```
Finally, retrieve the stored key:
```
curl -L http://127.0.0.1:12380/my-key
```
### Running a local cluster
raftkvd --id 1 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 12380
raftkvd --id 2 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 22380
raftkvd --id 3 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 32380
This will bring up three raftkvd instances.
Now it's possible to write a key-value pair to any member of the cluster and likewise retrieve it from any member.
### Fault Tolerance
To test cluster recovery, first start a cluster and write a value "foo":
```sh
# start all nodes before
raftkvd --id 1 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 12380
raftkvd --id 2 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 22380
raftkvd --id 3 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 32380
curl -L http://127.0.0.1:12380/my-key -XPUT -d foo
```
Next, remove a node and replace the value with "bar" to check cluster availability:
```sh
# kill the node 2 before
kill ...
curl -L http://127.0.0.1:12380/my-key -XPUT -d bar
curl -L http://127.0.0.1:32380/my-key
```
Finally, bring the node back up and verify it recovers with the updated value "bar":
```sh
# restart node 2 before
raftkvd --id 2 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 22380
curl -L http://127.0.0.1:22380/my-key
```
### Dynamic cluster reconfiguration
Nodes can be added to or removed from a running cluster using requests to the REST API.
For example, suppose we have a 3-node cluster that was started with the commands:
```sh
raftkvd --id 1 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 12380
raftkvd --id 2 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 22380
raftkvd --id 3 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 32380
```
A fourth node with ID 4 can be added by issuing a POST:
```sh
curl -L http://127.0.0.1:12380/4 -XPOST -d http://127.0.0.1:42379
```
Then the new node can be started as the others were, using the --join option:
```sh
raftkvd --id 4 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379,http://127.0.0.1:42379 --port 42380 --join
```
The new node should join the cluster and be able to service key/value requests.
We can remove a node using a DELETE request:
```sh
curl -L http://127.0.0.1:12380/3 -XDELETE
```
Node 3 should shut itself down once the cluster has processed this request.
## reference
https://github.com/etcd-io/etcd