{"id":15061753,"url":"https://github.com/impact-eintr/raftd","last_synced_at":"2025-09-06T10:40:55.577Z","repository":{"id":43210797,"uuid":"450782405","full_name":"impact-eintr/raftd","owner":"impact-eintr","description":" raftd 基于raft和bolt的分布式KV数据库 由于简单实现了租约系统 可以用于简单的服务发现 基于gin框架提供http服务","archived":false,"fork":false,"pushed_at":"2022-03-15T03:41:07.000Z","size":106,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-25T07:25:47.543Z","etag":null,"topics":["btree","distributed-systems","gin","golang","kv-store","lsm-tree","microservice","raft"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/impact-eintr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-01-22T10:17:35.000Z","updated_at":"2025-01-20T10:02:21.000Z","dependencies_parsed_at":"2022-08-26T07:11:31.815Z","dependency_job_id":null,"html_url":"https://github.com/impact-eintr/raftd","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/impact-eintr/raftd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/impact-eintr%2Fraftd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/impact-eintr%2Fraftd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/impact-eintr%2Fraftd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/impact-eintr%2Fraftd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/impact-eintr","download_url":"https://codeload.github.com/impact-eintr/raftd/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/impact-eintr%2Fraftd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273892837,"owners_count":25186562,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["btree","distributed-systems","gin","golang","kv-store","lsm-tree","microservice","raft"],"created_at":"2024-09-24T23:24:40.238Z","updated_at":"2025-09-06T10:40:55.555Z","avatar_url":"https://github.com/impact-eintr.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# raftd\nraftd 基于`raft`算法，支持数据持久化(bolt)的分布式KV数据库，可以用于简单的服务发现，基于gin框架提供http服务\n\n## Usage\n\n``` sh\ncd cmd/raftd\n\ngo build\n\n./raftd -id node01 -haddr 127.0.0.1:8001 -raddr 127.0.0.1:8101 ~/.raftd01\n\n./raftd -id node02 -haddr 127.0.0.1:8002 -raddr 127.0.0.1:8102 -join 127.0.0.1:8001 ~/.raftd02\n\n./raftd -id node03 -haddr 127.0.0.1:8003 -raddr 127.0.0.1:8103 -join 127.0.0.1:8001 ~/.raftd03\n```\n\n``` sh\ncurl -X PUT 127.0.0.1:8001/key/test -T ./test.png\n\ncurl 127.0.0.1:8001/key/test --output 1.png\n```\n\n\n\u003e 步骤：\n\n1. 启动的时候只make() 然后 调用 tx.ForEach() 遍历整个 db 建立缓存\n2. 更新/删除 KV 的时候同步更新缓存\n3. 集群同步的时候 直接使用 Store.m 进行数据传输\n\n\u003e 节点同步的步骤：\n\n1. 先 fsm.db.Close() 然后 截断 data.db\n2. 重新建立 db bolt.Open(...)\n3. range snapshot.m 然后执行一个 Batch Update 将数据同步到 fsm.db 和 fsm.m 中\n4. TODO 失败的情况\n\n## 租约系统(可以用于服务发现)\n\n### Usage\n\n获取租约并续租\n``` sh\n# 服务1\nID=$(curl -sL -XPOST 127.0.0.1:8001/lease/grant\\?ttl=10\\\u0026name=/esq/node-1);while true;do curl -sL -XPOST 127.0.0.1:8001/lease/keepalive/$ID -d 'key=nodeinfo' -d 'data={\"http_addr\":\"127.0.0.1:9001\",\"tcp_addr\":\"127.0.0.1:9002\",\"node_id\":1,\"weight\":1}';sleep 3;done\n\n\n# 服务2\nID=$(curl -sL -XPOST 127.0.0.1:8001/lease/grant\\?ttl=10\\\u0026name=/esq/node-2);while true;do curl -sL -XPOST 127.0.0.1:8001/lease/keepalive/$ID -d 'key=nodeinfo1' -d 'data={\"http_addr\":\"127.0.0.1:9003\",\"tcp_addr\":\"127.0.0.1:9004\",\"node_id\":2,\"weight\":2}';sleep 3;done\n```\n\n获取键值对\n\n``` sh\ncurl -sL -XPOST 127.0.0.1:8001/lease/kv/nodeinfo -d 'prefix=/esq/node'\n```\n\n### 实现原理\n\n1. 创建一个DefaultLeaseBucketName的Bucket\n2. 用雪花算法生成一个LeaseId，这个Id作为一个子桶，里面存放共享这个租约的键值对，注意有一个Key是不允许覆写的，KeyName为 `meta`，存放当前租约的元数据，包括 LeaseTTL LeaseAliveCount LeaseStatus LeaseName\n3. 启动一个 goroutine 每秒检测当前租约的状态，一旦有客户端租用了当前租约，租约的状态就从 CREATE 转为 ALIVE，并且给租约中的所有键值对的TTL减去一，在操作完所有的键值对后，统计TTL不为0的AliveCount，如果AliveCount==0，撤销当前租约(退出当前goroutine,删除这个LeaseId的子桶)\n4. 续租：客户端在申请完租约后，会从服务端获取租约号，使用租约号可以续租，一旦续租，对应租约中的对应键值对的TTL会重新置为这个租约的TTL(从 `meta` 中获取)\n\n\n### TODO\n- 申请了却从不租用的租约该怎么处理？这些租约的状态为CREATE，如果每次同步状态机时全部重启loopCheck，会消耗一定的资源，如果同步的时候直接删除这些租约属实 鸭子睁眼——大可不必 了，目前有一个可能的解决方案是 PutInPool\n\n- TimeToLive API\n\n- 存放租约的树换成LSM-Tree，KV系统属于读多写少，租约系统属于读写都比较多，每秒会有多个协程更新整棵树，可以考虑为租约系统单独做一个存储引擎，准备参考\u003chttps://github.com/dgraph-io/badger/tree/v1.0.0\u003e,这个库比较成熟，值得借鉴\n\n- 关于 LSM Tree 存放租约 key 应该设计为 LeaseID value 设计为 metaSize(4bytes)valueSize(4bytes)meta(struct_json_bytes)value(map_json_bytes) TODO:考虑使用某个更高性能的json库\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimpact-eintr%2Fraftd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fimpact-eintr%2Fraftd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimpact-eintr%2Fraftd/lists"}