{"id":13629738,"url":"https://github.com/eraft-io/eraft","last_synced_at":"2026-01-12T06:49:20.438Z","repository":{"id":40466433,"uuid":"369146778","full_name":"eraft-io/eraft","owner":"eraft-io","description":"A generic raft library","archived":false,"fork":false,"pushed_at":"2024-09-05T15:46:27.000Z","size":4790,"stargazers_count":259,"open_issues_count":0,"forks_count":43,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-09-05T21:27:19.158Z","etag":null,"topics":["cloudnative","database","distributed-consensus-algorithms","distributed-systems","raft","redis","rocksdb"],"latest_commit_sha":null,"homepage":"https://eraft.cn","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eraft-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-20T09:05:38.000Z","updated_at":"2024-09-05T15:46:41.000Z","dependencies_parsed_at":"2023-02-08T06:15:52.983Z","dependency_job_id":"e69b9b72-e19a-4d77-b83d-7e7a2f691595","html_url":"https://github.com/eraft-io/eraft","commit_stats":{"total_commits":207,"total_committers":10,"mean_commits":20.7,"dds":"0.37681159420289856","last_synced_commit":"d8aa526b29638744e29ba5c573127f6d4885ca2a"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eraft-io%2Feraft","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eraft-io%2Feraft/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eraft-io%2Feraft/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eraft-io%2Feraft/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eraft-io","download_url":"https://codeload.github.com/eraft-io/eraft/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223751421,"owners_count":17196635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudnative","database","distributed-consensus-algorithms","distributed-systems","raft","redis","rocksdb"],"created_at":"2024-08-01T22:01:18.033Z","updated_at":"2026-01-12T06:49:20.433Z","avatar_url":"https://github.com/eraft-io.png","language":"Go","funding_links":[],"categories":["C++"],"sub_categories":[],"readme":"[![Language](https://img.shields.io/badge/Language-Go-blue.svg)](https://golang.org/)\n[![License](https://img.shields.io/badge/license-MIT-green)](https://opensource.org/licenses/MIT)\n\n[中文](README_cn.md) | English\n\n### Overview\n\n### Why we need build a distributed system?\n\nLet's first look at the drawbacks of traditional single-node client-server (C/S) or browser-server (B/S) systems:\nA single-node architecture means relying on just one machine. The performance of any single machine is inherently limited, and higher-performance machines are significantly more expensive—consider IBM mainframes, for example, which come at a very high cost. Moreover, if this machine fails or the process crashes due to bugs in the code, the system has no fault tolerance and becomes completely unavailable.\n\nAfter we analyze the shortcomings of single-node systems, we can summarize the design goals of distributed systems\n\n#### 1. Scalability\n\nThe distributed system we design must be scalable. The scalability here is that we can obtain higher total system\nthroughput and better performance by using more machines. Of course, it is not that the more machines, the better the\nperformance. For some complex computing scenarios, more nodes are not necessarily better performance.\n\n#### 2. Availability\n\nThe distributed system will not stop services directly if a machine in the system fails. After a machine fails, the\nsystem can quickly switch traffic to a normal machine and continue to provide services.\n\n#### 3. Consistency\n\nTo achieve this, the most important algorithm is the replication algorithm. We need a replication algorithm to ensure\nthat the data of the dead machine and the machine that is cut to replace it are consistent, usually in the field of\ndistributed systems. Consistency algorithm to ensure the smooth progress of replication.\n\n### Consistency Algorithm\n\nIt is recommended to read [raft small paper](https://raft.github.io/raft.pdf)\n\nTake a look with the following questions:\n\n- what is split brain?\n\n- What is our solution to the split brain?\n\n- why does majority help avoid split brain?\n\n- why the logs?\n\n- why a leader?\n\n- how to ensure at most one leader in a term?\n\n- how does a server learn about a newly elected leader?\n\n- what happens if an election doesn't succeed?\n\n- how does Raft avoid split votes?\n\n- how to choose the election timeout?\n\n- what if old leader isn't aware a new leader is elected?\n\n- how can logs disagree?\n\n- what would we like to happen after a server crashes?\n\n### data sharding\n\nWell, through the basic Raft algorithm, we can achieve a highly available raft server group. We have solved the previous\nissues of availability and consistency, but the problem still exists. There is only one leader in a raft server group to\nreceive read and write traffic. Of course, you can use followers to share part of the read traffic to improve\nperformance (there will be some issues related to transactions, which we will discuss later). But there is a limit to\nwhat the system can provide.\n\nAt this time, we need to think about slicing the requests written by the client, just like map reduce, in the first\nstage of map, first cut the huge data set into small ones for processing.\n\nThe hash sharding method is used in eraft. We map data into buckets through a hash algorithm, and then different raft\ngroups are responsible for some of the buckets. How many buckets a raft group can be responsible for can be adjusted.\n\n### System Architecture\n\n#### Concept introduction\n\n##### bucket\n\nIt is the logical unit of data management in the cluster, and a grouped service can be responsible for the data of\nmultiple buckets\n\n##### config table\n\nCluster configuration table, which mainly maintains the mapping relationship between cluster service groups and buckets.\nBefore clients access cluster data, they need to go to this table to query the list of service groups where the bucket\nis located.\n\n#### service module\n\n##### metaserver\n\nIt is mainly responsible for the version management of the cluster configuration table. It internally maintains a\nversion chain of the cluster configuration table, which can record changes to the cluster configuration.\n\n##### shardkvserver\n\nIt is mainly responsible for cluster data storage. Generally, three machines form a raft group to provide\nhigh-availability services to the outside world.\n\n### Build\n\npre-dependencies\n\ngo version \u003e= go 1.21\n\ndownload code and make it\n\n```\ngit clone https://github.com/eraft-io/eraft.git\n\ncd eraft\nmake\n```\n\n### run cluster\n\n```\ngo test -run TestBasicClusterRW tests/integration_test.go -v\n```\n\nrun basic cluster bench\n\n```\ngo test -run TestClusterRwBench tests/integration_test.go -v\n```\n\n### unit test\ngenerate unit test report\n\n```\nmake gen-test-coverage \n```\n\nview unit test report\n\n```\nmake view-test-coverage\n```\n\n### Book\n\nBook title: 【Distributed Data Services: Transaction Models, Processing Language, Consistency and Architecture.】\n\nISBN：978-7-111-73737-7\n\nThis book provides a detailed introduction to the implementation principles and code analysis of the eRaft prototype\nsystem in the distributed data services protocol library.\n\nThe eraft t project is to industrialize the mit6.824 lab operation into a distributed storage system. We will introduce\nthe principles of distributed systems in the simplest and straightforward way, and guide you to design and implement an\nindustrialized distributed storage system.\n\n### Newest document\n\nIf you want to check the latest documents, please visit [eraft official website](https://eraft.cn)\n\n### Video tutorials\n\n[bilibili](https://space.bilibili.com/389476201/channel/collectiondetail?sid=481263\u0026spm_id_from=333.788.0.0)\n\n### Ebook for this project\n\n[Shopping link](https://3.cn/1W-jAWMR)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feraft-io%2Feraft","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feraft-io%2Feraft","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feraft-io%2Feraft/lists"}