{"id":13694252,"url":"https://github.com/tidwall/finn","last_synced_at":"2025-05-03T01:31:59.490Z","repository":{"id":57487444,"uuid":"68643538","full_name":"tidwall/finn","owner":"tidwall","description":"Fast Raft framework using the Redis protocol for Go","archived":true,"fork":false,"pushed_at":"2020-11-03T18:32:52.000Z","size":168,"stargazers_count":542,"open_issues_count":0,"forks_count":24,"subscribers_count":17,"default_branch":"master","last_synced_at":"2024-08-03T17:19:10.249Z","etag":null,"topics":["distributed-computing","golang","raft","redis"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tidwall.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-09-19T20:26:38.000Z","updated_at":"2024-07-09T09:49:40.000Z","dependencies_parsed_at":"2022-09-01T23:02:52.497Z","dependency_job_id":null,"html_url":"https://github.com/tidwall/finn","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tidwall%2Ffinn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tidwall%2Ffinn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tidwall%2Ffinn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tidwall%2Ffinn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tidwall","download_url":"https://codeload.github.com/tidwall/finn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224346420,"owners_count":17296213,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-computing","golang","raft","redis"],"created_at":"2024-08-02T17:01:27.974Z","updated_at":"2024-11-12T20:31:48.848Z","avatar_url":"https://github.com/tidwall.png","language":"Go","funding_links":[],"categories":["开源类库","Open source library","Go"],"sub_categories":["算法","Algorithm"],"readme":"**This project has been archived. Please check out [Uhaha](https://github.com/tidwall/uhaha) for a fitter, happier, more productive Raft framework.**\n\n\u003cp align=\"center\"\u003e\n\u003cimg \n    src=\"logo.jpg\" \n    width=\"314\" height=\"200\" border=\"0\" alt=\"FINN\"\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://goreportcard.com/report/github.com/tidwall/finn\"\u003e\u003cimg src=\"https://goreportcard.com/badge/github.com/tidwall/finn?style=flat-square\" alt=\"Go Report Card\"\u003e\u003c/a\u003e\n\u003ca href=\"https://godoc.org/github.com/tidwall/finn\"\u003e\u003cimg src=\"https://img.shields.io/badge/api-reference-blue.svg?style=flat-square\" alt=\"GoDoc\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\nFinn is a fast and simple framework for building [Raft](https://raft.github.io/) implementations in Go. It uses [Redcon](https://github.com/tidwall/redcon) for the network transport and [Hashicorp Raft](https://github.com/hashicorp/raft). There is also the option to use [LevelDB](https://github.com/syndtr/goleveldb), [BoltDB](https://github.com/boltdb/bolt) or [FastLog](https://github.com/tidwall/raft-fastlog) for log persistence.\n\n\nFeatures\n--------\n\n- Simple API for quickly creating a [fault-tolerant](https://en.wikipedia.org/wiki/Fault_tolerance) cluster\n- Fast network protocol using the [raft-redcon](https://github.com/tidwall/raft-redcon) transport\n- Optional [backends](#log-backends) for log persistence. [LevelDB](https://github.com/syndtr/goleveldb), [BoltDB](https://github.com/boltdb/bolt), or [FastLog](https://github.com/tidwall/raft-fastlog)\n- Adjustable [consistency and durability](#consistency-and-durability) levels\n- A [full-featured example](#full-featured-example) to help jumpstart integration\n- [Built-in raft commands](#built-in-raft-commands) for monitoring and managing the cluster\n- Supports the [Redis log format](http://build47.com/redis-log-format-levels/)\n- Works with clients such as [redigo](https://github.com/garyburd/redigo), [redis-py](https://github.com/andymccurdy/redis-py), [node_redis](https://github.com/NodeRedis/node_redis), [jedis](https://github.com/xetorthio/jedis), and [redis-cli](http://redis.io/topics/rediscli)\n\n\nGetting Started\n---------------\n\n### Installing\n\nTo start using Finn, install Go and run `go get`:\n\n```sh\n$ go get -u github.com/tidwall/finn\n```\n\nThis will retrieve the library.\n\n### Example\n\nHere's an example of a Redis clone that accepts the GET, SET, DEL, and KEYS commands.\n\nYou can run a [full-featured version](#full-featured-example) of this example from a terminal:\n\n```\ngo run example/clone.go\n```\n\n```go\npackage main\n\nimport (\n\t\"encoding/json\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n\n\t\"github.com/tidwall/finn\"\n\t\"github.com/tidwall/match\"\n\t\"github.com/tidwall/redcon\"\n)\n\nfunc main() {\n\tn, err := finn.Open(\"data\", \":7481\", \"\", NewClone(), nil)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer n.Close()\n\tselect {}\n}\n\ntype Clone struct {\n\tmu   sync.RWMutex\n\tkeys map[string][]byte\n}\n\nfunc NewClone() *Clone {\n\treturn \u0026Clone{keys: make(map[string][]byte)}\n}\n\nfunc (kvm *Clone) Command(m finn.Applier, conn redcon.Conn, cmd redcon.Command) (interface{}, error) {\n\tswitch strings.ToLower(string(cmd.Args[0])) {\n\tdefault:\n\t\treturn nil, finn.ErrUnknownCommand\n\tcase \"set\":\n\t\tif len(cmd.Args) != 3 {\n\t\t\treturn nil, finn.ErrWrongNumberOfArguments\n\t\t}\n\t\treturn m.Apply(conn, cmd,\n\t\t\tfunc() (interface{}, error) {\n\t\t\t\tkvm.mu.Lock()\n\t\t\t\tkvm.keys[string(cmd.Args[1])] = cmd.Args[2]\n\t\t\t\tkvm.mu.Unlock()\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t\tfunc(v interface{}) (interface{}, error) {\n\t\t\t\tconn.WriteString(\"OK\")\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t)\n\tcase \"get\":\n\t\tif len(cmd.Args) != 2 {\n\t\t\treturn nil, finn.ErrWrongNumberOfArguments\n\t\t}\n\t\treturn m.Apply(conn, cmd, nil,\n\t\t\tfunc(interface{}) (interface{}, error) {\n\t\t\t\tkvm.mu.RLock()\n\t\t\t\tval, ok := kvm.keys[string(cmd.Args[1])]\n\t\t\t\tkvm.mu.RUnlock()\n\t\t\t\tif !ok {\n\t\t\t\t\tconn.WriteNull()\n\t\t\t\t} else {\n\t\t\t\t\tconn.WriteBulk(val)\n\t\t\t\t}\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t)\n\tcase \"del\":\n\t\tif len(cmd.Args) \u003c 2 {\n\t\t\treturn nil, finn.ErrWrongNumberOfArguments\n\t\t}\n\t\treturn m.Apply(conn, cmd,\n\t\t\tfunc() (interface{}, error) {\n\t\t\t\tvar n int\n\t\t\t\tkvm.mu.Lock()\n\t\t\t\tfor i := 1; i \u003c len(cmd.Args); i++ {\n\t\t\t\t\tkey := string(cmd.Args[i])\n\t\t\t\t\tif _, ok := kvm.keys[key]; ok {\n\t\t\t\t\t\tdelete(kvm.keys, key)\n\t\t\t\t\t\tn++\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tkvm.mu.Unlock()\n\t\t\t\treturn n, nil\n\t\t\t},\n\t\t\tfunc(v interface{}) (interface{}, error) {\n\t\t\t\tn := v.(int)\n\t\t\t\tconn.WriteInt(n)\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t)\n\tcase \"keys\":\n\t\tif len(cmd.Args) != 2 {\n\t\t\treturn nil, finn.ErrWrongNumberOfArguments\n\t\t}\n\t\tpattern := string(cmd.Args[1])\n\t\treturn m.Apply(conn, cmd, nil,\n\t\t\tfunc(v interface{}) (interface{}, error) {\n\t\t\t\tvar keys []string\n\t\t\t\tkvm.mu.RLock()\n\t\t\t\tfor key := range kvm.keys {\n\t\t\t\t\tif match.Match(key, pattern) {\n\t\t\t\t\t\tkeys = append(keys, key)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tkvm.mu.RUnlock()\n\t\t\t\tsort.Strings(keys)\n\t\t\t\tconn.WriteArray(len(keys))\n\t\t\t\tfor _, key := range keys {\n\t\t\t\t\tconn.WriteBulkString(key)\n\t\t\t\t}\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t)\n\t}\n}\n\nfunc (kvm *Clone) Restore(rd io.Reader) error {\n\tkvm.mu.Lock()\n\tdefer kvm.mu.Unlock()\n\tdata, err := ioutil.ReadAll(rd)\n\tif err != nil {\n\t\treturn err\n\t}\n\tvar keys map[string][]byte\n\tif err := json.Unmarshal(data, \u0026keys); err != nil {\n\t\treturn err\n\t}\n\tkvm.keys = keys\n\treturn nil\n}\n\nfunc (kvm *Clone) Snapshot(wr io.Writer) error {\n\tkvm.mu.RLock()\n\tdefer kvm.mu.RUnlock()\n\tdata, err := json.Marshal(kvm.keys)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif _, err := wr.Write(data); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n```\n\nThe Applier Type\n----------------\nEvery `Command()` call provides an `Applier` type which is responsible for handling all Read or Write operation. In the above example you will see one `m.Apply(conn, cmd, ...)` for each command.\n\nThe signature for the `Apply()` function is:\n```go\nfunc Apply(\n\tconn redcon.Conn, \n\tcmd redcon.Command,\n\tmutate func() (interface{}, error),\n\trespond func(interface{}) (interface{}, error),\n) (interface{}, error)\n```\n\n- `conn` is the client connection making the call. It's possible that this value may be `nil` for commands that are being replicated on Follower nodes. \n- `cmd` is the command to process.\n- `mutate` is the function that handles modifying the node's data. \nPassing `nil` indicates that the operation is read-only.\nThe `interface{}` return value will be passed to the `respond` func.\nReturning an error will cancel the operation and the error will be returned to the client.\n- `respond` is used for responding to the client connection. It's also used for read-only operations. The `interface{}` param is what was passed from the `mutate` function and may be `nil`. \nReturning an error will cancel the operation and the error will be returned to the client.\n\n*Please note that the `Apply()` command is required for modifying or accessing data that is shared on all of the nodes.\nOptionally you can forgo the call altogether for operations that are unique to the node.*\n\nSnapshots\n---------\nAll Raft commands are stored in one big log file that will continue to grow. The log is stored on disk, in memory, or both. At some point the server will run out of memory or disk space.\nSnapshots allows for truncating the log so that it does not take up all of the server's resources.\n\nThe two functions `Snapshot` and `Restore` are used to create a snapshot and restore a snapshot, respectively.\n\nThe `Snapshot()` function passes a writer that you can write your snapshot to.\nReturn `nil` to indicate that you are done writing. Returning an error will cancel the snapshot. If you want to disable snapshots altogether:\n\n```go\nfunc (kvm *Clone) Snapshot(wr io.Writer) error {\n\treturn finn.ErrDisabled\n}\n```\n\nThe `Restore()` function passes a reader that you can use to restore your snapshot from.\n\n*Please note that the Raft cluster is active during a snapshot operation. \nIn the example above we use a read-lock that will force the cluster to delay all writes until the snapshot is complete.\nThis may not be ideal for your scenario.*\n\nFull-featured Example\n---------------------\n\nThere's a command line Redis clone that supports all of Finn's features. Print the help options:\n\n```\ngo run example/clone.go -h\n```\n\nFirst start a single-member cluster:\n```\ngo run example/clone.go\n```\n\nThis will start the clone listening on port 7481 for client and server-to-server communication.\n\nNext, let's set a single key, and then retrieve it:\n\n```\n$ redis-cli -p 7481 SET mykey \"my value\"\nOK\n$ redis-cli -p 7481 GET mykey\n\"my value\"\n```\n\nAdding members:\n```\ngo run example/clone.go -p 7482 -dir data2 -join :7481\ngo run example/clone.go -p 7483 -dir data3 -join :7481\n```\n\nThat's it. Now if node1 goes down, node2 and node3 will continue to operate.\n\n\nBuilt-in Raft Commands\n----------------------\nHere are a few commands for monitoring and managing the cluster:\n\n- **RAFTADDPEER addr**  \nAdds a new member to the Raft cluster\n- **RAFTREMOVEPEER addr**  \nRemoves an existing member\n- **RAFTPEERS addr**  \nLists known peers and their status\n- **RAFTLEADER**  \nReturns the Raft leader, if known\n- **RAFTSNAPSHOT**  \nTriggers a snapshot operation\n- **RAFTSTATE**  \nReturns the state of the node\n- **RAFTSTATS**  \nReturns information and statistics for the node and cluster\n\nConsistency and Durability\n--------------------------\n\n### Write Durability\n\nThe `Options.Durability` field has the following options:\n\n- `Low` - fsync is managed by the operating system, less safe\n- `Medium` - fsync every second, fast and safer\n- `High` - fsync after every write, very durable, slower\n\n### Read Consistency\n\nThe `Options.Consistency` field has the following options:\n\n- `Low` - all nodes accept reads, small risk of [stale](http://stackoverflow.com/questions/1563319/what-is-stale-state) data\n- `Medium` - only the leader accepts reads, itty-bitty risk of stale data during a leadership change\n- `High` - only the leader accepts reads, the raft log index is incremented to guaranteeing no stale data\n\nFor example, setting the following options:\n\n```go\nopts := finn.Options{\n\tConsistency: High,\n\tDurability: High,\n}\nn, err := finn.Open(\"data\", \":7481\", \"\", \u0026opts)\n```\n\nProvides the highest level of durability and consistency.\n\nLog Backends\n------------\nFinn supports the following log databases.\n\n- [FastLog](https://github.com/tidwall/raft-fastlog) - log is stored in memory and persists to disk, very fast reads and writes, log is limited to the amount of server memory.\n- [LevelDB](https://github.com/syndtr/goleveldb) - log is stored only to disk, supports large logs.\n- [Bolt](https://github.com/boltdb/bolt) - log is stored only to disk, supports large logs.\n\nContact\n-------\nJosh Baker [@tidwall](http://twitter.com/tidwall)\n\nLicense\n-------\nFinn source code is available under the MIT [License](/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftidwall%2Ffinn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftidwall%2Ffinn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftidwall%2Ffinn/lists"}