{"id":13412520,"url":"https://github.com/mbrostami/consistenthash","last_synced_at":"2026-01-27T18:04:35.110Z","repository":{"id":144210228,"uuid":"257950901","full_name":"mbrostami/consistenthash","owner":"mbrostami","description":"A Go library that implements Consistent Hashing (+Block Partitioning)","archived":false,"fork":false,"pushed_at":"2024-04-04T01:08:41.000Z","size":38,"stargazers_count":25,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-04-22T13:32:02.717Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mbrostami.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-22T16:01:25.000Z","updated_at":"2024-06-21T13:04:06.060Z","dependencies_parsed_at":"2024-01-07T21:54:06.021Z","dependency_job_id":"aa9770b1-780b-4d53-bd7c-f6395e83d463","html_url":"https://github.com/mbrostami/consistenthash","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/mbrostami/consistenthash","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbrostami%2Fconsistenthash","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbrostami%2Fconsistenthash/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbrostami%2Fconsistenthash/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbrostami%2Fconsistenthash/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mbrostami","download_url":"https://codeload.github.com/mbrostami/consistenthash/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbrostami%2Fconsistenthash/sbom","scorecard":{"id":631408,"data":{"date":"2025-08-11","repo":{"name":"github.com/mbrostami/consistenthash","commit":"eeffd0ceeecc2a82a1549139c4c5fc0111f27b09"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":0,"reason":"Found 0/25 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 6 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-21T07:59:17.198Z","repository_id":144210228,"created_at":"2025-08-21T07:59:17.198Z","updated_at":"2025-08-21T07:59:17.198Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28817795,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T18:01:38.485Z","status":"ssl_error","status_checked_at":"2026-01-27T18:01:27.499Z","response_time":168,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T20:01:25.635Z","updated_at":"2026-01-27T18:04:35.093Z","avatar_url":"https://github.com/mbrostami.png","language":"Go","readme":"[![Build Status](https://travis-ci.com/mbrostami/consistenthash.svg?branch=master)](https://travis-ci.com/mbrostami/consistenthash)\n[![Go Report Card](https://goreportcard.com/badge/github.com/mbrostami/consistenthash)](https://goreportcard.com/report/github.com/mbrostami/consistenthash)\n\n# Consistent Hashing\n\nA Go library that implements Consistent Hashing with zero dependency   \nThis package is implemented based on [golang/groupcache](https://github.com/golang/groupcache) with some performance improvements\n\n### Improvements:\n\n- `Remove` function added - sort and remove a node (and replicas) from the hash ring\n- int hashes replaced with uint32\n- Number of replicas is now configurable while adding new node (useful when capacity is not the same for all nodes)\n\n# Technical Details:\nA **HashMap** that maps the hash of the key to the original values, something like `0xFF =\u003e \"node number one\"`  \n\nA **BlockMap** that has dynamic number of blocks and each block has sorted list of items, and each item has Key and Pointer to the HashMap. The block number is calculated as:   \n`blockNumber = hash(key) / (MaxUint32 / number of blocks)`  \nThis will lead us to have a kind of sorted blocks.  \n\nA **ReplicaMap** that keeps the number of replicas for each key ONLY if the replicas for specific key is different than the default replicas. So if you have 10k keys with default replica set to 100 and you add a new key with 120 replicas, the ReplicaMap will have 1 record.\n\n### Re-Balancing:\nTo add a new record we need to know how many blocks we should have. Considering current number of keys and a control option D we will have number of keys divided by D as number of expected blocks.  \n\nAs the result will change by adding D number of new keys we will need a re-balancing mechanism to add or remove blocks. To make it efficient we can start re-balancing if the expected blocks is twice more or less than current number of blocks.  \n\n**As an example**:  \nIf we have D = 200 with 100k existing keys, there will be 1000 blocks each contains approximately 500 keys. Adding 200 new keys will require us to have 1001 blocks. So expected blocks is 1001 but current is 1000. As we add more keys the expected blocks becomes 2001 and current remains 1000. In this case re-balancing process starts and adds 1001 more blocks. As a result we will have 2001 blocks. Next re-balancing will happen when we need 4002 blocks and so on.  \n\nThis process happens as follows:  \nLooping over blocks from the last one. Looping over items in each block starting from end, calculating the new block number for each key. Because items are sorted, as we reach to a key that doesn't belong to a different block, we shift items to the target block.  \nHere is an example:  \n```\nmax hash value = 1000\ntotal keys = 100\nD = 5\nblockSize = 1000 / (100/5) = 50\nblockNumber = hash / blockSize\nNumber of blocks: 1000 / blockSize = 20\n10  / 50 =\u003e 0\n20  / 50 =\u003e 0\n50  / 50 =\u003e 1\n...\n1000 / 50 =\u003e 20\n\nBlock[0]  = [10,20,30,40]\nBlock[1]  = [50,60,70,80]\nBlock[4]  = [210,220,230,240]\n...\n\nRe-balancing:\nnewBlockSize = 1000/ (200/5) = 25\nnewBlockNumber = hash / newBlockSize\nExpected blocks: 1000 / newBlockSize = 40\niter b20: // starting from last block\n...\niter b4:\nBlock[0]  = [10,20,30,40]\nBlock[1]  = [50,60,70,80]\nBlock[8]  = [210,220]  // moved from block[4]\nBlock[9]  = [230,240]  // moved from block[4]\nitr b1:\nBlock[0]  = [10,20,30,40]\nBlock[2]  = [50,60,70] // moved from block[1]\nBlock[3]  = [80]       // moved from block[1]\nBlock[8]  = [210,220]  // moved from block[4]\nBlock[9]  = [230,240]  // moved from block[4]\nitr b0:\nBlock[0]  = [10,20]\nBlock[1]  = [30,40].   // moved from block[0]\nBlock[2]  = [50,60,70] // moved from block[1]\nBlock[3]  = [80]       // moved from block[1]\nBlock[8]  = [210,220]  // moved from block[4]\nBlock[9]  = [230,240]  // moved from block[4]\n```\nTime complexity of the above would be O(n) in worst case where n is the number of keys.  \nYou can set D value by passing an option to constructor: `WithBlockPartitioning(5)`  \n\n### Adding Item:\nTime Complexity is between O(1) and O(log k) where k is the maximum number of keys in a block. (excluding shifting items)\n\nTo add a new item to a block, we need to calculate the block number and do a binary search to find the position where we want to insert the item, and update the list in that block. If the item is the original node, we need to add the key and value to the HashMap as well. All the items in the block (including replicas) will use the pointer to the HashMap.  \n\n### Removing Item:\nTime Complexity is between O(1) and O(log k) where k is the maximum number of keys in a block. (excluding shifting items) \n\nTo remove an item, we find the block number using hash of the key, doing a binary search in the block, finding removing the item in the block. The re-balancing might be necessary if we remove 1/2 of all the stored keys.  \n\n### Lookup:\nTime Complexity is between O(1) and O(log k) where k is the maximum number of keys in a block.  \n\nTo find a key, we find the block number using hash of the key, doing a binary search to find the closest hash to the lookup key. You might go to the next non-empty block and get the first item.  \n\n# Usage\n\n`go get github.com/mbrostami/consistenthash/v2`\n\n### Simple use case\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/mbrostami/consistenthash/v2\"\n)\n\nfunc main() {\n\n\tch := consistenthash.New(consistenthash.WithDefaultReplicas(10))\n\tch.Add([]byte(\"templateA\"), []byte(\"templateB\"))\n\n\tassignedTemplate := ch.Get([]byte(\"userA\")) // assigned template should always be the same for `userA`\n\tfmt.Printf(\"assigned: %s\", assignedTemplate)\n}\n```\n\n### Weighted load\n\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/mbrostami/consistenthash/v2\"\n)\n\nfunc main() {\n\tch := consistenthash.New(consistenthash.WithDefaultReplicas(10))\n\tch.Add([]byte(\"127.0.0.1:1001\"), []byte(\"127.0.0.1:1002\"))\n\tch.AddReplicas(40, []byte(\"127.0.0.1:1003\")) // 4x more weight\n\n\trKey := \"something like request url or user id\"\n\tnode := ch.Get([]byte(rKey)) // find upper closest node\n\tfmt.Println(node) // this will print out one of the nodes\n}\n\n```\n\n## More about consistent hashing  \n\nYou can find some explanation in this blog post: https://liuzhenglaichn.gitbook.io/system-design/advanced/consistent-hashing  \n\nThe code example to achieve a hash ring similar to the following picture:  \n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/mbrostami/consistenthash/v2\"\n)\n\n\nfunc main() {\n\tch := consistenthash.New(consistenthash.WithDefaultReplicas(3))\n\tch.Add([]byte(\"A\"), []byte(\"B\"), []byte(\"C\"))\n\n\tfmt.Println(ch.Get([]byte(\"Alica\")))\n\tfmt.Println(ch.Get([]byte(\"Bob\")))\n\tfmt.Println(ch.Get([]byte(\"Casey\")))\n\n}\n\n```\n\n![Image](https://1865312850-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M4Bkp-b8HYQgJF1rkOc%2F-M5FwA4YIBAqAjZvdVpU%2F-M5FyQai2CtC2j4GGr5K%2Fimage.png?alt=media\u0026token=77d5d346-f37f-4f28-8f64-66a1627d2deb)\n","funding_links":[],"categories":["Relational Databases","分布式系统","Distributed Systems","Go"],"sub_categories":["检索及分析资料库","Search and Analytic Databases","SQL 查询语句构建库","Advanced Console UIs"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmbrostami%2Fconsistenthash","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmbrostami%2Fconsistenthash","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmbrostami%2Fconsistenthash/lists"}