{"id":25133093,"url":"https://github.com/goncalotomas/consistent-hashing","last_synced_at":"2025-06-20T07:33:30.067Z","repository":{"id":88127854,"uuid":"126917432","full_name":"goncalotomas/consistent-hashing","owner":"goncalotomas","description":"Very simple consistent hashing implementation in Erlang","archived":false,"fork":false,"pushed_at":"2018-03-27T02:35:59.000Z","size":11,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-03T01:27:44.815Z","etag":null,"topics":["consistent-hashing","erlang"],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/goncalotomas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-27T02:32:15.000Z","updated_at":"2018-03-27T16:13:27.000Z","dependencies_parsed_at":"2023-05-18T09:45:58.920Z","dependency_job_id":null,"html_url":"https://github.com/goncalotomas/consistent-hashing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/goncalotomas/consistent-hashing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goncalotomas%2Fconsistent-hashing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goncalotomas%2Fconsistent-hashing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goncalotomas%2Fconsistent-hashing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goncalotomas%2Fconsistent-hashing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/goncalotomas","download_url":"https://codeload.github.com/goncalotomas/consistent-hashing/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goncalotomas%2Fconsistent-hashing/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260901191,"owners_count":23079723,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consistent-hashing","erlang"],"created_at":"2025-02-08T15:34:11.715Z","updated_at":"2025-06-20T07:33:25.054Z","avatar_url":"https://github.com/goncalotomas.png","language":"Erlang","readme":"consistent_hashing\n=====\n\nBasic implementation of [consistent hashing][1]. The implementation and explanation is a product of my interpretation,\nand may be incorrect. If you see that I messed up somewhere in either part and you feel like being cool, please create\nan issue or [drop me an email][2].\n\n## Introduction\n\nConsistent Hashing was developed for distributed caching to solve a particular problem: dynamic adding and/or removal of\nnodes while minimising the amount of keys that need to be moved around. If by any chance you're wondering how to get\nfrom here to something like [Dynamo][3], just be patient and remember the goal is to build a distributed cache. At this\npoint it is best not to think about replication or fault tolerance. Note that if a cache server crashes, it is assumed\nthat you can still fetch the data elsewhere.\n\nThis is an OTP application that uses logging to explain you what is happening. If you use default settings, a ring with\n4 nodes will be created (each running on an Erlang process). You are able to add and remove servers from the ring by\nusing the `consistent_hashing` module.\n\n## Limitations\n\nThere are many ways to improve this code, and keep in mind that this is a very rudimentary implementation with just some\neducational value. No particular attention was given to the efficiency of the data structures and algorithms used. There\nis no concern for replication or fault tolerance as well. Implementing replication should be as simple as defining a\nrule (replicating factor, `k`) and have a single key be inserted at `k` contiguous nodes in the ring.  \nIn order to minimise the birthday problem causing nodes with entries very close to each other in the hash interval, one\ncould use virtual nodes, allowing a single node to register multiple identifiers (e.g. `node1-1@127.0.0.1`,\n`node1-2@127.0.0.1`), which would help further distributing the load among the servers. If a server `a` has twice the\ncapacity as another server `b`, we can register `a` with double the identifiers as `b`.\n\nBuild\n-----\n\n    $ rebar3 compile\n\nRun\n-----\n\n    $ rebar3 shell --name shell@consistent_hashing\n\n\n```erl-sh\n15:14:47.966 [info] Application lager started on node shell@consistent_hashing\n15:14:47.981 [info] Starting server node1 with hash \"14427F20533241D95D4C94AF71FADA9C\"...\n15:14:47.982 [info] Starting server node2 with hash \"8A44B008A23F97B8628B966E30EFCFD0\"...\n15:14:47.982 [info] Starting server node3 with hash \"7676C8E239797C8A69F0EE1889422807\"...\n15:14:47.982 [info] Starting server node4 with hash \"C53C5F838AEBDDB66DE4C6AA4C82E00A\"...\n15:14:47.982 [info] Starting server node5 with hash \"52AC1988EA9BC2E810BBF281F4D64E56\"...\n15:14:47.982 [info] Starting server node6 with hash \"A12D6508A0C0C064C22FC9A10690B144\"...\n15:14:47.982 [info] Starting server node7 with hash \"95E87B8D2512F885EF0F1BDCC3FC541D\"...\n15:14:47.982 [info] Starting server node8 with hash \"E3F21A1267EDD012E71250E5632ABEF5\"...\n15:14:47.982 [info] Starting server node9 with hash \"97F4F378EBDA20F6D0B9915D49DE943F\"...\n15:14:47.982 [info] Starting server node10 with hash \"D254AC8DF316AEFDB557F828101A0D39\"...\n1\u003e consistent_hashing:get(\"key\").\n15:15:15.268 [info] Key hash is \"3C6E0B8A9C15224A8228B9A98CA1531D\", should be in node with hash \"52AC1988EA9BC2E810BBF281F4D64E56\" (node5)\nmiss\n2\u003e consistent_hashing:put(\"key\", \u003c\u003c\"something\"\u003e\u003e).\n15:15:35.368 [info] Key hash is \"3C6E0B8A9C15224A8228B9A98CA1531D\", asking node with hash \"52AC1988EA9BC2E810BBF281F4D64E56\" (node5) to store key\nok\n3\u003e consistent_hashing:add_node(node15, \"node15identifier\").\n15:18:36.997 [info] Checking if there are any keys to migrate from node successor...\n15:18:36.997 [info] Server doesn't have any keys with lower hash value than \"5407FDABA42A5BB4FB2A9AA5A0F5DC9D\", nothing to migrate\nok\n4\u003e %% Any migration that is necessary will be logged to the console\n4\u003e consistent_hashing:remove_node(\"node15identifier\").\nok\n5\u003e consistent_hashing:get(\"key\").\n15:18:55.167 [info] Key hash is \"3C6E0B8A9C15224A8228B9A98CA1531D\", should be in node with hash \"52AC1988EA9BC2E810BBF281F4D64E56\" (node5)\n\u003c\u003c\"something\"\u003e\u003e\n```\n\n[1]: https://www.akamai.com/es/es/multimedia/documents/technical-publication/consistent-hashing-and-random-trees-distributed-caching-protocols-for-relieving-hot-spots-on-the-world-wide-web-technical-publication.pdf\n[2]: mailto:goncalo@goncalotomas.com\n[3]: https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoncalotomas%2Fconsistent-hashing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoncalotomas%2Fconsistent-hashing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoncalotomas%2Fconsistent-hashing/lists"}