{"id":15631223,"url":"https://github.com/ultrabug/uhashring","last_synced_at":"2025-05-16T09:07:11.281Z","repository":{"id":1686391,"uuid":"43653582","full_name":"ultrabug/uhashring","owner":"ultrabug","description":"Full featured consistent hashing python library compatible with ketama","archived":false,"fork":false,"pushed_at":"2025-04-09T14:59:55.000Z","size":88,"stargazers_count":207,"open_issues_count":7,"forks_count":28,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-05-09T12:06:11.582Z","etag":null,"topics":["consistent-hashing","hash-table","ketama","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ultrabug.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":"FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"ultrabug","custom":"https://paypal.me/alexysjacob1"}},"created_at":"2015-10-04T21:31:56.000Z","updated_at":"2025-04-09T14:59:59.000Z","dependencies_parsed_at":"2024-06-18T15:15:25.802Z","dependency_job_id":"c79ebdaa-2a8d-4be1-8ad0-31a23d684447","html_url":"https://github.com/ultrabug/uhashring","commit_stats":{"total_commits":75,"total_committers":9,"mean_commits":8.333333333333334,"dds":"0.33333333333333337","last_synced_commit":"0c6e050d0a5493f00fdfe6ea5f53eaf837f892ab"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ultrabug%2Fuhashring","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ultrabug%2Fuhashring/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ultrabug%2Fuhashring/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ultrabug%2Fuhashring/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ultrabug","download_url":"https://codeload.github.com/ultrabug/uhashring/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254501558,"owners_count":22081528,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consistent-hashing","hash-table","ketama","python"],"created_at":"2024-10-03T10:39:35.749Z","updated_at":"2025-05-16T09:07:06.271Z","avatar_url":"https://github.com/ultrabug.png","language":"Python","funding_links":["https://github.com/sponsors/ultrabug","https://paypal.me/alexysjacob1"],"categories":[],"sub_categories":[],"readme":"# uhashring\n\n![version](https://img.shields.io/pypi/v/uhashring.svg)\n![ci](https://github.com/ultrabug/uhashring/actions/workflows/ci.yml/badge.svg)\n\n**uhashring** implements **consistent hashing** in pure Python.\n\nConsistent hashing is mostly used on distributed\nsystems/caches/databases as this avoid the total reshuffling of your\nkey-node mappings when adding or removing a node in your ring (called\ncontinuum on libketama). More information and details about this can be\nfound in the *literature* section.\n\nThis full featured implementation offers:\n\n-   a lot of **convenient methods** to use your consistent hash ring in\n    real world applications.\n-   simple **integration** with other libs such as memcache through\n    monkey patching.\n-   a full [ketama](https://github.com/RJ/ketama) compatibility if you\n    need to use it (see important mention below).\n-   all the missing functions in the libketama C python binding (which\n    is not even available on pypi) for ketama users.\n-   possibility to **use your own weight and hash functions** if you\n    don't care about the ketama compatibility.\n-   **instance-oriented usage** so you can use your consistent hash ring\n    object directly in your code (see advanced usage).\n-   native **pypy support**, since this is a pure python library.\n-   tests of implementation, key distribution and ketama compatibility.\n\nPer node weight is also supported and will affect the nodes distribution\non the ring.\n\n## Python 2 EOL\n\nIf you need Python 2 support, make sure to use **uhashring==1.2** as\nv1.2 is the last release that will support it.\n\n## IMPORTANT\n\nSince v1.0 **uhashring** default has changed to use a md5 hash function\nwith 160 vnodes (points) per node in the ring.\n\nThis change was motivated by the fact that the ketama hash function has\nmore chances of collisions and thus requires a complete ring\nregeneration when the nodes topology change. This could lead to degraded\nperformances on rapidly changing or unstable environments where nodes\nkeep going down and up. The md5 implementation provides a linear\nperformance when adding or removing a node from the ring!\n\nReminder: when using **uhashring** with the ketama implementation and\nget 40 vnodes and 4 replicas = 160 points per node in the ring.\n\n## Usage\n\n### Basic usage\n\n**uhashring** is very simple and efficient to use:\n\n```python\nfrom uhashring import HashRing\n\n# create a consistent hash ring of 3 nodes of weight 1\nhr = HashRing(nodes=['node1', 'node2', 'node3'])\n\n# get the node name for the 'coconut' key\ntarget_node = hr.get_node('coconut')\n```\n\n### Ketama usage\n\nSimply set the **hash_fn** parameter to **ketama**:\n\n```python\nfrom uhashring import HashRing\n\n# create a consistent hash ring of 3 nodes of weight 1\nhr = HashRing(nodes=['node1', 'node2', 'node3'], hash_fn='ketama')\n\n# get the node name for the 'coconut' key\ntarget_node = hr.get_node('coconut')\n```\n\n### Advanced usage\n\n```python\nfrom uhashring import HashRing\n\n# Mapping of dict configs\n# Omitted config keys will get a default value, so\n# you only need to worry about the ones you need\nnodes = {\n    'node1': {\n            'hostname': 'node1.fqdn',\n            'instance': redis.StrictRedis(host='node1.fqdn'),\n            'port': 6379,\n            'vnodes': 40,\n            'weight': 1\n        },\n    'node2': {\n            'hostname': 'node2.fqdn',\n            'instance': redis.StrictRedis(host='node2.fqdn'),\n            'port': 6379,\n            'vnodes': 40\n        },\n    'node3': {\n            'hostname': 'node3.fqdn',\n            'instance': redis.StrictRedis(host='node3.fqdn'),\n            'port': 6379\n        }\n    }\n\n# create a new consistent hash ring with the nodes\nhr = HashRing(nodes)\n\n# set the 'coconut' key/value on the right host's redis instance\nhr['coconut'].set('coconut', 'my_value')\n\n# get the 'coconut' key from the right host's redis instance\nhr['coconut'].get('coconut')\n\n# delete the 'coconut' key on the right host's redis instance\nhr['coconut'].delete('coconut')\n\n# get the node config for the 'coconut' key\nconf = hr.get('coconut')\n```\n\n### Default node configuration\n\n**uhashring** offers advanced node configuration for real applications,\nthis is the default you get for every added node:\n\n```python\n{\n    'hostname': nodename,\n    'instance': None,\n    'port': None,\n    'vnodes': 40,\n    'weight': 1\n}\n```\n\n### Adding / removing nodes\n\nYou can add and remove nodes from your consistent hash ring at any time.\n\n```python\nfrom uhashring import HashRing\n\n# this is a 3 nodes consistent hash ring\nhr = HashRing(nodes=['node1', 'node2', 'node3'])\n\n# this becomes a 2 nodes consistent hash ring\nhr.remove_node('node2')\n\n# add back node2\nhr.add_node('node2')\n\n# add node4 with a weight of 10\nhr.add_node('node4', {'weight': 10})\n```\n\n### Customizable node weight calculation\n\n```python\nfrom uhashring import HashRing\n\ndef weight_fn(**conf):\n    \"\"\"Returns the last digit of the node name as its weight.\n\n    :param conf: node configuration in the ring, example:\n        {\n         'hostname': 'node3',\n         'instance': None,\n         'nodename': 'node3',\n         'port': None,\n         'vnodes': 40,\n         'weight': 1\n        }\n    \"\"\"\n    return int(conf['nodename'][-1])\n\n# this is a 3 nodes consistent hash ring with user defined weight function\nhr = HashRing(nodes=['node1', 'node2', 'node3'], weight_fn=weight_fn)\n\n# distribution with custom weight assignment\nprint(hr.distribution)\n\n# \u003e\u003e\u003e Counter({'node3': 240, 'node2': 160, 'node1': 80})\n```\n\n### Customizable hash function\n\n```python\nfrom uhashring import HashRing\n\n# import your own hash function (must be a callable)\n# in this example, MurmurHash v3\nfrom mmh3 import hash as m3h\n\n# this is a 3 nodes consistent hash ring with user defined hash function\nhr = HashRing(nodes=['node1', 'node2', 'node3'], hash_fn=m3h)\n\n# now all lookup operations will use the m3h hash function\nprint(hr.get_node('my key hashed by your function'))\n```\n\n### HashRing options\n\n-   **nodes**: nodes used to create the continuum (see doc for format).\n-   **hash_fn**: use this callable function to hash keys, can be set to\n    'ketama' to use the ketama compatible implementation.\n-   **vnodes**: default number of vnodes per node.\n-   **weight_fn**: user provided function to calculate the node's\n    weight, gets the node conf dict as kwargs.\n-   **replicas**: use this to change ketama ring replicas (default: 4)\n\n### Available methods\n\n-   **add_node(nodename, conf)**: add (or overwrite) the node in the\n    ring with the given config.\n-   **get(key)**: returns the node object dict matching the hashed key.\n-   **get_key(key)**: alias of the current hashi method, returns the\n    hash of the given key.\n-   **get_instances()**: returns a list of the instances of all the\n    configured nodes.\n-   **get_node(key)**: returns the node name of the node matching the\n    hashed key.\n-   **get_node_hostname(key)**: returns the hostname of the node\n    matching the hashed key.\n-   **get_node_instance(key)**: returns the instance of the node\n    matching the hashed key.\n-   **get_node_port(key)**: returns the port of the node matching the\n    hashed key.\n-   **get_node_pos(key)**: returns the index position of the node\n    matching the hashed key.\n-   **get_node_weight(key)**: returns the weight of the node matching\n    the hashed key.\n-   **get_nodes()**: returns a list of the names of all the configured\n    nodes.\n-   **get_points()**: returns a ketama compatible list of (position,\n    nodename) tuples.\n-   **get_server(key)**: returns a ketama compatible (position,\n    nodename) tuple.\n-   **hashi(key)**: returns the hash of the given key (on ketama mode,\n    this is the same as libketama).\n-   **iterate_nodes(key, distinct)**: hash_ring compatibility\n    implementation, same as range but returns tuples as a generator.\n-   **print_continuum()**: prints a ketama compatible continuum report.\n-   **range(key, size, unique)**: returns a (unique) list of max (size)\n    nodes' configuration available in the consistent hash ring.\n-   **regenerate**: regenerate the ring from the current nodes\n    configuration, useful only when using *weight_fn*.\n-   **remove_node(nodename)**: remove the given node from the ring\n\n### Available properties\n\n-   **conf**: dict of all the nodes and their configuration.\n-   **continuum**: same as ring.\n-   **distribution**: counter of the nodes distribution in the\n    consistent hash ring.\n-   **nodes**: same as conf.\n-   **ring**: hash key/node mapping of the consistent hash ring.\n-   **size**: size of the consistent hash ring.\n\n## Integration (monkey patching)\n\nYou can benefit from a consistent hash ring using **uhashring** monkey\npatching on the following libraries:\n\n### python-memcached\n\n```python\nimport memcache\n\nfrom uhashring import monkey\nmonkey.patch_memcache()\n\nmc = memcache.Client(['node1:11211', 'node2:11211'])\n```\n\n## Installation\n\n### Pypi\n\nUsing pip:\n\n```bash\n$ pip install uhashring\n```\n\n### Gentoo Linux\n\nUsing emerge:\n\n```bash\n$ sudo emerge -a uhashring\n```\n\n## Benchmark\n\nUsage of the ketama compatible hash (default) has some performance\nimpacts. Contributions are welcome as to ways of improving this !\n\n\u003e There is a big performance gap in the hash calculation between the\n\u003e ketama C binding and its pure python counterpart.\n\u003e\n\u003e Python 3 is doing way better than python 2 thanks to its native\n\u003e bytes/int representation.\n\u003e\n\u003e ***Quick benchmark, for 1 million generated ketama compatible keys:***\n\u003e -   python_ketama C binding: 0.8427069187164307 s\n\u003e -   python 2: 5.462762832641602 s\n\u003e -   python 3: 3.570068597793579 s\n\u003e -   pypy: 1.6146340370178223 s\n\u003e\n\u003e When using python 2 and ketama compatibility is not important, you can\n\u003e get a better hashing speed using the other provided hashing.\n\u003e\n\u003e hr = HashRing(nodes=[], compat=False)\n\u003e\n\u003e ***Quick benchmark, for 1 million generated hash keys:***\n\u003e -   python 2: 3.7595579624176025 s\n\u003e -   python 3: 3.268343687057495 s\n\u003e -   pypy: 1.9193649291992188 s\n\n## Literature\n\n-   consistent hashing:\n    \u003chttps://en.wikipedia.org/wiki/Consistent_hashing\u003e\n-   web caching paper:\n    \u003chttp://www8.org/w8-papers/2a-webserver/caching/paper2.html\u003e\n-   research paper:\n    \u003chttp://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.23.3738\u003e\n-   distributed hash table:\n    \u003chttps://en.wikipedia.org/wiki/Distributed_hash_table\u003e\n\n## License\n\nBSD\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fultrabug%2Fuhashring","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fultrabug%2Fuhashring","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fultrabug%2Fuhashring/lists"}