{"id":23108895,"url":"https://github.com/archishmansengupta/consistent-hashing","last_synced_at":"2025-06-17T02:33:29.673Z","repository":{"id":247148376,"uuid":"824968122","full_name":"ArchishmanSengupta/consistent-hashing","owner":"ArchishmanSengupta","description":"A Thread-safe Go library for distributed load balancing using consistent hashing with bounded loads","archived":false,"fork":false,"pushed_at":"2024-09-16T15:46:09.000Z","size":56,"stargazers_count":12,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-09-17T05:49:57.618Z","etag":null,"topics":["consistent-hashing","distributed-systems","golang","hash","load-balancing","murmurhash3","thread-safe"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ArchishmanSengupta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-06T12:29:40.000Z","updated_at":"2024-09-16T15:44:02.000Z","dependencies_parsed_at":"2024-07-06T22:40:18.708Z","dependency_job_id":"2b7e3fe7-d597-4068-b9f0-514b969bd7c6","html_url":"https://github.com/ArchishmanSengupta/consistent-hashing","commit_stats":null,"previous_names":["archishmansengupta/consistent-hashing"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArchishmanSengupta%2Fconsistent-hashing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArchishmanSengupta%2Fconsistent-hashing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArchishmanSengupta%2Fconsistent-hashing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArchishmanSengupta%2Fconsistent-hashing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ArchishmanSengupta","download_url":"https://codeload.github.com/ArchishmanSengupta/consistent-hashing/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230048473,"owners_count":18164744,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consistent-hashing","distributed-systems","golang","hash","load-balancing","murmurhash3","thread-safe"],"created_at":"2024-12-17T01:31:51.217Z","updated_at":"2024-12-17T01:31:51.854Z","avatar_url":"https://github.com/ArchishmanSengupta.png","language":"Go","readme":"---\n\n# Consistent-Hashing\n\nConsistent-Hashing is a Go library designed for distributed load balancing using consistent hashing, enhanced with bounded loads. It provides efficient key distribution across a set of hosts while ensuring that no single host becomes overloaded beyond a specified limit.\n\n## Installation\n\nTo install the package, execute the following command:\n\n```bash\ngo get github.com/ArchishmanSengupta/consistent-hashing\n```\n\n## Usage\n\nHere's a straightforward example of how to utilize the `consistent_hashing` package:\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/ArchishmanSengupta/consistent-hashing\"\n)\n\nfunc main() {\n\t// Create a new ConsistentHashing instance with default configuration\n\tch, err := consistent_hashing.NewWithConfig(consistent_hashing.Config{})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// Add hosts to the consistent hash ring\n\thosts := []string{\"host1\", \"host2\", \"host3\", \"host4\"}\n\tctx := context.Background()\n\tfor _, host := range hosts {\n\t\terr := ch.Add(ctx, host)\n\t\tif err != nil {\n\t\t\tlog.Printf(\"Error adding host %s: %v\", host, err)\n\t\t}\n\t}\n\n\t// Distribute some keys\n\tkeys := []string{\"key1\", \"key2\", \"key3\", \"key4\", \"key5\"}\n\tfor _, key := range keys {\n\t\thost, err := ch.GetLeast(ctx, key)\n\t\tif err != nil {\n\t\t\tlog.Printf(\"Error getting host for key %s: %v\", key, err)\n\t\t\tcontinue\n\t\t}\n\t\tfmt.Printf(\"Key %s assigned to host %s\\n\", key, host)\n\t\t\n\t\t// Increase the load for the assigned host\n\t\terr = ch.IncreaseLoad(ctx, host)\n\t\tif err != nil {\n\t\t\tlog.Printf(\"Error increasing load for host %s: %v\", host, err)\n\t\t}\n\t}\n\n\t// Print current loads\n\tloads := ch.GetLoads()\n\tfmt.Println(\"Current loads:\")\n\tfor host, load := range loads {\n\t\tfmt.Printf(\"%s: %d\\n\", host, load)\n\t}\n\n\t// Remove a host from the ring\n\thostToRemove := \"host2\"\n\terr = ch.Remove(ctx, hostToRemove)\n\tif err != nil {\n\t\tlog.Printf(\"Error removing host %s: %v\", hostToRemove, err)\n\t}\n\n\t// Print updated list of hosts\n\tfmt.Println(\"Remaining hosts after removal:\", ch.Hosts())\n}\n```\n## Output\n```\nKey key1 assigned to host host4\nKey key2 assigned to host host3\nKey key3 assigned to host host2\nKey key4 assigned to host host1\nKey key5 assigned to host host4\nCurrent loads:\nhost2: 1\nhost3: 1\nhost4: 2\nhost1: 1\nRemaining hosts after removal: [host1 host3 host4]\n```\n\n## Features\n\n- **Consistent Hashing with Bounded Loads**: Distributes load evenly across hosts while limiting maximum host load.\n- **Customizable Configuration**: Adjust replication factor, load factor, and hash function to suit specific requirements.\n- **Thread-Safe Operations**: Ensures safe concurrent access for adding hosts, distributing keys, and managing loads.\n- **Efficient Key Distribution**: Uses consistent hashing principles for efficient key assignment and lookup.\n\n## Configuration\n\nCustomize consistent hashing behavior by providing a `Config` struct during instance creation:\n\n```go\ncfg := consistent_hashing.Config{\n    ReplicationFactor: 20,    // Number of virtual nodes per host\n    LoadFactor:        1.25,  // Maximum load factor before redistribution\n    HashFunction:      fnv.New64a, // Custom hash function (optional)\n}\n\nch, err := consistent_hashing.NewWithConfig(cfg)\n```\n\n## Benchmarking\n\nUse the following command to run benchmarks:\n\n```bash\ngo test -bench=. -benchmem\n```\n\nExample benchmark results:\n\n```\ngoos: darwin\ngoarch: arm64\npkg: github.com/ArchishmanSengupta/consistent-hashing\nBenchmarkAdd-10                             4626          19088168 ns/op           23748 B/op        895 allocs/op\nBenchmarkGet-10                          6359968               186.5 ns/op            47 B/op          4 allocs/op\nBenchmarkGetLeast-10                         180           6606643 ns/op              24 B/op          3 allocs/op\nBenchmarkIncreaseLoad-10                13727469                83.96 ns/op           13 B/op          1 allocs/op\nBenchmarkRemove-10                           139           8671612 ns/op           17696 B/op       1306 allocs/op\nBenchmarkParallelOperations-10               884           1297025 ns/op             123 B/op          9 allocs/op\nPASS\nok      github.com/ArchishmanSengupta/consistent-hashing        160.723s\n```\n\n## API Reference\n\n### Methods\n\n- `NewWithConfig(cfg Config) (*ConsistentHashing, error)`: Creates a new instance of ConsistentHashing with specified configuration.\n- `Add(ctx context.Context, host string) error`: Adds a new host to the consistent hash ring.\n- `Get(ctx context.Context, key string) (string, error)`: Retrieves the host responsible for a given key.\n- `GetLeast(ctx context.Context, key string) (string, error)`: Retrieves the least loaded host for a given key.\n- `IncreaseLoad(ctx context.Context, host string) error`: Increases the load for a specified host.\n- `DecreaseLoad(ctx context.Context, host string) error`: Decreases the load for a specified host.\n- `GetLoads() map[string]int64`: Retrieves the current load for all hosts.\n- `Hosts() []string`: Retrieves the list of all hosts in the ring.\n- `Remove(ctx context.Context, host string) error`: Removes a host from the ring.\n\n## Examples\n\n### Adding and Removing Hosts\n\n```go\nch, _ := consistent_hashing.NewWithConfig(consistent_hashing.Config{})\nctx := context.Background()\n\n// Adding hosts\nch.Add(ctx, \"host1\")\nch.Add(ctx, \"host2\")\nch.Add(ctx, \"host3\")\n\n// Removing a host\nerr := ch.Remove(ctx, \"host2\")\nif err != nil {\n    log.Printf(\"Error removing host: %v\", err)\n}\n\nfmt.Println(\"Current hosts:\", ch.Hosts())\n```\n\n### Distributing Keys with Load Balancing\n\n```go\nch, _ := consistent_hashing.NewWithConfig(consistent_hashing.Config{})\nctx := context.Background()\n\n// Add hosts\nfor i := 1; i \u003c= 5; i++ {\n    ch.Add(ctx, fmt.Sprintf(\"host%d\", i))\n}\n\n// Distribute keys\nkeys := []string{\"user1\", \"user2\", \"user3\", \"user4\", \"user5\", \"user6\", \"user7\", \"user8\", \"user9\", \"user10\"}\nfor _, key := range keys {\n    host, _ := ch.GetLeast(ctx, key)\n    fmt.Printf(\"Key %s assigned to %s\\n\", key, host)\n    ch.IncreaseLoad(ctx, host)\n}\n\n// Print final loads\nloads := ch.GetLoads()\nfor host, load := range loads {\n    fmt.Printf(\"%s load: %d\\n\", host, load)\n}\n```\n\n## Contributing\n\nContributions are welcome! Feel free to submit a Pull Request with your enhancements or bug fixes.\n\n--- \n\nThis README now includes detailed instructions on adding, removing hosts, and showcases example usage scenarios for distributing keys and benchmarking performance. It should provide comprehensive guidance for users looking to integrate and leverage the Consistent-Hashing library effectively.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farchishmansengupta%2Fconsistent-hashing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farchishmansengupta%2Fconsistent-hashing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farchishmansengupta%2Fconsistent-hashing/lists"}