{"id":30679709,"url":"https://github.com/monsieurtib/keyed-semaphore","last_synced_at":"2025-09-10T18:46:07.992Z","repository":{"id":291138891,"uuid":"976689320","full_name":"MonsieurTib/keyed-semaphore","owner":"MonsieurTib","description":"A Go library providing context-aware semaphores where concurrency limits are applied per unique arbitrary key, enabling fine-grained resource locking.","archived":false,"fork":false,"pushed_at":"2025-05-09T17:07:31.000Z","size":32,"stargazers_count":31,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-12T07:45:33.284Z","etag":null,"topics":["concurrency-control","go","golang","rate-limiting","resource-locker","semaphore"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MonsieurTib.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-02T15:07:30.000Z","updated_at":"2025-06-03T18:18:42.000Z","dependencies_parsed_at":"2025-05-02T17:29:38.644Z","dependency_job_id":null,"html_url":"https://github.com/MonsieurTib/keyed-semaphore","commit_stats":null,"previous_names":["monsieurtib/keyed-semaphore"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MonsieurTib/keyed-semaphore","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MonsieurTib%2Fkeyed-semaphore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MonsieurTib%2Fkeyed-semaphore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MonsieurTib%2Fkeyed-semaphore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MonsieurTib%2Fkeyed-semaphore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MonsieurTib","download_url":"https://codeload.github.com/MonsieurTib/keyed-semaphore/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MonsieurTib%2Fkeyed-semaphore/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273143429,"owners_count":25053123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-01T02:00:09.058Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency-control","go","golang","rate-limiting","resource-locker","semaphore"],"created_at":"2025-09-01T14:44:38.294Z","updated_at":"2025-09-01T14:44:40.504Z","avatar_url":"https://github.com/MonsieurTib.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Build](https://github.com/MonsieurTib/keyed-semaphore/actions/workflows/go.yml/badge.svg)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=MonsieurTib_keyed-semaphore\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=MonsieurTib_keyed-semaphore)\n[![Go Report Card](https://goreportcard.com/badge/github.com/MonsieurTib/keyed-semaphore)](https://goreportcard.com/report/github.com/MonsieurTib/keyed-semaphore)\n\n\n# Keyed Semaphore for Go\n\n`keyed-semaphore` provides a semaphore implementation in Go where the locking is based on arbitrary keys. This allows you to limit concurrency for operations associated with specific identifiers, rather than just globally.\nFor example, you can use it to limit the number of concurrent operations per user ID, per resource ID, or any other comparable key type.\n\n## Features\n\n*   **Key-Based Concurrency Limiting:** Limit concurrent access based on generic keys (any Go `comparable` type).\n*   **Sharded Implementation for Enhanced Scalability:** Includes a `ShardedKeyedSemaphore` that distributes keys across multiple internal `KeyedSemaphore` instances. This significantly reduces lock contention and improves performance, especially under high load with many unique keys.\n*   **Configurable Concurrency:** Set the maximum number of concurrent acquirers *per key*.\n*   **Context-Aware Waiting:** The `Wait` method respects `context.Context` for cancellation and deadlines.\n*   **Non-Blocking TryWait:** A `TryWait` method is available to attempt acquiring the semaphore without blocking, also respecting `context.Context`.\n*   **Dynamic Creation:** Semaphores for keys are created on demand when first accessed.\n*   **Robust Memory Management:** Implemented reference counting for automatic and safe cleanup of internal resources associated with a key when it's no longer in active use, preventing memory leaks.\n*   **Improved Concurrency Safety:** Hardened against race conditions for reliable behavior under high concurrent access.\n\n## Installation\n\n```bash\ngo get github.com/MonsieurTib/keyed-semaphore\n```\n\n## Usage\n\n### Basic KeyedSemaphore\n\nThe following example demonstrates using `KeyedSemaphore` with string keys. Note that `KeyedSemaphore` now supports generic key types.\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"fmt\"\n    ks \"github.com/MonsieurTib/keyed-semaphore\"\n    \"sync\"\n    \"time\"\n)\n\nfunc worker(id int, resourceID string, semaphore *ks.KeyedSemaphore[string], wg *sync.WaitGroup) {\n    defer wg.Done()\n    ctx := context.Background()\n    fmt.Printf(\"Worker %d: Attempting lock for resource '%s'...\\n\", id, resourceID)\n\n    err := semaphore.Wait(ctx, resourceID) \n    if err != nil {\n       fmt.Printf(\"Worker %d: Failed lock for resource '%s': %v\\n\", id, resourceID, err)\n       return\n    }\n\n    fmt.Printf(\"Worker %d: Acquired lock for resource '%s'. Working...\\n\", id, resourceID)\n    time.Sleep(time.Second * 1) // Simulate work\n    fmt.Printf(\"Worker %d: Work done. Releasing lock for resource '%s'.\\n\", id, resourceID)\n\n    err = semaphore.Release(resourceID)\n    if err != nil {\n       fmt.Printf(\"Worker %d: Failed to release lock for resource '%s': %v\\n\", id, resourceID, err)\n    }\n}\n\nfunc main() {\n    // Create a new KeyedSemaphore for string keys, allowing 2 concurrent operations per key.\n    maxConcurrentPerKey := 2\n    semaphore := ks.NewKeyedSemaphore[string](maxConcurrentPerKey)\n\n    var wg sync.WaitGroup\n    numWorkers := 5\n    resourceKey1 := \"document-123\"\n\n    fmt.Printf(\"Starting %d workers for resource '%s' (max %d concurrent)...\\n\",\n       numWorkers, resourceKey1, maxConcurrentPerKey)\n\n    for i := 1; i \u003c= numWorkers; i++ {\n    wg.Add(1)\n       go worker(i, resourceKey1, semaphore, \u0026wg)\n    }\n\n    // Example with a different key type (if you were using int keys)\n    // type UserID int\n    // userIDKey := UserID(42)\n    // semaphoreInt := ks.NewKeyedSemaphore[UserID](maxConcurrentPerKey)\n    // ... then use semaphoreInt with userIDKey ...\n\n    wg.Wait()\n    fmt.Println(\"All workers finished for resourceKey1.\")\n}\n\n```\n\n### Sharded KeyedSemaphore for Higher Concurrency\n\nFor scenarios with a very large number of unique keys or extremely high contention, `ShardedKeyedSemaphore` can provide better performance by dividing keys among several independent `KeyedSemaphore` instances (shards).\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"fmt\"\n    ks \"github.com/MonsieurTib/keyed-semaphore\" \n    \"strconv\"\n    \"sync\"\n    \"time\"\n)\n\nfunc shardedWorker(id int, resourceID string, shardedSem *ks.ShardedKeyedSemaphore[string], wg *sync.WaitGroup) {\n    defer wg.Done()\n    ctx := context.Background()\n    fmt.Printf(\"ShardedWorker %d: Attempting lock for resource '%s'...\\n\", id, resourceID)\n    shard := shardedSem.GetShard(resourceID)\n    err := shard.Wait(ctx, resourceID)\n    if err != nil {\n       fmt.Printf(\"ShardedWorker %d: Failed lock for resource '%s': %v\\n\", id, resourceID, err)\n       return\n    }\n\n    fmt.Printf(\"ShardedWorker %d: Acquired lock for resource '%s'. Working...\\n\", id, resourceID)\n    time.Sleep(time.Millisecond * 500) // Simulate work\n    fmt.Printf(\"ShardedWorker %d: Work done. Releasing lock for resource '%s'.\\n\", id, resourceID)\n\n    err = shard.Release(resourceID) \n    if err != nil {\n       fmt.Printf(\"ShardedWorker %d: Failed to release lock for resource '%s': %v\\n\", id, resourceID, err)\n    }\n}\n\nfunc main() {\n    shardCount := 16 // Number of internal shards\n    maxConcurrentPerKey := 2\n\n    // For string keys, you can use the provided HashString function or your own.\n    shardedSemaphore := ks.NewShardedKeyedSemaphore[string](shardCount, maxConcurrentPerKey, ks.HashString)\n\n    var wg sync.WaitGroup\n    numWorkers := 20 \n\n    fmt.Printf(\"Starting %d sharded workers (max %d concurrent per key, across %d shards)...\\n\",\n       numWorkers, maxConcurrentPerKey, shardCount)\n\n    for i := 1; i \u003c= numWorkers; i++ {\n       wg.Add(1)\n       // Simulate different keys that might fall into different shards\n       resourceKey := \"item-\" + strconv.Itoa(i%5)\n\n       // Pass the main shardedSemaphore instance to the worker\n       go shardedWorker(i, resourceKey, shardedSemaphore, \u0026wg)\n    }\n\n    wg.Wait()\n    fmt.Println(\"All sharded workers finished.\")\n}\n```\n\n## API Overview\n\n### Type Definition for Hasher (used by ShardedKeyedSemaphore)\n`type Hasher[K comparable] func(K) uint64`\n* A function type that takes a key of generic type `K` and returns a `uint64` hash value.\n* `HashString(key string) uint64` is provided as a convenient hasher for string keys using `xxhash`.\n\n### `KeyedSemaphore[K comparable]`\n\n*   `NewKeyedSemaphore[K comparable](maxSize int) *KeyedSemaphore[K]`: Creates a new keyed semaphore manager. `maxSize` defines the maximum concurrent holders for *each* key. `K` can be any Go `comparable` type.\n*   `Wait(ctx context.Context, key K) error`: Waits to acquire the semaphore for the given `key`. Blocks until acquisition is possible or the `ctx` is cancelled/times out. Returns `ctx.Err()` on cancellation/timeout.\n*   `TryWait(ctx context.Context, key K) bool`: Attempts to acquire the semaphore for the given `key` without blocking. Respects `context.Context` for early cancellation. Returns `true` if successful, `false` otherwise.\n*   `Release(key K) error`: Releases the semaphore for the given `key`. Returns an error if the semaphore for the key was not previously acquired or if issues occur during release.\n\n### `ShardedKeyedSemaphore[K comparable]`\n\n*   `NewShardedKeyedSemaphore[K comparable](shardCount, maxSize int, hasher Hasher[K]) *ShardedKeyedSemaphore[K]`: Creates a new sharded keyed semaphore manager.\n    *   `shardCount`: The number of internal `KeyedSemaphore` instances (shards).\n    *   `maxSize`: The maximum concurrent holders for *each* key within its designated shard.\n    *   `hasher`: A `Hasher[K]` function to distribute keys among shards.\n*   `(sks *ShardedKeyedSemaphore[K]) GetShard(key K) *KeyedSemaphore[K]`: Returns the specific `KeyedSemaphore` instance (shard) responsible for the given `key`. Operations (`Wait`, `TryWait`, `Release`) should then be called on this returned shard.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit issues and pull requests.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmonsieurtib%2Fkeyed-semaphore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmonsieurtib%2Fkeyed-semaphore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmonsieurtib%2Fkeyed-semaphore/lists"}