{"id":30773467,"url":"https://github.com/timmikeladze/s3-mutex","last_synced_at":"2026-01-20T16:56:52.247Z","repository":{"id":292051612,"uuid":"953722242","full_name":"TimMikeladze/s3-mutex","owner":"TimMikeladze","description":"A simple distributed locking mechanism for Node.js applications using AWS S3 as the backend storage.","archived":false,"fork":false,"pushed_at":"2025-08-07T14:26:19.000Z","size":198,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-28T11:22:03.810Z","etag":null,"topics":["distributed-locking","lock","s3-lock","s3-locking","s3-mutex"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TimMikeladze.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-24T01:02:26.000Z","updated_at":"2025-08-07T14:26:23.000Z","dependencies_parsed_at":"2025-05-07T22:56:01.248Z","dependency_job_id":"2766a005-89b6-4d8d-b833-2dc7554e700f","html_url":"https://github.com/TimMikeladze/s3-mutex","commit_stats":null,"previous_names":["timmikeladze/s3-mutex"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/TimMikeladze/s3-mutex","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimMikeladze%2Fs3-mutex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimMikeladze%2Fs3-mutex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimMikeladze%2Fs3-mutex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimMikeladze%2Fs3-mutex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TimMikeladze","download_url":"https://codeload.github.com/TimMikeladze/s3-mutex/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimMikeladze%2Fs3-mutex/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273699712,"owners_count":25152285,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-04T02:00:08.968Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-locking","lock","s3-lock","s3-locking","s3-mutex"],"created_at":"2025-09-05T01:52:12.886Z","updated_at":"2026-01-20T16:56:52.242Z","avatar_url":"https://github.com/TimMikeladze.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# S3-Mutex\n\nA distributed locking mechanism for Node.js applications using AWS S3 as the backend storage.\n\n## Features\n\n- **Distributed locking**: Coordinate access across multiple services\n- **Automatic bucket creation**: Optionally create S3 buckets if they don't exist\n- **Deadlock detection**: Priority-based mechanism for deadlock resolution\n- **Timeout handling**: Automatic lock expiration with configurable timeouts\n- **Lock heartbeat**: Automatic lock refresh during long operations\n- **Retry with backoff and jitter**: Configurable retry mechanism\n- **Error handling**: Specific handling for S3 service issues\n- **Cleanup utilities**: Tools for managing stale locks\n\n\u003e **⚠️ Warning**: S3-based locking has significant limitations compared to purpose-built locking solutions. S3 operations have higher latency and are not optimized for high-frequency lock operations. Consider alternatives like Redis, DynamoDB, or ZooKeeper for mission-critical applications.\n\n## Installation\n\n```bash\nnpm install s3-mutex\n# or\nyarn add s3-mutex\n# or\npnpm add s3-mutex\n```\n\n## Usage\n\n### Basic usage\n\n```typescript\nimport { S3Client } from \"@aws-sdk/client-s3\";\nimport { S3Mutex } from \"s3-mutex\";\n\n// Option 1: Initialize with existing S3 client\nconst s3Client = new S3Client({\n  region: \"us-east-1\",\n  // other configuration options\n});\n\nconst mutex = new S3Mutex({\n  s3Client,\n  bucketName: \"my-locks-bucket\",\n  keyPrefix: \"locks/\", // optional, defaults to \"locks/\"\n});\n\n// Option 2: Let S3Mutex create the S3 client\nconst mutex2 = new S3Mutex({\n  bucketName: \"my-locks-bucket\",\n  s3ClientConfig: {\n    region: \"us-east-1\",\n    forcePathStyle: true, // useful for MinIO/LocalStack\n    // other S3ClientConfig options\n  },\n});\n\n// Option 3: Automatically create bucket if it doesn't exist\nconst mutex3 = new S3Mutex({\n  bucketName: \"my-locks-bucket\",\n  createBucketIfNotExists: true, // Create bucket automatically\n  s3ClientConfig: {\n    region: \"us-east-1\",\n    // other S3ClientConfig options\n  },\n});\n\n// Acquire a lock\nconst acquired = await mutex.acquireLock(\"my-resource-lock\");\nif (acquired) {\n  try {\n    // Do work with the exclusive lock\n    await doSomething();\n  } finally {\n    // Release the lock when done\n    await mutex.releaseLock(\"my-resource-lock\");\n  }\n} else {\n  console.log(\"Failed to acquire lock\");\n}\n```\n\n### Using the withLock helper\n\nThe `withLock` helper method simplifies working with locks by automatically releasing them:\n\n```typescript\n// Execute a function with an automatic lock\nconst result = await mutex.withLock(\"my-resource-lock\", async () =\u003e {\n  // This function is executed only when the lock is acquired\n  const data = await processResource();\n  return data;\n});\n\nif (result === null) {\n  // Lock acquisition failed\n  console.log(\"Could not acquire lock\");\n} else {\n  // Lock was acquired, function executed, and lock released\n  console.log(\"Process completed with result:\", result);\n}\n```\n\n## Configuration Options\n\n```typescript\nconst mutex = new S3Mutex({\n  // Required: bucket name\n  bucketName: \"my-locks-bucket\",\n  \n  // Either provide an existing S3 client\n  s3Client: s3Client,\n  \n  // OR provide S3 client configuration (s3-mutex will create the client)\n  s3ClientConfig: {\n    region: \"us-east-1\",\n    forcePathStyle: true,       // Useful for MinIO/LocalStack\n    endpoint: \"http://localhost:9000\", // For local development\n    credentials: {\n      accessKeyId: \"your-key\",\n      secretAccessKey: \"your-secret\"\n    }\n  },\n  \n  // Optional configuration with defaults\n  createBucketIfNotExists: false,  // Create bucket if it doesn't exist\n  keyPrefix: \"locks/\",          // Prefix for lock keys in S3\n  maxRetries: 5,                // Max number of acquisition attempts\n  retryDelayMs: 200,            // Base delay between retries (exponential backoff)\n  maxRetryDelayMs: 5000,        // Max delay between retries\n  useJitter: true,              // Add randomness to retry delays\n  lockTimeoutMs: 60000,         // Lock expiration (1 minute)\n  clockSkewToleranceMs: 1000,   // Tolerance for clock differences\n});\n```\n\n## API Reference\n\n### Constructor\n\n```typescript\nnew S3Mutex(options: S3MutexOptions)\n```\n\n### Methods\n\n- **acquireLock(lockName: string, timeoutMs?: number, priority?: number): Promise\u003cboolean\u003e**: Acquire a named lock with optional timeout and priority\n- **releaseLock(lockName: string, force?: boolean): Promise\u003cboolean\u003e**: Release a lock, with optional force parameter\n- **refreshLock(lockName: string): Promise\u003cboolean\u003e**: Refresh a lock's expiration time\n- **isLocked(lockName: string): Promise\u003cboolean\u003e**: Check if a lock is currently held and not expired\n- **isOwnedByUs(lockName: string): Promise\u003cboolean\u003e**: Check if we own a specific lock\n- **deleteLock(lockName: string, force?: boolean): Promise\u003cboolean\u003e**: Completely remove a lock file from S3\n- **withLock\u003cT\u003e(lockName: string, fn: () =\u003e Promise\u003cT\u003e, options?: {timeoutMs?: number, retries?: number}): Promise\u003cT | null\u003e**: Execute a function with an automatic lock\n- **cleanupStaleLocks(options?: {prefix?: string, olderThan?: number, dryRun?: boolean}): Promise\u003c{cleaned: number, total: number, stale: number}\u003e**: Find and clean up expired locks\n\n### Lock Priority and Deadlock Prevention\n\nS3-Mutex includes deadlock prevention through priority-based acquisition. When multiple processes attempt to acquire locks, those with higher priority values will be favored if deadlock conditions are detected.\n\n```typescript\n// Basic priority usage (higher value = higher priority)\nconst acquired = await mutex.acquireLock(\"resource-lock\", undefined, 10);\n\n// Example: High-priority background job\nconst backgroundJobLock = await mutex.acquireLock(\n  \"critical-maintenance\",\n  30000, // 30 second timeout\n  100    // High priority\n);\n\n// Example: Low-priority routine task\nconst routineLock = await mutex.acquireLock(\n  \"routine-cleanup\",\n  10000, // 10 second timeout\n  1      // Low priority\n);\n```\n\n**How Priority Works:**\n- When a deadlock is potentially detected, higher priority requests can force-acquire locks\n- Priority only matters during deadlock resolution, not normal acquisition\n- Use priorities strategically: critical operations get higher values, routine tasks get lower values\n\n## Bucket Management\n\n### Automatic Bucket Creation\n\nS3-Mutex can automatically create the S3 bucket if it doesn't exist. This is particularly useful for development environments or when deploying to new AWS accounts.\n\n```typescript\nconst mutex = new S3Mutex({\n  bucketName: \"my-locks-bucket\",\n  createBucketIfNotExists: true, // Enable automatic bucket creation\n  s3ClientConfig: {\n    region: \"us-east-1\",\n  },\n});\n\n// The bucket will be created automatically on first use\nconst acquired = await mutex.acquireLock(\"my-resource-lock\");\n```\n\n**Important Notes:**\n- Bucket creation requires appropriate IAM permissions (`s3:CreateBucket`)\n- If the bucket already exists, no error is thrown\n- The bucket is created with default settings (no versioning, no lifecycle policies)\n- For production use, consider creating buckets manually with proper configuration\n\n### Manual Bucket Creation\n\nFor production environments, it's recommended to create buckets manually:\n\n```bash\n# Using AWS CLI\naws s3 mb s3://my-locks-bucket --region us-east-1\n\n# Or using CloudFormation/Terraform for infrastructure as code\n```\n\n## Advanced Usage\n\n### Handling Stale Locks\n\n```typescript\n// Find and clean up stale locks\nconst results = await mutex.cleanupStaleLocks({\n  prefix: \"locks/myapp/\",  // Optional prefix to limit cleanup scope\n  olderThan: Date.now() - 3600000,  // Optional custom age (default is lockTimeoutMs)\n  dryRun: true,  // Optional: just report stale locks without deleting\n});\n\nconsole.log(`Found ${results.stale} stale locks out of ${results.total} total locks`);\nconsole.log(`Cleaned up ${results.cleaned} locks`);\n\n// Cleanup all stale locks with default settings\nconst quickCleanup = await mutex.cleanupStaleLocks();\n\n// Cleanup locks older than 2 hours\nconst oldLockCleanup = await mutex.cleanupStaleLocks({\n  olderThan: Date.now() - (2 * 60 * 60 * 1000)\n});\n```\n\n### Force-releasing a Lock\n\n```typescript\n// Force release a lock (use with caution)\nawait mutex.releaseLock(\"resource-lock\", true);\n\n// Force delete a lock file completely\nawait mutex.deleteLock(\"resource-lock\", true);\n\n// Check lock ownership before operations\nif (await mutex.isOwnedByUs(\"resource-lock\")) {\n  await mutex.refreshLock(\"resource-lock\");\n  // do work\n  await mutex.releaseLock(\"resource-lock\");\n}\n```\n\n## Best Practices\n\n1. **Set appropriate timeouts**: Configure lock timeouts that match your workload duration\n2. **Handle failure gracefully**: Always check if lock acquisition was successful\n3. **Use the withLock helper**: Ensures locks are always released, even if errors occur\n4. **Implement proper error handling**: Be prepared for S3 service errors and throttling\n5. **Run periodic cleanup**: Use the cleanupStaleLocks method to maintain your lock storage\n6. **Consider performance implications**: S3 operations have higher latency than in-memory solutions\n7. **Test thoroughly under load**: Verify lock reliability under your specific workload conditions\n8. **Have a fallback strategy**: Plan for occasional lock failures in production environments\n9. **Monitor lock contention**: High contention may indicate need for architectural changes\n10. **Use appropriate priorities**: Reserve high priorities for critical operations, use low priorities for routine tasks\n11. **Handle null returns from withLock**: The `withLock` method returns `null` if lock acquisition fails\n12. **Consider clock skew**: Set `clockSkewToleranceMs` appropriately for your distributed environment\n\n## Development and Testing\n\n### Prerequisites\n\n- Node.js 18+\n- Docker (for running S3-compatible storage locally)\n\n### Local Development Setup\n\n1. **Start MinIO (S3-compatible storage) for testing:**\n\n```bash\n# Using Docker Compose (if available in the project)\ndocker-compose up -d\n\n# Or run MinIO directly\ndocker run -d \\\n  --name minio \\\n  -p 9000:9000 \\\n  -p 9001:9001 \\\n  -e MINIO_ROOT_USER=root \\\n  -e MINIO_ROOT_PASSWORD=password \\\n  quay.io/minio/minio server /data --console-address \":9001\"\n```\n\n2. **Install dependencies:**\n\n```bash\npnpm install\n```\n\n3. **Run tests:**\n\n```bash\n# Run tests (requires MinIO running)\npnpm test\n\n# Run tests with coverage\npnpm test:ci\n\n# Build the project\npnpm build\n\n# Lint code\npnpm lint\n```\n\n### Testing with Different S3 Implementations\n\nThe library is tested with:\n\n- **MinIO** (recommended for local development)\n- **LocalStack** (AWS services emulation)\n- **AWS S3** (production)\n\n#### Environment Variables for Testing\n\n```bash\n# S3 endpoint (default: http://localhost:9000)\nS3_ENDPOINT=http://localhost:9000\n\n# S3 region (default: us-east-1)\nS3_REGION=us-east-1\n\n# S3 credentials (defaults: root/password for MinIO)\nS3_ACCESS_KEY=root\nS3_SECRET_KEY=password\n```\n\n### Example Test Configuration\n\n```typescript\nimport { S3Client } from \"@aws-sdk/client-s3\";\nimport { S3Mutex } from \"s3-mutex\";\n\n// Test configuration for MinIO\nconst testMutex = new S3Mutex({\n  bucketName: \"test-locks-bucket\",\n  createBucketIfNotExists: true, // Automatically create test bucket\n  s3ClientConfig: {\n    forcePathStyle: true,\n    endpoint: \"http://localhost:9000\",\n    region: \"us-east-1\",\n    credentials: {\n      accessKeyId: \"root\",\n      secretAccessKey: \"password\",\n    },\n  },\n  // Faster settings for testing\n  maxRetries: 3,\n  retryDelayMs: 100,\n  lockTimeoutMs: 1000,\n});\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimmikeladze%2Fs3-mutex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimmikeladze%2Fs3-mutex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimmikeladze%2Fs3-mutex/lists"}