{"id":28410565,"url":"https://github.com/lennarttenwolde/dequeueable","last_synced_at":"2026-04-16T14:07:53.777Z","repository":{"id":37495762,"uuid":"495538295","full_name":"lennarttenwolde/Dequeueable","owner":"lennarttenwolde","description":"A project that handles dequeuing queue messages from known Cloud Providers ","archived":false,"fork":false,"pushed_at":"2024-09-12T11:53:05.000Z","size":341,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-07T07:40:35.431Z","etag":null,"topics":["azure","azure-queue","azure-storage","containers","csharp","dequeue","dotnet","message-queue","queue","queues"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lennarttenwolde.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-05-23T19:00:39.000Z","updated_at":"2024-10-05T10:40:54.000Z","dependencies_parsed_at":"2024-09-12T21:17:28.412Z","dependency_job_id":"5d5c47d9-672e-4fc4-9106-fd9a8a234f4f","html_url":"https://github.com/lennarttenwolde/Dequeueable","commit_stats":{"total_commits":57,"total_committers":3,"mean_commits":19.0,"dds":0.4736842105263158,"last_synced_commit":"5c1c8ae3f77788253ff0cbeb183a7ebe45fb62b0"},"previous_names":["lennarttenwolde/dequeueable","lenndewolten/dequeueable"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/lennarttenwolde/Dequeueable","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lennarttenwolde%2FDequeueable","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lennarttenwolde%2FDequeueable/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lennarttenwolde%2FDequeueable/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lennarttenwolde%2FDequeueable/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lennarttenwolde","download_url":"https://codeload.github.com/lennarttenwolde/Dequeueable/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lennarttenwolde%2FDequeueable/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261197468,"owners_count":23123722,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","azure-queue","azure-storage","containers","csharp","dequeue","dotnet","message-queue","queue","queues"],"created_at":"2025-06-02T11:36:06.915Z","updated_at":"2026-04-16T14:07:53.766Z","avatar_url":"https://github.com/lennarttenwolde.png","language":"C#","readme":"# Dequeueable\n\nThis project is an **opinionated**, cloud-native ephemeral job runner for Azure Queue Storage.\n\nIt is designed to be triggered by external queue scalers (e.g., KEDA), process a **single message**, and immediately shut down upon completion. If no message is found, the host shuts down without executing.\n\n- Built as a Console App\n- Compatible with optimized alpine/dotnet images\n- Works with KEDA or any other external queue scaler\n\n## Getting started\n\nScaffold a new project, you can either use a console or web app.\n\n1. Add a class that implements the `IQueueJob`.\n2. Add `.AddDequeueable\u003cYourJob\u003e` in the DI container.\n3. Call `.RunJobAsync` on the host builder to run as a job.\n\n```csharp\nawait Host.CreateDefaultBuilder(args)\n.ConfigureServices(services =\u003e\n{\n    services.AddDequeueable\u003cTestJob\u003e(options =\u003e\n    {\n       // ...\n    });\n}).RunJobAsync();\n```\n\n### Configurations\n\nYou can configure the host via the `appsettings.json` or via the `IOptions` pattern during registration.\n\n**Appsettings**\n\nUse the `Dequeueable` section to configure the settings:\n\n```json\n\"Dequeueable\": {\n    \"ConnectionString\": \"UseDevelopmentStorage=true\",\n    \"QueueName\": \"queue-name\"\n  }\n```\n\n**Options**\n\n```csharp\nawait Host.CreateDefaultBuilder(args)\n    .ConfigureServices((context, services) =\u003e\n    {\n        services.AddDequeueable\u003cTestJob\u003e(options =\u003e\n        {\n            options.AuthenticationScheme = new DefaultAzureCredential();\n             options.VisibilityTimeoutInSeconds = 600;\n            options.QueueName = \"testqueue\";\n        });\n    })\n    .RunJobAsync();\n```\n\n### Settings\n\nThe library uses the `IOptions` pattern to inject the configured app settings. These settings will be validated on startup.\n\n#### Host options\n\nThese options can be set for the job project:\n\n| Setting                    | Description                                                                                                                                            | Default                                                                    | Required                           |\n| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------- | ---------------------------------- |\n| QueueName                  | The queue used to retrieve the messages.                                                                                                               |                                                                            | Yes                                |\n| ConnectionString           | The connection string used to authenticate to the queue.                                                                                               |                                                                            | Yes, when not using Azure Identity |\n| PoisonQueueSuffix          | Suffix that will be used after the QueueName, eg queuename-suffix.                                                                                     | poison                                                                     | No                                 |\n| AccountName                | The storage account name, used for identity flow.                                                                                                      |                                                                            | Only when using Identity           |\n| QueueUriFormat             | The uri format to the queue storage. Used for identity flow. Use ` {accountName}` and `{queueName}` for variable substitution.                         | https://{accountName}.queue.core.windows.net/{queueName}                   | No                                 |\n| AuthenticationScheme       | Token credential used to authenticate via AD, Any token credential provider can be used that inherits the abstract class `Azure.Core.TokenCredential`. |                                                                            | Yes, if you want to use Identity   |\n| MaxDequeueCount            | Max dequeue count before moving to the poison queue.                                                                                                   | 5                                                                          | No                                 |\n| VisibilityTimeoutInSeconds | The timeout after the queue message is visible again for other services.                                                                               | 300                                                                        | No                                 |\n| QueueClientOptions         | Provides the client configuration options for connecting to Azure Queue Storage.                                                                       | `new QueueClientOptions { MessageEncoding = QueueMessageEncoding.Base64 }` | No                                 |\n\n## Authentication\n\n### SAS\n\nYou can authenticate to the storage account \u0026 queue by setting the ConnectionString:\n\n```json\n\"Dequeueable\": {\n    \"ConnectionString\": \"UseDevelopmentStorage=true\",\n    ...\n  }\n```\n\n```csharp\n    services.AddDequeueable\u003cTestJob\u003e(options =\u003e\n    {\n        // ...\n        options.ConnectionString = \"UseDevelopmentStorage=true\";\n    });\n```\n\n### Identity\n\nAuthenticating via Azure Identity is also possible and the recommended option. Make sure that the identity used have the following roles on the storage account\n\n- 'Storage Queue Data Contributor'\n- 'Storage Blob Data Contributor' - Only when making use of the distributed lock.\n\nSet the `AuthenticationScheme` and the `AccountName` options to authenticate via azure AD:\n\n```csharp\n    services.AddDequeueable\u003cTestJob\u003e(options =\u003e\n    {\n        options.AuthenticationScheme = new DefaultAzureCredential();\n        options.AccountName = \"thestorageaccountName\";\n    });\n```\n\nAny token credential provider can be used that inherits the abstract class `Azure.Core.TokenCredential`\n\nThe `QueueUriFormat` options is used to format the correct URI to the queue. When making use of the distributed lock, the `BlobUriFormat` is used to format the correct URI to the blob lease.\n\n### Custom QueueProvider\n\nThere are plenty ways to construct the QueueClient, and not all are by default supported. You can override the default implementations to retrieve the queue client by implementing the `IQueueClientProvider`. You still should register your custom provider in your DI container, specific registration order is not needed:\n\n```csharp\ninternal class MyCustomQueueProvider : IQueueClientProvider\n    {\n        public QueueClient GetQueue()\n        {\n            return new QueueClient(new Uri(\"https://myaccount.chinacloudapi.cn/myqueue\"), new QueueClientOptions { MessageEncoding = QueueMessageEncoding.Base64 });\n        }\n\n        public QueueClient GetPoisonQueue()\n        {\n            return new QueueClient(new Uri(\"https://myaccount.chinacloudapi.cn/mypoisonqueue\"), new QueueClientOptions { MessageEncoding = QueueMessageEncoding.Base64 });\n        }\n    }\n```\n\n## Distributed Lock\n\nA distributed lock can be applied to the job to ensure that only a single instance of the job is executed at any given time. It uses the blob lease and therefore **distributed** lock is guaranteed. The blob is leased for the duration configured by LeaseDurationInSeconds (default: 60 seconds). The lease will be released if no longer required. It will be automatically renewed if executing the message(s) takes longer.\n\nNOTE: The blob files will not be automatically deleted. If needed, consider specifying data lifecycle rules for the blob container: https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview\n\n\u003e If making use of the **Identity Flow**, the lease requires the **Storage Blob Data Contributor** role because the library writes and manages the lease blob for distributed locking.\n\nTo run the host with a distributed lock, call the `.WithDistributedLock()` in the DI container:\n\n```csharp\nservices.AddDequeueable\u003cTestJob\u003e()\n.WithDistributedLock(opt =\u003e\n    {\n        opt.Scope = \"id\";\n    });\n```\n\nOnly messages containing a JSON format is supported. The scope should **always** be a property in the message body that exists.\n\nGiven a queue message with the following body:\n\n```json\n{\n  \"Id\": \"d89c209a-6b81-4266-a768-8cde6f613753\"\n  // ...\n}\n```\n\nWhen the scope is set to `\"Id\"` on the job. Only a single message containing Id \"d89c209a-6b81-4266-a768-8cde6f613753\" will be executed at an given time. This is case sensitive, the scope string must match the JSON key exactly!\n\nNested properties are also supported. Given a queue message with the following body:\n\n```json\n{\n  \"My\": {\n    \"Nested\": {\n      \"Property\": 500\n    }\n  }\n  // ...\n}\n```\n\nWhen the scope is set to `\"My:Nested:Property\"` on the job. Only a single message containing `500` will be executed at an given time.\n\n### Lock Options\n\nYou can specify the following lock options via the `.WithDistributedLock(opt =\u003e {})` or via the `appsettings.json` using the Dequeueable:DistributedLock section:\n\n```json\n{\n  \"Dequeueable\": {\n    \"DistributedLock\": {\n      \"Scope\": \"id\"\n    }\n  }\n}\n```\n\n| Setting                  | Description                                                                                                                                     | Default                                                                  | Required |\n| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | -------- |\n| LeaseDurationInSeconds   | The duration of the Blob lease, in seconds.                                                                                                     | 60                                                                       | No       |\n| MinimumPollingIntervalInSeconds | The minimum polling interval to check if a new lease can be acquired.                                                                           | 10                                                                       | No       |\n| MaximumPollingIntervalInSeconds | The maximum polling interval to check if a new lease can be acquired.                                                                           | 120                                                                      | No       |\n| MaxRetries               | The max retries to acquire a lease.                                                                                                             | 3                                                                        | No       |\n| ContainerName            | The container name for the lock files.                                                                                                          | webjobshost                                                              | No       |\n| BlobUriFormat            | The uri format to the blob storage. Used for identity flow. Use ` {accountName}`, `{containerName}` and `{blobName}` for variable substitution. | \"https://{accountName}.blob.core.windows.net/{containerName}/{blobName}\" | No       |\n\n### Custom BlobClientProvider\n\nThere are plenty ways to construct the BlobClient, and not all are by default supported. You can override the default implementations to retrieve the blob client for the lease by implementing the `IBlobClientProvider`. You still should register your custom provider in your DI container, specific registration order is not needed:\n\n```csharp\ninternal class MyCustomBlobClientProvider : IBlobClientProvider\n    {\n        public BlobClient GetClient(string blobName)\n        {\n            return new BlobClient(new Uri($\"https://myaccount.chinacloudapi.cn/mycontainer/{blobName}\"),\n                new BlobClientOptions { GeoRedundantSecondaryUri = new Uri($\"https://mysecaccount.chinacloudapi.cn/mycontainer/{blobName}\") });\n        }\n    }\n```\n\n## Timeouts\n\n### Visibility Timeout Queue Message\n\nThe visibility timeout of the queue messages is automatically updated. It will be updated when the half `VisibilityTimeout` option is reached. Choose this setting wisely to prevent talkative hosts. When renewing the timeout fails, the host cannot guarantee if the message is executed only once. Therefore the CancelationToken is set to Cancelled. It is up to you how to handle this scenario!\n\n### Lease timeout\n\nThe lease timeout of the blob lease is automatically updated. It will be updated when the half lease is reached. When renewing the timeout fails, the host cannot guarantee the lock. Therefore the CancelationToken is set to Cancelled. It is up to you how to handle this scenario!\n\n## Sample\n\n- [Job Console app](https://github.com/lenndewolten/Dequeueable/blob/main/samples/Dequeueable.SampleJob/README.md)\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flennarttenwolde%2Fdequeueable","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flennarttenwolde%2Fdequeueable","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flennarttenwolde%2Fdequeueable/lists"}