{"id":21730096,"url":"https://github.com/nventive/backgroundprocessing","last_synced_at":"2025-08-23T19:09:36.280Z","repository":{"id":98348386,"uuid":"198631272","full_name":"nventive/BackgroundProcessing","owner":"nventive","description":".NET Core server-side background processing.","archived":false,"fork":false,"pushed_at":"2019-08-05T18:21:51.000Z","size":125,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-12-30T14:41:22.180Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nventive.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-24T12:27:48.000Z","updated_at":"2022-02-02T05:01:47.000Z","dependencies_parsed_at":"2023-09-06T00:46:49.866Z","dependency_job_id":null,"html_url":"https://github.com/nventive/BackgroundProcessing","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nventive%2FBackgroundProcessing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nventive%2FBackgroundProcessing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nventive%2FBackgroundProcessing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nventive%2FBackgroundProcessing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nventive","download_url":"https://codeload.github.com/nventive/BackgroundProcessing/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235628177,"owners_count":19020540,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-26T04:11:53.365Z","updated_at":"2025-01-25T20:08:41.052Z","avatar_url":"https://github.com/nventive.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BackgroundProcessing\n\n.NET Core server-side background processing.\n\nAllows background processing of jobs/tasks in ASP.NET Core and Azure Functions.\nSupport for in-memory queue and Azure Storage Queue.\n\nIntegrated with .NET Core Dependency Injection.\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)\n[![Build Status](https://dev.azure.com/nventive-public/nventive/_apis/build/status/nventive.BackgroundProcessing?branchName=master)](https://dev.azure.com/nventive-public/nventive/_build/latest?definitionId=8\u0026branchName=master)\n![Nuget](https://img.shields.io/nuget/v/BackgroundProcessing.Core.svg)\n\n## Getting Started\n\n### Vocabulary\n\nHere are the terms used throughout the components:\n\n- `IBackgroundCommand`: The POCO command that allows transfer of execution; this is the unit that gets serialized; contains the parameters of the execution.\n- `IBackgroundCommandHandler`: Handles the execution of `IBackgrounCommand`. This is where the business logic is implemented.\n- `IBackgroundDispatcher`: The dispatcher is responsible for queueing commands for later execution.\n- `IBackgroundProcessor`: The processor is responsible for selecting and invoking appropriate `IBackgroundCommandHandler`.\n\n### Basic usage\n\nInstall the package:\n\n```\nInstall-Package BackgroundProcessing.Core\n```\n\nDefine the commands \u0026 handlers\n\n```csharp\nusing BackgroundProcessing.Core;\n\n///\u003csummary\u003e\n/// Defines a command\n///\u003c/summary\u003e\npublic class MyCommand : BackgroundCommand\n{\n    ///\u003csummary\u003e\n    /// Gets or sets custom parameters.\n    ///\u003c/summary\u003e\n    public string MyParameter { get; set; }\n}\n\n///\u003csummary\u003e\n/// Defines the corresponding command handler\n///\u003c/summary\u003e\npublic class MyCommandHandler : IBackgroundCommandHandler\u003cMyCommand\u003e\n{\n    public MyCommandHandler(IDependencyOne dependency)\n    {\n        // Handle dependencies\n    }\n\n    public async Task HandleAsync(MyCommand command, CancellationToken cancellationToken = default)\n    {\n        // Handles the command by executing business logic\n    }\n}\n\n```\n\nRegisters the command handler\n\n```csharp\nusing Microsoft.Extensions.DependencyInjection;\n\n// in your Startup.cs, or wherever services are registered.\n// This will register all the IBackgroundCommandHandler in the assembly.\nservices.AddBackgroundCommandHandlersFromAssemblyContaining\u003cMyCommandHandler\u003e();\n```\n\nWhen you want to dispatch a command for execution, use the registered `IBackgroundDispatcher`:\n\n```csharp\nusing BackgroundProcessing.Core;\n\npublic class MyService\n{\n    private readonly IBackgroundDispatcher _dispatcher;\n\n    public MyService(IBackgroundDispatcher dispatcher)\n    {\n        _dispatcher = dispatcher ?? new ArgumentNullException(nameof(dispatcher));\n    }\n\n    public async Task DispatchCommand()\n    {\n        var command = new MyCommand { MyParameter = \"foo\" };\n        await _dispatcher.DispatchAsync(command);\n    }\n}\n\n```\n\n### Getting Started with .NET Core Host in-memory\n\nThis will registered a `IBackgroundDispatcher`, a `IBackgroundProcessor` and a `IHostingService` that communicates via an in-memory queue.\nThis is very easy to get started with, but should not be used in any production scenario, as you will loose pending commands in case of process stop/restart.\n\n```csharp\nusing Microsoft.Extensions.DependencyInjection;\n\n// in your Startup.cs, or wherever services are registered.\nservices.AddHostingServiceConcurrentQueueBackgroundProcessing();\n```\n\n### Getting Started with .NET Core `IHostingService` and an Azure Storage Queue\n\nThis is the easiest deployment that can be suitable for production; it only requires a .NET Core host (such as ASP.NET Core) and an [Azure Storage Queue](https://azure.microsoft.com/en-in/services/storage/queues/).\n\nFirst, ensure that you created an Azure Storage Account.\n\nThen, install the package:\n\n```\nInstall-Package BackgroundProcessing.Azure.Storage.Queue\n```\n\nAnd register the services:\n\n```csharp\nusing Microsoft.Extensions.DependencyInjection;\n\n// in your Startup.cs, or wherever services are registered.\n\n// Convenient method to create cloud queues.\nvar cloudQueueProvider = CloudQueueProvider.FromConnectionStringName(\"ConnectionStringName\", \"queue name\");\nservices.AddAzureStorageQueueBackgroundDispatcher(cloudQueueProvider);\nservices.AddAzureStorageQueueBackgroundProcessing(cloudQueueProvider);\n```\n\n### Getting started using Azure Functions processing and Azure Storage Queue\n\nThis method allows dispatching commands to an Azure Storage Queue and the processing\ninside an Azure Functions. This is probably the best production option.\n\nFirst, ensure that you created an Azure Storage Account.\n\nThen, install the package:\n\n```\nInstall-Package BackgroundProcessing.Azure.Storage.Queue\n```\n\nThen, in the process responsible for dispatching commands, register the dispatcher:\n\n```csharp\nusing Microsoft.Extensions.DependencyInjection;\n\n// in your Startup.cs, or wherever services are registered.\nservices.AddAzureStorageQueueBackgroundDispatcher();\n```\n\nLast, create an Azure Functions project and:\n\n1. Add the Azure Functions Queue Bindings\n```\nInstall-Package Microsoft.Azure.WebJobs.Extensions.Storage\n```\n\n2. Register the services by creating a `FunctionsStartup` class\n```csharp\nusing Microsoft.Extensions.DependencyInjection;\n\n/// \u003csummary\u003e\n/// Create a FunctionStartup to configure dependency injection\n/// \u003c/summary\u003e\npublic class Startup : FunctionsStartup\n{\n    /// \u003cinheritdoc /\u003e\n    public override void Configure(IFunctionsHostBuilder builder)\n    {\n        // Register your services...\n        builder\n          .Services\n          // Register the command handlers if it's not already done\n          .AddBackgroundCommandHandlersFromAssemblyContaining\u003c\u003e();\n          .AddAzureFunctionsQueueStorageProcessing();\n    }\n}\n```\n\n3. Create a function with a queue trigger\n\n```csharp\nusing System;\nusing System.Threading.Tasks;\nusing BackgroundProcessing.Azure.Storage.Queue;\nusing Microsoft.Azure.WebJobs;\nusing Microsoft.WindowsAzure.Storage.Queue;\n\npublic class BackgroundProcessingFunction\n{\n    private readonly AzureFunctionsQueueStorageHandler _handler;\n\n    public BackgroundProcessingFunction(\n        AzureFunctionsQueueStorageHandler handler)\n    {\n        _handler = handler ?? throw new ArgumentNullException(nameof(handler));\n    }\n\n    [FunctionName(\"BackgroundProcessingFunction\")]\n    public async Task Run(\n      [QueueTrigger(\"queue name\", Connection = \"ConnectionString\")] CloudQueueMessage message)\n    {\n        await _handler.HandleAsync(message);\n    }\n}\n```\n\n## Features\n\n### Handle processor exceptions\n\nIt is possible to get notified and received `Exceptions` during the execution of handlers.\n\nTo do so, you can register a `IBackgroundProcessor` decorator.\n\n```csharp\nusing BackgroundProcessing.Core;\n\npublic class ErrorHandlerBackgroundProcessorDecorator: IBackgroundProcessor\n{\n    private readonly IBackgroundProcessor _wrappedProcessor;\n\n    public ErrorHandlerBackgroundProcessorDecorator(IBackgroundProcessor wrappedProcessor)\n    {\n        _wrappedProcessor = _wrappedProcessor ?? throw new ArgumentNullException(nameof(wrappedProcessor));\n    }\n\n    public async Task ProcessAsync(IBackgroundCommand command, CancellationToken cancellationToken = default)\n    {\n        try\n        {\n            await _wrappedProcessor.ProcessAsync(command, cancellationToken);\n        } catch (Exception ex)\n        {\n            // Handle the exception.\n        }\n    }\n}\n\n// Then during the registration of the processor/services:\nusing Microsoft.Extensions.DependencyInjection;\n\nservices\n    .Add...Processing()\n    .DecorateProcessor\u003cErrorHandlerBackgroundProcessorDecorator\u003e();\n```\n\nThis technique can be extended to handle any cross-cutting concern when dispatching or processing commands.\n\n### Integration with Application Insights\n\nAdd the package\n```\nInstall-Package BackgroundProcessing.Azure.ApplicationInsights\n```\n\nThen register the decorators:\n\n```csharp\n// Then during the registration of the processor/services:\nusing Microsoft.Extensions.DependencyInjection;\n\nservices\n    .Add...Processing() // or Add...Dispatcher()\n    .AddApplicationInsightsDecorators();\n```\n\n### Get and store execution events\n\nIt is possible to store the history of execution events related to commands.\nThe component provides a `IBackgroundCommandEventRepository` interface to allow\nstorage and retrieval of events related to commands.\n\n### Store events in the ASP.NET Core cache.\n\nInstall the corresponding package:\n\n```\nInstall-Package BackgroundProcessing.Caching\n```\n\nThen configure the repository:\n\n```csharp\n// Then during the registration of the processor/services:\nusing Microsoft.Extensions.DependencyInjection;\n\n// To use the IMemoryCache\nservices\n    .AddMemoryCache()\n    .Add...Processing() // or Add...Dispatcher()\n    .AddBackgroundCommandEventsRepositoryDecorators()\n    .AddMemoryCacheEventRepository();\n\n// To use the IDistributedCache\nservices\n    .AddDistributedMemoryCache() // Or any other IDistributedCache implementation\n    .Add...Processing() // or Add...Dispatcher()\n    .AddBackgroundCommandEventsRepositoryDecorators()\n    .AddDistributedCacheEventRepository();\n```\n\nThe expiration of events in the cache is configurable (defaults to 10 minutes).\n\n### Store events in an Azure Table Storage\n\nInstall the corresponding package:\n\n```\nInstall-Package BackgroundProcessing.Azure.Storage.Table\n```\n\nThen configure the repository:\n\n```csharp\n// Then during the registration of the processor/services:\nusing Microsoft.Extensions.DependencyInjection;\n\n// This takes care of instantiating the CloudTable instance and\n// creating the table if it does not exists.\nvar cloudTableProvider = CloudTableProvider.FromConnectionStringName(\"ConnectionStringName\", \"TableName\");\n\nservices\n    .Add...Processing() // or Add...Dispatcher()\n    .AddBackgroundCommandEventsRepositoryDecorators()\n    .AddCloudTableEventRepository(cloudTableProvider);\n```\n\n### Query events\n\nTo query events, just require and use the `IBackgroundCommandEventRepository` interface.\n\nExample of such usage in a ASP.NET Core Controller with a polling REST API:\n\n```csharp\nusing System.Threading.Tasks;\nusing BackgroundProcessing.Core;\nusing BackgroundProcessing.Core.Events;\nusing Microsoft.AspNetCore.Http;\nusing Microsoft.AspNetCore.Mvc;\n\nnamespace BackgroundProcessing.WebSample\n{\n    // See http://restalk-patterns.org/long-running-operation-polling.html\n    [ApiController]\n    public class SampleController : ControllerBase\n    {\n        [HttpPost(\"api/v1/commands\")]\n        [ProducesResponseType(StatusCodes.Status202Accepted)]\n        [ProducesDefaultResponseType]\n        public async Task\u003cIActionResult\u003e DispatchCommand(\n            [FromServices] IBackgroundDispatcher dispatcher)\n        {\n            var command = new SampleCommand();\n            await dispatcher.DispatchAsync(command);\n            return AcceptedAtRoute(nameof(GetCommandStatus), new { id = command.Id });\n        }\n\n        [HttpGet(\"api/v1/commands/{id}\", Name = nameof(GetCommandStatus))]\n        [ProducesResponseType(StatusCodes.Status204NoContent)]\n        [ProducesResponseType(StatusCodes.Status303SeeOther)]\n        [ProducesResponseType(StatusCodes.Status404NotFound)]\n        [ProducesDefaultResponseType]\n        public async Task\u003cIActionResult\u003e GetCommandStatus(\n            [FromRoute] string id,\n            [FromServices] IBackgroundCommandEventRepository repository)\n        {\n            var latestEvent = await repository.GetLatestEventForCommandId(id);\n            if (latestEvent is null)\n            {\n                return NotFound();\n            }\n\n            switch (latestEvent.Status)\n            {\n                case BackgroundCommandEventStatus.Dispatching:\n                case BackgroundCommandEventStatus.Processing:\n                    // Indicate the desired polling frequency, in seconds.\n                    Response.Headers.Add(\"Retry-After\", \"10\");\n                    return NoContent();\n                case BackgroundCommandEventStatus.Processed:\n                    // Once executed, redirect to the output.\n                    return this.SeeOther(Url.RouteUrl(nameof(GetCommandOutput), new { id }));\n                default:\n                    return StatusCode(StatusCodes.Status500InternalServerError);\n            }\n        }\n\n        [HttpGet(\"api/v1/commands/{id}/output\", Name = nameof(GetCommandOutput))]\n        [ProducesResponseType(StatusCodes.Status200OK)]\n        [ProducesResponseType(StatusCodes.Status404NotFound)]\n        [ProducesDefaultResponseType]\n        public async Task\u003cIActionResult\u003e GetCommandOutput([FromRoute] string id)\n        {\n            // This part is up to you / your business logic.\n            if (id is null)\n            {\n                return NotFound();\n            }\n\n            return Ok();\n        }\n    }\n}\n\n```\n\n### Further customizations\n\n- Most of the dispatcher and services can be further customize by looking at the options; e.g. when applicable, the degree of execution parallelism, the queue polling frequency and the message batch size can be adjusted for your scenario\n- Override `IBackgroundProcessor` to customize the way `IBackgroundCommandHandler` are resolved and executed, regardless of the queueing scenario\n- Override `IBackgroundCommandSerializer` to adjust `IBackgroundCommand` serialization (defaults to JSON.NET)\n- Commands can implement `IBackgroundCommand` (instead of deriving from `BackgroundCommand`) if needed\n\n## Changelog\n\nPlease consult the [CHANGELOG](CHANGELOG.md) for more information about version\nhistory.\n\n## License\n\nThis project is licensed under the Apache 2.0 license - see the\n[LICENSE](LICENSE) file for details.\n\n## Contributing\n\nPlease read [CONTRIBUTING.md](CONTRIBUTING.md) for details on the process for\ncontributing to this project.\n\nBe mindful of our [Code of Conduct](CODE_OF_CONDUCT.md).\n\nTo run the unit tests locally, you must use the [Azure Storage Emulator](https://docs.microsoft.com/en-us/azure/storage/common/storage-use-emulator).\n\n## Acknowledgments\n\n- [Dataflow](https://docs.microsoft.com/en-us/dotnet/standard/parallel-programming/dataflow-task-parallel-library) for hosting services execution management\n- [Scrutor](https://github.com/khellang/Scrutor) for decorators service registration\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnventive%2Fbackgroundprocessing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnventive%2Fbackgroundprocessing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnventive%2Fbackgroundprocessing/lists"}