https://github.com/nikiforovall/channels-composition-and-otel
Job Offloading Pattern using an a pipeline built with System.Threading.Channels and OpenTelemetry
https://github.com/nikiforovall/channels-composition-and-otel
Last synced: about 1 month ago
JSON representation
Job Offloading Pattern using an a pipeline built with System.Threading.Channels and OpenTelemetry
- Host: GitHub
- URL: https://github.com/nikiforovall/channels-composition-and-otel
- Owner: NikiforovAll
- Created: 2024-04-19T14:24:24.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-04-22T07:21:01.000Z (about 1 year ago)
- Last Synced: 2025-03-16T18:42:29.070Z (about 1 month ago)
- Language: C#
- Homepage: https://nikiforovall.github.io/dotnet/async/2024/04/21/job-offloading-pattern.html
- Size: 419 KB
- Stars: 9
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# System.Threading.Channels Composition and OpenTelemetry. Job Offloading Pattern
The project demonstrates how to build an in-memory pipeline that allows to offload long-running tasks. As a result, we have an improved user experience because the result is returned as soon as a task is scheduled.
```csharp
app.MapPost(
"/start",
async (
[FromBody] Payload payload,
IBackgroundProcessor writer,
CancellationToken cancellationToken
) =>
{
await writer.QueueBackgroundWorkItemAsync(payload, cancellationToken);
return TypedResults.Ok(new { Success = true });
}
);
```
```csharp
public class ProcessorChannel : IBackgroundProcessor
{
public ProcessorChannel()
{
var options = processorOptions.Value;
_pipeline = Channel.CreateBounded(10);
Writer = _pipeline;
Reader = _pipeline
.Pipe(InitPipeline, capacity: 10)
.PipeAsync(
maxConcurrency: 5,
Step1,
capacity: 10
)
.PipeAsync(
maxConcurrency: 2,
Step2,
capacity: 10
)
.Pipe(FinishPipeline);
}
}
```
## Demo
```bash
dotnet run --project ./src/AppHost
```
```bash
./scripts/bombardier.sh 100
```
`ProcessorChannelSettings.UseUnifiedSpanForAllPipelines=false`
`ProcessorChannelSettings.UseUnifiedSpanForAllPipelines=true`
```bash
dotnet-counters monitor -n API --counters MyService.Pipelines
```
## Reference
*
*
*
*
*
*
*
*
*