Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/aabs/ActorSrcGen
ActorSrcGen is a C# Source Generator allowing the conversion of simple C# classes into dataflow compatible pipelines supporting the actor model.
https://github.com/aabs/ActorSrcGen
actor-model asynchronous-programming csharp-sourcegenerator high-performance-computing parallel-programming source-generator sourcegenerator tpl-dataflow
Last synced: 2 months ago
JSON representation
ActorSrcGen is a C# Source Generator allowing the conversion of simple C# classes into dataflow compatible pipelines supporting the actor model.
- Host: GitHub
- URL: https://github.com/aabs/ActorSrcGen
- Owner: aabs
- License: mit
- Created: 2023-10-25T22:33:07.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2024-08-29T01:51:41.000Z (5 months ago)
- Last Synced: 2024-08-29T08:33:56.049Z (5 months ago)
- Topics: actor-model, asynchronous-programming, csharp-sourcegenerator, high-performance-computing, parallel-programming, source-generator, sourcegenerator, tpl-dataflow
- Language: C#
- Homepage:
- Size: 189 KB
- Stars: 6
- Watchers: 1
- Forks: 0
- Open Issues: 4
-
Metadata Files:
- Readme: ReadMe.md
- License: LICENSE.txt
Awesome Lists containing this project
- RSCG_Examples - https://github.com/aabs/ActorSrcGen
README
# Welcome To ActorSrcGen
ActorSrcGen is a C# Source Generator that converts simple C# classes into TPL Dataflow-compatible pipelines. It simplifies working with TPL Dataflow by generating boilerplate code to handle errors without interrupting the pipeline, ideal for long-lived processes with ingesters that continually pump messages into the pipeline.
## Getting Started
1. **Install the package**:
```shell
dotnet add package ActorSrcGen
```
2. **Declare the pipeline class**:
```csharp
[Actor]
public partial class MyPipeline
{
}
```
The class must be `partial` to allow the source generator to add boilerplate code.
If you are using Visual Studio, you can see the generated part of the code under the
ActorSrcGen analyzer:
3. **Create ingester functions**:
```csharp
[Ingest(1)]
[NextStep(nameof(DoSomethingWithRequest))]
public async Task ReceivePollRequest(CancellationToken cancellationToken)
{
return await GetTheNextRequest();
}
```
Ingesters define a `Priority` and are visited in priority order. If no messages are available, the pipeline sleeps for a second before retrying.
4. **Implement pipeline steps**:
```csharp
[FirstStep("decode incoming poll request")]
[NextStep(nameof(ActOnTheRequest))]
public PollRequest DecodeRequest(string json)
{
Console.WriteLine(nameof(DecodeRequest));
var pollRequest = JsonSerializer.Deserialize(json);
return pollRequest;
}
```
The first step controls the pipeline's interface. Implement additional steps as needed, ensuring input and output types match.
1. Now **implement other steps** are needed in the pipeline. The outputs and input types
of successive steps need to match.
```csharp
[Step]
[NextStep(nameof(DeliverResults))]
public PollResults ActOnTheRequest(PollRequest req)
{
Console.WriteLine(nameof(ActOnTheRequest));
var result = SomeApiClient.GetTheResults(req.Id);
return result;
}
```
5. **Define the last step**:
```csharp
[LastStep]
public bool DeliverResults(PollResults res)
{
return myQueue.TryPush(res);
}
```
6. **Generated code example**:
```csharp
using System.Threading.Tasks.Dataflow;
using Gridsum.DataflowEx;
public partial class MyActor : Dataflow, IActor< string >
{
public MyActor(DataflowOptions dataflowOptions = null) : base(DataflowOptions.Default)
{
_DeliverResults = new TransformBlock( (PollResults x) => {
try
{
return DeliverResults(x);
}
catch
{
return default;
}
},
new ExecutionDataflowBlockOptions() {
BoundedCapacity = 1,
MaxDegreeOfParallelism = 1
});
RegisterChild(_DeliverResults);
_ActOnTheRequest = new TransformBlock( (PollRequest x) => {
try
{
return ActOnTheRequest(x);
}
catch
{
return default;
}
},
new ExecutionDataflowBlockOptions() {
BoundedCapacity = 1,
MaxDegreeOfParallelism = 1
});
RegisterChild(_ActOnTheRequest);
_DecodeRequest = new TransformBlock( (string x) => {
try
{
return DecodeRequest(x);
}
catch
{
return default;
}
},
new ExecutionDataflowBlockOptions() {
BoundedCapacity = 1,
MaxDegreeOfParallelism = 1
});
RegisterChild(_DecodeRequest);
_ActOnTheRequest.LinkTo(_DeliverResults, new DataflowLinkOptions { PropagateCompletion = true });
_DecodeRequest.LinkTo(_ActOnTheRequest, new DataflowLinkOptions { PropagateCompletion = true });
}
TransformBlock _DeliverResults;
TransformBlock _ActOnTheRequest;
TransformBlock _DecodeRequest;
public override ITargetBlock InputBlock { get => _DecodeRequest ; }
public override ISourceBlock< bool > OutputBlock { get => _DeliverResults; }
public bool Call(string input) => InputBlock.Post(input);
public async Task Cast(string input) => await InputBlock.SendAsync(input);
public async Task AcceptAsync(CancellationToken cancellationToken)
{
try
{
var result = await _DeliverResults.ReceiveAsync(cancellationToken);
return result;
}
catch (OperationCanceledException operationCanceledException)
{
return await Task.FromCanceled(cancellationToken);
}
}
public async Task Ingest(CancellationToken ct)
{
// start the message pump
while (!ct.IsCancellationRequested)
{
var foundSomething = false;
try
{
// cycle through ingesters IN PRIORITY ORDER.
{
var msg = await ReceivePollRequest(ct);
if (msg != null)
{
Call(msg);
foundSomething = true;
// then jump back to the start of the pump
continue;
}
}
if (!foundSomething)
await Task.Delay(1000, ct);
}
catch (TaskCanceledException)
{
// if nothing was found on any of the receivers, then sleep for a while.
continue;
}
catch (Exception e)
{
// _logger.LogError(e, "Exception suppressed");
}
}
}
}
```
7. **Using the pipeline**:
```csharp
var actor = new MyActor(); // this is your pipeline
try
{
// call into the pipeline synchronously
if (actor.Call("""
{ "something": "here" }
"""))
Console.WriteLine("Called Synchronously");
// stop the pipeline after 10 secs
var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));
// kick off an endless process to keep ingesting input into the pipeline
var t = Task.Run(async () => await actor.Ingest(cts.Token), cts.Token);
// consume results from the last step via the AcceptAsync method
while (!cts.Token.IsCancellationRequested)
{
var result = await actor.AcceptAsync(cts.Token);
Console.WriteLine($"Result: {result}");
}
await t; // cancel the message pump task
await actor.SignalAndWaitForCompletionAsync(); // wait for all pipeline tasks to complete
}
catch (OperationCanceledException _)
{
Console.WriteLine("All Done!");
}
```
## Benefits
- Simplifies TPL Dataflow usage: Automatically generates boilerplate code.
- Concurrency: Efficient use of multiple CPU cores.
- Fault tolerance: Errors in pipeline steps are trapped and handled.
- Encapsulation: Easier to reason about and test code.
## Acknowledgements
Built on [DataflowEx](https://github.com/gridsum/DataflowEx) and [Bnaya.SourceGenerator.Template](https://github.com/bnayae/Bnaya.SourceGenerator.Template).