Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/LGouellec/kafka-streams-dotnet
.NET Stream Processing Library for Apache Kafka 🚀
https://github.com/LGouellec/kafka-streams-dotnet
event-streaming kafka kafka-streams kafka-streams-dotnet kafka-streams-net streamiz
Last synced: 3 months ago
JSON representation
.NET Stream Processing Library for Apache Kafka 🚀
- Host: GitHub
- URL: https://github.com/LGouellec/kafka-streams-dotnet
- Owner: LGouellec
- License: mit
- Created: 2020-01-03T13:07:44.000Z (almost 5 years ago)
- Default Branch: develop
- Last Pushed: 2024-08-01T21:45:21.000Z (3 months ago)
- Last Synced: 2024-08-02T14:12:17.697Z (3 months ago)
- Topics: event-streaming, kafka, kafka-streams, kafka-streams-dotnet, kafka-streams-net, streamiz
- Language: C#
- Homepage: https://lgouellec.github.io/kafka-streams-dotnet/
- Size: 11.9 MB
- Stars: 455
- Watchers: 33
- Forks: 74
- Open Issues: 19
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
- awesome-streaming - Streamiz - a .Net Stream Processing Library for Apache Kafka (Table of Contents / Streaming Library)
README
# .NET Stream Processing Library for Apache Kafka TM · [](https://github.com/LGouellec/streamiz-kafka-net/blob/master/LICENSE) · [](https://discord.gg/J7Jtxum) 
| Package | Nuget version | Downloads |
|---|---|---|
| Streamiz.Kafka.Net |  |  |
| Streamiz.Kafka.Net.SchemaRegistry.SerDes |  |  |
| Streamiz.Kafka.Net.SchemaRegistry.SerDes.Avro |  |  |
| Streamiz.Kafka.Net.SchemaRegistry.SerDes.Protobuf |  |  |
| Streamiz.Kafka.Net.SchemaRegistry.SerDes.Json |  |  |
| Streamiz.Kafka.Net.Metrics.Prometheus |  |  |
| Streamiz.Kafka.Net.Metrics.OpenTelemetry |  |  |
## Quality Statistics
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
## Project Statistics
----
Streamiz Kafka .NET is .NET stream processing library for Apache Kafka.
```
KAFKA is a registered trademark of The Apache Software Foundation and
has been licensed for use by Streamiz. Streamiz has no
affiliation with and is not endorsed by The Apache Software Foundation.
```
# Try it with Gitpod
[](https://gitpod.io/#https://github.com/LGouellec/kafka-streams-dotnet)
## Step 1
Waiting run task is complete. The task is consider complete some seconds after viewing this message `"🚀 Enjoy Streamiz the .NET Stream processing library for Apache Kafka (TM)"`
## Step 2
Switch to `producer` terminal and send sentences or word. The sample case is "Count the number words" similar to [here](https://developpaper.com/kafka-stream-word-count-instance/)
## Step 3
Switch to `consumer`terminal and check aggregation result
# Documentation
Read the full documentation on https://lgouellec.github.io/kafka-streams-dotnet/
# Installation
Nuget packages are listed to [nuget.org](https://www.nuget.org/packages/Streamiz.Kafka.Net/)
Install the last version with :
```shell
dotnet add package Streamiz.Kafka.Net
```
# Usage
There, a sample streamiz application :
``` csharp
static async System.Threading.Tasks.Task Main(string[] args)
{
var config = new StreamConfig();
config.ApplicationId = "test-app";
config.BootstrapServers = "localhost:9092";
StreamBuilder builder = new StreamBuilder();
var kstream = builder.Stream("stream");
var ktable = builder.Table("table", InMemory.As("table-store"));
kstream.Join(ktable, (v, v1) => $"{v}-{v1}")
.To("join-topic");
Topology t = builder.Build();
KafkaStream stream = new KafkaStream(t, config);
Console.CancelKeyPress += (o, e) => {
stream.Dispose();
};
await stream.StartAsync();
}
```
# Compare Kafka Streams vs Streamiz
| **_Features_** | **Kafka Streams (JAVA) supported** | **Streamiz supported** | **Comment** |
|:------------------------------------------------------------:|:----------------------------------:|:----------------------:|:------------------------------------------:|
| Stateless processors | X | X | |
| RocksDb store | X | X | |
| Standby replicas | X | | No plan for now |
| InMemory store | X | X | |
| Transformer, Processor API | X | X | |
| Punctuate | X | X | |
| KStream-KStream Join | X | X | |
| KTable-KTable Join | X | X | |
| KTable-KTable FK Join | X | | Plan for future |
| KStream-KTable Join | X | X | |
| KStream-GlobalKTable Join | X | X | |
| KStream Async Processing (external call inside the topology) | | X | Not supported in Kafka Streams JAVA |
| Hopping window | X | X | |
| Tumbling window | X | X | |
| Sliding window | X | | No plan for now |
| Session window | X | | No plan for now |
| Cache | X | X | EA 1.6.0 |
| Suppress(..) | X | | No plan for now |
| Interactive Queries | X | | No plan for now |
| State store batch restoring | X | | No plan for now |
| Exactly Once (v1 and v2) | X | X | EOS V1 supported, EOS V2 not supported yet |
# Contributing
Maintainers:
- [lgouellec](https://github.com/LGouellec)
**Streamiz Kafka .Net** is a community project. We invite your participation through issues and pull requests! You can peruse the [contributing guidelines](CONTRIBUTING.md).
When adding or changing a service please add tests and documentations.
# Support
You can found support [here](https://discord.gg/J7Jtxum)
# Star History
