Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/LGouellec/streamiz
.NET Stream Processing Library for Apache Kafka 🚀
https://github.com/LGouellec/streamiz
event-streaming kafka kafka-streams kafka-streams-dotnet kafka-streams-net streamiz
Last synced: about 2 months ago
JSON representation
.NET Stream Processing Library for Apache Kafka 🚀
- Host: GitHub
- URL: https://github.com/LGouellec/streamiz
- Owner: LGouellec
- License: mit
- Created: 2020-01-03T13:07:44.000Z (about 5 years ago)
- Default Branch: develop
- Last Pushed: 2024-11-04T19:44:40.000Z (2 months ago)
- Last Synced: 2024-11-08T04:46:29.738Z (about 2 months ago)
- Topics: event-streaming, kafka, kafka-streams, kafka-streams-dotnet, kafka-streams-net, streamiz
- Language: C#
- Homepage: https://lgouellec.github.io/streamiz/
- Size: 12.1 MB
- Stars: 465
- Watchers: 33
- Forks: 74
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Roadmap: roadmap.md
Awesome Lists containing this project
- awesome-streaming - Streamiz - a .Net Stream Processing Library for Apache Kafka (Table of Contents / Streaming Library)
README
> [!IMPORTANT]
We’re excited to hear from you and would love to get your feedback on the product. Your insights are invaluable and will help us shape the future of our product to better meet your needs. The [survey](https://docs.google.com/forms/d/1OVISLOQY0FLcvh9KhSEPZ5K2Lgqzl7NkciCyd6w9_kU/) will only take a few minutes, and your responses will be completely confidential.
# .NET Stream Processing Library for Apache Kafka TM · [](https://github.com/LGouellec/streamiz/blob/master/LICENSE) · [](https://discord.gg/J7Jtxum) 
| Package | Nuget version | Downloads |
|---|---|---|
| Streamiz.Kafka.Net |  |  |
| Streamiz.Kafka.Net.SchemaRegistry.SerDes |  |  |
| Streamiz.Kafka.Net.SchemaRegistry.SerDes.Avro |  |  |
| Streamiz.Kafka.Net.SchemaRegistry.SerDes.Protobuf |  |  |
| Streamiz.Kafka.Net.SchemaRegistry.SerDes.Json |  |  |
| Streamiz.Kafka.Net.Metrics.Prometheus |  |  |
| Streamiz.Kafka.Net.Metrics.OpenTelemetry |  |  |
## Quality Statistics
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
[](https://sonarcloud.io/dashboard?branch=master&id=LGouellec_kafka-streams-dotnet)
----
Streamiz Kafka .NET is .NET stream processing library for Apache Kafka.
```
KAFKA is a registered trademark of The Apache Software Foundation and
has been licensed for use by Streamiz. Streamiz has no
affiliation with and is not endorsed by The Apache Software Foundation.
```
# Try it with Gitpod
[](https://gitpod.io/#https://github.com/LGouellec/streamiz)
## Step 1
Waiting run task is complete. The task is consider complete some seconds after viewing this message `"🚀 Enjoy Streamiz the .NET Stream processing library for Apache Kafka (TM)"`
## Step 2
Switch to `producer` terminal and send sentences or word. The sample case is "Count the number words" similar to [here](https://developpaper.com/kafka-stream-word-count-instance/)
## Step 3
Switch to `consumer`terminal and check aggregation result
# Documentation
Read the full documentation on https://lgouellec.github.io/streamiz/
# Installation
Nuget packages are listed to [nuget.org](https://www.nuget.org/packages/Streamiz.Kafka.Net/)
Install the last version with :
```shell
dotnet add package Streamiz.Kafka.Net
```
# Usage
There, a sample streamiz application :
``` csharp
static async System.Threading.Tasks.Task Main(string[] args)
{
var config = new StreamConfig();
config.ApplicationId = "test-app";
config.BootstrapServers = "localhost:9092";
StreamBuilder builder = new StreamBuilder();
var kstream = builder.Stream("stream");
var ktable = builder.Table("table", InMemory.As("table-store"));
kstream.Join(ktable, (v, v1) => $"{v}-{v1}")
.To("join-topic");
Topology t = builder.Build();
KafkaStream stream = new KafkaStream(t, config);
Console.CancelKeyPress += (o, e) => {
stream.Dispose();
};
await stream.StartAsync();
}
```
# Compare Kafka Streams vs Streamiz
| **_Features_** | **Kafka Streams (JAVA) supported** | **Streamiz supported** | **Comment** |
|:------------------------------------------------------------:|:----------------------------------:|:----------------------:|:------------------------------------------:|
| Stateless processors | X | X | |
| RocksDb store | X | X | |
| Standby replicas | X | | No plan for now |
| InMemory store | X | X | |
| Transformer, Processor API | X | X | |
| Punctuate | X | X | |
| KStream-KStream Join | X | X | |
| KTable-KTable Join | X | X | |
| KTable-KTable FK Join | X | | Plan for future |
| KStream-KTable Join | X | X | |
| KStream-GlobalKTable Join | X | X | |
| KStream Async Processing (external call inside the topology) | | X | Not supported in Kafka Streams JAVA |
| Hopping window | X | X | |
| Tumbling window | X | X | |
| Sliding window | X | | No plan for now |
| Session window | X | | No plan for now |
| Cache | X | X | EA 1.6.0 |
| Suppress(..) | X | X | Plan for 1.7.0 |
| Interactive Queries | X | | No plan for now |
| State store batch restoring | X | | No plan for now |
| Exactly Once v2 | X | X | |
# Community Support
Feel free to reach out to our community support [here](https://discord.gg/J7Jtxum) anytime; we're here to help you with any questions or issues you may have!