https://github.com/masesgroup/kefcore
Entity Framework Core provider for Apache Kafka™ — use Kafka as a distributed database, and more, with full LINQ support. Compatible with any Kafka™ wire-protocol broker.
https://github.com/masesgroup/kefcore
apache-kafka c-sharp confluent confluent-kafka database dotnet dotnet-core dotnet-framework dotnet-standard entity-framework kafka kafka-cluster kafka-streams orm
Last synced: about 1 month ago
JSON representation
Entity Framework Core provider for Apache Kafka™ — use Kafka as a distributed database, and more, with full LINQ support. Compatible with any Kafka™ wire-protocol broker.
- Host: GitHub
- URL: https://github.com/masesgroup/kefcore
- Owner: masesgroup
- License: apache-2.0
- Created: 2022-02-10T02:55:00.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2026-05-20T21:07:29.000Z (about 1 month ago)
- Last Synced: 2026-05-21T02:05:41.474Z (about 1 month ago)
- Topics: apache-kafka, c-sharp, confluent, confluent-kafka, database, dotnet, dotnet-core, dotnet-framework, dotnet-standard, entity-framework, kafka, kafka-cluster, kafka-streams, orm
- Language: C#
- Homepage: https://kefcore.masesgroup.com/
- Size: 52.2 MB
- Stars: 9
- Watchers: 0
- Forks: 0
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# KEFCore: [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/)
KEFCore is the [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/).
Based on [KNet client-side features](https://github.com/masesgroup/KNet) it allows to use [Apache Kafka™](https://kafka.apache.org/) as a distributed database and more: KNet client-side features are also compatible with any broker that implements the Kafka wire protocol — see [Backend compatibility](#backend-compatibility) below.
### Libraries and Tools
|Core | Templates | Json Serialization | Avro Serialization | Protobuf Serialization |
|:---: |:---: |:---: |:---: |:---: |
|[](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet) [](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet) | [](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet.Templates) [](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet.Templates) | [](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet.Serialization) [](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet.Serialization) | [](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet.Serialization.Avro) [](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet.Serialization.Avro) | [](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet.Serialization.Protobuf) [](https://www.nuget.org/packages/MASES.EntityFrameworkCore.KNet.Serialization.Protobuf) |
### Pipelines
[](https://github.com/masesgroup/KEFCore/actions/workflows/build.yaml)
[](https://github.com/masesgroup/KEFCore/actions/workflows/release.yaml)
### Project disclaimer
KEFCore is a project curated by MASES Group and supported by the open-source community.
Its primary scope is to support other MASES Group projects — both open-source and commercial — though it is freely available for any use. Dedicated community and commercial subscription plans are available.
The repository and releases may contain bugs. The release cycle depends on critical issues discovered and/or enhancement requests from this or other dependent projects.
Looking for [Entity Framework Core](https://learn.microsoft.com/ef/core/) and [Apache Kafka™](https://kafka.apache.org/) expertise? MASES Group can help you design, build, deploy, and manage [Entity Framework Core](https://learn.microsoft.com/ef/core/) and [Apache Kafka™](https://kafka.apache.org/) applications. [Find out more.](src/documentation/articles/support.md)
---
## Scope of the project
KEFCore provides an [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/), enabling .NET applications to use Kafka topics as a data store through the standard EF Core programming model — `DbContext`, LINQ queries, and strongly-typed entities — with no Kafka-specific consumer or producer code.
The [EF Core introduction page](https://learn.microsoft.com/ef/core/) opens with this example:
```c#
public class BloggingContext : DbContext
{
public DbSet Blogs { get; set; }
public DbSet Posts { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer(
@"Server=(localdb)\mssqllocaldb;Database=Blogging;Trusted_Connection=True;ConnectRetryCount=0");
}
}
```
With KEFCore, replacing the SQL Server backend with an Apache Kafka™ cluster requires changing a single line:
```c#
optionsBuilder.UseKEFCore("my-application", "localhost:9092");
```
From that point on, standard EF Core code works unchanged against Kafka topics:
```c#
// Query
var blogs = await db.Blogs
.Where(b => b.Rating > 3)
.OrderBy(b => b.Url)
.ToListAsync();
// Write
db.Blogs.Add(new Blog { Url = "http://sample.com" });
await db.SaveChangesAsync();
```
KEFCore is developed following the guidelines in the official [EntityFrameworkCore repository](https://github.com/dotnet/efcore) and the [Writing a provider](https://docs.microsoft.com/ef/core/providers/writing-a-provider) documentation published by Microsoft.
Currently the project tries to support, at our best, the [official supported Apache Kafka™ binary distribution](https://kafka.apache.org/downloads):
| KEFCore | State | KNet | Apache Kafka™ | .NET | JVM™ |
|:---: |:---: |:---: |:---: |:---: |:---: |
| 2.6.*+ | Active | 3.2.x | 4.2.x | [](https://dotnet.microsoft.com/) | [](https://www.oracle.com/java/) |
| 2.5.* | Deprecated | 2.9.x | 3.9.x | [](https://dotnet.microsoft.com/) | [](https://www.oracle.com/java/) |
---
## Backend compatibility
[Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/) uses the official Apache Kafka™ Java client packages directly through [KNet client-side features](https://github.com/masesgroup/KNet). This architecture has a direct impact on backend compatibility.
**Client-side features** — Producer, Consumer, Admin Client, Kafka Streams, KNet Streams SDK, KNet Connect SDK, KNetPS scriptable cmdlets — communicate with the broker exclusively through the Kafka wire protocol and are therefore compatible with **any broker that implements it**, not only Apache Kafka™ itself.
Examples of compatible brokers: [Redpanda](https://redpanda.com/), [Amazon MSK](https://aws.amazon.com/msk/), [Confluent Platform / Cloud](https://www.confluent.io/), [Aiven for Apache Kafka™](https://aiven.io/kafka), [IBM Event Streams](https://www.ibm.com/products/event-streams), [WarpStream](https://www.warpstream.com/), [AutoMQ](https://www.automq.com/), and others.
See [Supported Backends](src/documentation/articles/backends.md) for the full compatibility matrix covering all KNet feature areas.
---
## Community and Contribution
If you find [Entity Framework Core](https://learn.microsoft.com/ef/core/) provider for [Apache Kafka™](https://kafka.apache.org/) useful:
* Leave a ⭐ on the repository
* Open [issues](https://github.com/masesgroup/KEFCore/issues) to report bugs 🐛 or request features
* Submit Pull Requests to improve the project
This project adheres to the Contributor [Covenant code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to coc_reporting@masesgroup.com.
---
## Summary
* [Getting started](src/documentation/articles/gettingstarted.md)
* [How it works](src/documentation/articles/howitworks.md)
* [Supported backends](src/documentation/articles/backends.md)
* [Usage](src/documentation/articles/usage.md)
* [Use cases](src/documentation/articles/usecases.md)
* [Templates usage](src/documentation/articles/usageTemplates.md)
* [Options](src/documentation/articles/options.md)
* [Conventions](src/documentation/articles/conventions.md)
* [Serialization](src/documentation/articles/serialization.md)
* [Schema migration](src/documentation/articles/migration.md)
* [Performance tips](src/documentation/articles/performancetips.md)
* [Troubleshooting](src/documentation/articles/troubleshooting.md)
* [External application](src/documentation/articles/externalapplication.md)
* [Roadmap](src/documentation/articles/roadmap.md)
* [Current state](src/documentation/articles/currentstate.md)
* [KEFCoreDbContext](src/documentation/articles/kefcoredbcontext.md)
---
## Runtime engine
KEFCore uses [KNet](https://github.com/masesgroup/KNet), and indeed [JCOBridge](https://www.jcobridge.com) with its [features](https://www.jcobridge.com/features/), to obtain many benefits:
* **Cyber-security**:
* [JVM™](https://en.wikipedia.org/wiki/Java_virtual_machine) and [CLR, or CoreCLR,](https://en.wikipedia.org/wiki/Common_Language_Runtime) runs in the same process, but are insulated from each other;
* JCOBridge does not make any code injection into JVM™;
* JCOBridge does not use any other communication mechanism than JNI;
* .NET (CLR) inherently inherits the cyber-security levels of running JVM™ and Apache Kafka™;
* **Direct access the JVM™ from any .NET application**:
* Any Java/Scala class behind Apache Kafka™ can be directly managed: Consumer, Producer, Administration, Streams, Server-side, and so on;
* No need to learn new APIs: we try to expose the same APIs in C# style;
* No extra validation cycle on protocol and functionality: bug fix, improvements, new features are immediately available;
* Documentation is shared.
> [!NOTE]
> [JCOBridge 2.6.\*](https://www.jcobridge.com) can be used for free without any obligations. A commercial license must be purchased — or the software uninstalled — if you derive direct or indirect income from its usage.
### JCOBridge resources
Have a look at the following JCOBridge resources:
|JCOBridge | 2.5.* series | 2.6.* series |
|:---: |:---: |:---: |
|KEFCore | > 1.0.* series | > 2.6.1 series |
|Release notes|[Link](https://www.jcobridge.com/release-notes/)| [Link](https://www.jcobridge.com/release-notes/)|
|Community Edition|[Conditions](https://www.jcobridge.com/pricing-25/)|[Conditions](https://www.jcobridge.com/pricing-26/)|
|Commercial Edition|[Information](https://www.jcobridge.com/pricing-25/)|[Information](https://www.jcobridge.com/pricing-26/)|
Latest release: [](https://www.nuget.org/packages/MASES.JCOBridge)
---
KAFKA is a registered trademark of The Apache Software Foundation. KEFCore has no affiliation with and is not endorsed by The Apache Software Foundation.
Microsoft is a registered trademark of Microsoft Corporation.
EntityFramework is a registered trademark of Microsoft Corporation.