https://github.com/ethlo/kfka
Simple embeddable, persistent, rewindable, and filterable queue implementation with no dependencies.
https://github.com/ethlo/kfka
java kafka queue
Last synced: about 1 year ago
JSON representation
Simple embeddable, persistent, rewindable, and filterable queue implementation with no dependencies.
- Host: GitHub
- URL: https://github.com/ethlo/kfka
- Owner: ethlo
- License: apache-2.0
- Created: 2016-11-05T15:27:51.000Z (over 9 years ago)
- Default Branch: main
- Last Pushed: 2025-01-20T21:30:06.000Z (over 1 year ago)
- Last Synced: 2025-03-29T18:11:24.148Z (over 1 year ago)
- Topics: java, kafka, queue
- Language: Java
- Homepage:
- Size: 278 KB
- Stars: 5
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# kfka
[](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22com.ethlo.kfka%22%20a%3A%22kfka%22)
[](https://www.codacy.com/app/ethlo/kfka?utm_source=github.com&utm_medium=referral&utm_content=ethlo/kfka&utm_campaign=Badge_Grade)
[](LICENSE)
This project aims to give a subset of the benefits of [Apache Kafka](https://kafka.apache.org/) (persistent, queryable
event queue) without the overhead of managing a Kafka cluster, which sometimes may be overkill for lower message
volumes.
Kfka currently supports MySQL as a back-end, and has zero dependencies which makes it ideal as an embedded message
queue.
# Usage
### Configuration
```java
final Duration retentionTime = Duration.ofDays(3);
final Duration cleanInterval = Duration.ofMinutes(30);
final TaskScheduler taskScheduler = ...;
final DataSource dataSource = ...;
final RowMapper rowMapper = rs ->
new MyKfkaMessage.MessageBuilder()
.userId(rs.getInt("userId"))
.payload(rs.getBytes("payload"))
.timestamp(rs.getLong("timestamp"))
.topic(rs.getString("topic"))
.type(rs.getString("type"))
.messageId(rs.getString("message_id"))
.build();
final KfkaMessageStore msgStore = new MysqlKfkaMessageStore<>(dataSource, rowMapper, retentionTime);
final KfkaManager kfkaManager = new KfkaManagerImpl(msgStore);
taskScheduler.
scheduleAtFixedRate(kfkaManager::evictExpired, cleanInterval);
```
### MySQL table definition
```ddl
CREATE TABLE `kfka` (
`message_id` varbinary(16) NOT NULL PRIMARY KEY,
`timestamp` bigint NOT NULL,
`payload` blob NOT NULL,
`topic` varchar(255) NOT NULL,
`type` varchar(255) NOT NULL,
`userId` int,
PRIMARY KEY (`id`)
);
```
### Listen for events
```java
kfkaManager.addListener((msg)->
{
// TODO: Handle message
},
new
KfkaPredicate()
.
topic("my_topic")
.
rewind(100) // Rewind (up to) 100 messages
```
Use as a backend for SSE/Websockets messages
```java
kfkaManager.addListener((msg)->
{
// TODO: Handle message
},
new
KfkaPredicate()
.
topic("chat")
.
lastSeenMessageId(45_610) // Next message will be 45,611
.
addPropertyMatch("userId",123); // Filtering on custom property
```