{"id":15492282,"url":"https://github.com/ethlo/kfka","last_synced_at":"2025-04-22T19:26:33.081Z","repository":{"id":29280795,"uuid":"72934830","full_name":"ethlo/kfka","owner":"ethlo","description":"Simple embeddable, persistent, rewindable, and filterable queue implementation with no dependencies.","archived":false,"fork":false,"pushed_at":"2025-01-20T21:30:06.000Z","size":285,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-29T18:11:24.148Z","etag":null,"topics":["java","kafka","queue"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ethlo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-11-05T15:27:51.000Z","updated_at":"2025-01-20T21:30:10.000Z","dependencies_parsed_at":"2025-01-18T14:15:39.787Z","dependency_job_id":"cfd26d34-fdd7-46fa-8fab-0a8e987f92ab","html_url":"https://github.com/ethlo/kfka","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethlo%2Fkfka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethlo%2Fkfka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethlo%2Fkfka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethlo%2Fkfka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ethlo","download_url":"https://codeload.github.com/ethlo/kfka/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250307731,"owners_count":21409127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["java","kafka","queue"],"created_at":"2024-10-02T08:00:01.657Z","updated_at":"2025-04-22T19:26:33.057Z","avatar_url":"https://github.com/ethlo.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kfka\n\n[![Maven Central](https://img.shields.io/maven-central/v/com.ethlo.kfka/kfka.svg)](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22com.ethlo.kfka%22%20a%3A%22kfka%22)\n[![Codacy Badge](https://api.codacy.com/project/badge/Grade/ee97abb9994d44c7b61e533454368dd0)](https://www.codacy.com/app/ethlo/kfka?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=ethlo/kfka\u0026amp;utm_campaign=Badge_Grade)\n[![Hex.pm](https://img.shields.io/hexpm/l/plug.svg)](LICENSE)\n\nThis project aims to give a subset of the benefits of [Apache Kafka](https://kafka.apache.org/) (persistent, queryable\nevent queue) without the overhead of managing a Kafka cluster, which sometimes may be overkill for lower message\nvolumes.\n\nKfka currently supports MySQL as a back-end, and has zero dependencies which makes it ideal as an embedded message\nqueue.\n\n# Usage\n\n### Configuration\n\n```java\nfinal Duration retentionTime = Duration.ofDays(3);\nfinal Duration cleanInterval = Duration.ofMinutes(30);\n\nfinal TaskScheduler taskScheduler = ...;\nfinal DataSource dataSource = ...;\nfinal RowMapper\u003cMyKfkaMessage\u003e rowMapper = rs -\u003e\n        new MyKfkaMessage.MessageBuilder()\n                .userId(rs.getInt(\"userId\"))\n                .payload(rs.getBytes(\"payload\"))\n                .timestamp(rs.getLong(\"timestamp\"))\n                .topic(rs.getString(\"topic\"))\n                .type(rs.getString(\"type\"))\n                .messageId(rs.getString(\"message_id\"))\n                .build();\n\nfinal KfkaMessageStore msgStore = new MysqlKfkaMessageStore\u003c\u003e(dataSource, rowMapper, retentionTime);\nfinal KfkaManager kfkaManager = new KfkaManagerImpl(msgStore);\ntaskScheduler.\n\nscheduleAtFixedRate(kfkaManager::evictExpired, cleanInterval);\n```\n\n### MySQL table definition\n\n```ddl\nCREATE TABLE `kfka` (\n  `message_id` varbinary(16) NOT NULL PRIMARY KEY,\n  `timestamp` bigint NOT NULL,\n  `payload` blob NOT NULL,\n  `topic` varchar(255) NOT NULL,\n  `type` varchar(255) NOT NULL,\n  `userId` int,\n  PRIMARY KEY (`id`)\n);\n```\n\n### Listen for events\n\n```java\nkfkaManager.addListener((msg)-\u003e\n        {\n        // TODO: Handle message\n        },\n        new\n\nKfkaPredicate()\n  .\n\ntopic(\"my_topic\")\n  .\n\nrewind(100) // Rewind (up to) 100 messages\n```\n\nUse as a backend for SSE/Websockets messages\n\n```java\nkfkaManager.addListener((msg)-\u003e\n        {\n        // TODO: Handle message\n        },\n        new\n\nKfkaPredicate()\n  .\n\ntopic(\"chat\")\n  .\n\nlastSeenMessageId(45_610) // Next message will be 45,611\n  .\n\naddPropertyMatch(\"userId\",123); // Filtering on custom property\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethlo%2Fkfka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fethlo%2Fkfka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethlo%2Fkfka/lists"}