{"id":16825322,"url":"https://github.com/tychedelia/josefine","last_synced_at":"2025-08-11T17:07:42.076Z","repository":{"id":38331334,"uuid":"156034883","full_name":"tychedelia/josefine","owner":"tychedelia","description":"💃 Kafka/Raft implementation in Rust","archived":false,"fork":false,"pushed_at":"2024-07-30T22:20:15.000Z","size":836,"stargazers_count":24,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-04T03:24:53.447Z","etag":null,"topics":["kafka","raft","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tychedelia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-04T00:11:42.000Z","updated_at":"2024-12-06T01:58:14.000Z","dependencies_parsed_at":"2023-12-17T06:19:46.164Z","dependency_job_id":"dfcc8962-6834-485d-bc0e-8723fb28e5b4","html_url":"https://github.com/tychedelia/josefine","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tychedelia/josefine","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tychedelia%2Fjosefine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tychedelia%2Fjosefine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tychedelia%2Fjosefine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tychedelia%2Fjosefine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tychedelia","download_url":"https://codeload.github.com/tychedelia/josefine/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tychedelia%2Fjosefine/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269922906,"owners_count":24496999,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-11T02:00:10.019Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka","raft","rust"],"created_at":"2024-10-13T11:13:27.006Z","updated_at":"2025-08-11T17:07:42.023Z","avatar_url":"https://github.com/tychedelia.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# josefine\n\n\u003e So perhaps we shall not miss so very much after all, while Josephine, for her part, \n\u003e delivered from earthly afflictions, which however to her mind are the privilege of \n\u003e chosen spirits, will happily lose herself in the countless throng of the heroes of \n\u003e our people, and soon, since we pursue no history, be accorded the heightened relief\n\u003e of being forgotten along with all her brethren.\n\n\\- Franz Kafka, *Josefine, die Sängerin oder Das Volk der Mäuse*\n\n### Project Description\n\nA toy implementation of [Kafka](https://kafka.apache.org/), a distributed, replicated\nevent stream, using an implementation of [Chained Raft](https://decentralizedthoughts.github.io/2021-07-17-simplifying-raft-with-chaining/)\nfor managing cluster state.\n\nJosefine speaks the [Kafka wire protocol](https://kafka.apache.org/protocol.html), using\n100% Rust implementation of the protocol, [kafka-protocol-rs](https://github.com/tychedelia/kafka-protocol-rs).\n\nTraditionally, Kafka has used ZooKeeper in order to maintain cluster state and \nconfiguration, while storage and partitioning of the event log itself\nwas handled by Kafka brokers. Over time, more and more operations in the Kafka API\nhave come to be routed through the brokers themselves, and there is currently an\n[experimental version](https://developer.confluent.io/learn/kraft/) of Kafka \nthat removes the ZooKeeper dependency entirely.\n\nJosefine maintains a Raft cluster comprised of all brokers. Leadership elections\nin the logical Raft cluster are independent of any kind of replication of the data\nat the level of the API.\n\n## Chained Raft\n\nThe algorithm described in the [original Raft paper](https://raft.github.io/raft.pdf) is \na replicated log, where each entry is tagged with an index in the log and a leadership\nterm at which it was written. A consequence of this is that when nodes fail and elections\nare triggered in the Raft cluster, sometimes portions of a node's log must be rewritten\nwith the latest up-to-date log sent from a new leader. Ensuring this happens correctly\nis one of the most difficult parts of implementing Raft correctly.\n\nOn the other hand, in Chained Raft, log entries become \"blocks\" in an acyclic graph, which\nonly maintain a pointer to the next block. As the paper illustrates, by maintaining a reference\nto the current head of the chain, we can always walk backwards pointer by pointer to find the\nvalid current state of the chain. Rather than having to overwrite existing entries, un-replicated\nblocks simply become dead branches of the tree that can be garbage collected later.\n\nThis is particularly well suited for something like Kafka, which in many smaller installations\nhas relatively infrequent cluster state changes. Since the replication of data is handled by \nthe brokers themselves and not a state machine in Raft. Cluster metadata is mostly used for things\nlike ensuring topics and broker assignment to particular partitions.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftychedelia%2Fjosefine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftychedelia%2Fjosefine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftychedelia%2Fjosefine/lists"}