{"id":50863988,"url":"https://github.com/onrcanogul/mini-message-broker","last_synced_at":"2026-06-14T23:31:03.535Z","repository":{"id":364082005,"uuid":"1266066734","full_name":"onrcanogul/mini-message-broker","owner":"onrcanogul","description":"A from-scratch, minimal message broker in Java: append-only commit log + TCP produce/fetch. Rebuilding Kafka's core internals (log, offset, partitioning, replication) to learn them.","archived":false,"fork":false,"pushed_at":"2026-06-11T14:10:08.000Z","size":41,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-11T16:09:48.719Z","etag":null,"topics":["append-only-log","commit-log","distributed-systems","event-streaming","java","kafka","learning-project","message-broker","pub-sub","tcp"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/onrcanogul.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-11T09:44:37.000Z","updated_at":"2026-06-11T14:21:14.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/onrcanogul/mini-message-broker","commit_stats":null,"previous_names":["onrcanogul/mini-message-broker"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/onrcanogul/mini-message-broker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onrcanogul%2Fmini-message-broker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onrcanogul%2Fmini-message-broker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onrcanogul%2Fmini-message-broker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onrcanogul%2Fmini-message-broker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/onrcanogul","download_url":"https://codeload.github.com/onrcanogul/mini-message-broker/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onrcanogul%2Fmini-message-broker/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34342089,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-14T02:00:07.365Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["append-only-log","commit-log","distributed-systems","event-streaming","java","kafka","learning-project","message-broker","pub-sub","tcp"],"created_at":"2026-06-14T23:31:02.710Z","updated_at":"2026-06-14T23:31:03.525Z","avatar_url":"https://github.com/onrcanogul.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MiniBroker\n\nMinimal message broker (Kafka-style append-only log) in plain Java — no frameworks, JDK + JUnit only.\n\n**v0 scope:** single broker, single partition (`partition 0`), append-only log + TCP produce/fetch,\ncrash recovery (corrupt-tail truncation + log rediscovery on restart).\n\n## Layout\n\nPackages are split by responsibility; dependencies flow one way:\n`storage` → `protocol` → `broker` / `client` → `Main`.\n\n```\nsrc/main/java/minibroker/\n├── Main.java                 # entry point: wires everything up\n├── storage/                  # Record, StoredRecord, Log   (append-only file, recovery)\n├── protocol/                 # Protocol, ApiKey, ErrorCode, {Produce,Fetch}{Request,Response}\n├── broker/                   # Config, LogManager, RequestHandler, BrokerServer\n└── client/                   # BrokerClient (thin wire client)\nsrc/test/java/minibroker/     # tests mirror the package they cover; AcceptanceTest at root\nlib/                          # junit-platform-console-standalone.jar (downloaded, git-ignored)\nscripts/                      # setup.sh, build.sh, test.sh, run.sh\n```\n\n## Quick start\n\n```bash\nscripts/setup.sh    # one-time: download the JUnit jar into lib/\nscripts/build.sh    # compile main + tests into out/\nscripts/test.sh     # build + run the full JUnit suite\nscripts/run.sh      # build + start the broker (optional: scripts/run.sh /tmp/mb)\n```\n\nThe broker listens on port 9092 and stores logs under `./data` (or the dir you pass).\n\n\u003cdetails\u003e\n\u003csummary\u003eRaw commands (no scripts)\u003c/summary\u003e\n\n```bash\nJAR=lib/junit-platform-console-standalone.jar\njavac -d out -cp \"$JAR\" $(find src -name '*.java')           # build\njava -jar \"$JAR\" execute -cp out --scan-classpath            # test\njava -cp out minibroker.Main /tmp/mb                         # run\n```\n\u003c/details\u003e\n\n## Try it from code\n\nUse `BrokerClient` against a running broker:\n\n```java\ntry (BrokerClient c = BrokerClient.connect(\"localhost\", 9092)) {\n    long base = c.produce(\"orders\", List.of(\n        new Record(\"k1\".getBytes(), \"hello\".getBytes(), 0)));   // -\u003e baseOffset\n    BrokerClient.FetchResult res = c.fetch(\"orders\", 0, 1 \u003c\u003c 20);\n    // res.records(), res.highWatermark()\n}\n```\n\n## On-disk record format (big-endian, `partition.log`)\n\n```\nrecLen(int32) | offset(int64) | timestamp(int64) |\nkeyLen(int32, -1=null) | key | valLen(int32) | value | crc32(int32)\n```\n\n`recLen` frames each record (skip / half-tail detection); `crc32` detects corruption.\nOn restart, the last partial/corrupt record is truncated; intact records are kept.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonrcanogul%2Fmini-message-broker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fonrcanogul%2Fmini-message-broker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonrcanogul%2Fmini-message-broker/lists"}