{"id":13470494,"url":"https://github.com/travisjeffery/jocko","last_synced_at":"2025-05-14T12:10:53.690Z","repository":{"id":39708275,"uuid":"70539263","full_name":"travisjeffery/jocko","owner":"travisjeffery","description":"Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)","archived":false,"fork":false,"pushed_at":"2023-11-13T02:21:21.000Z","size":11269,"stargazers_count":4973,"open_issues_count":60,"forks_count":361,"subscribers_count":226,"default_branch":"master","last_synced_at":"2025-04-11T04:59:08.995Z","etag":null,"topics":["distributed-systems","go","kafka","messaging","queue","streaming"],"latest_commit_sha":null,"homepage":"https://twitter.com/travisjeffery","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/travisjeffery.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":"FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"travisjeffery"}},"created_at":"2016-10-11T00:09:47.000Z","updated_at":"2025-04-08T16:58:09.000Z","dependencies_parsed_at":"2022-08-09T15:24:52.562Z","dependency_job_id":"15204c2d-6396-4f58-a927-a62a6c688644","html_url":"https://github.com/travisjeffery/jocko","commit_stats":{"total_commits":515,"total_committers":24,"mean_commits":"21.458333333333332","dds":0.09126213592233012,"last_synced_commit":"9613083803fc7d0fefd10d6d0cf00223d13ba301"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisjeffery%2Fjocko","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisjeffery%2Fjocko/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisjeffery%2Fjocko/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisjeffery%2Fjocko/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/travisjeffery","download_url":"https://codeload.github.com/travisjeffery/jocko/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254140760,"owners_count":22021219,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-systems","go","kafka","messaging","queue","streaming"],"created_at":"2024-07-31T16:00:31.268Z","updated_at":"2025-05-14T12:10:53.634Z","avatar_url":"https://github.com/travisjeffery.png","language":"Go","funding_links":["https://github.com/sponsors/travisjeffery"],"categories":["Go","Databases","Libraries","Streaming \u0026 Messaging","Repositories","Message Queue Middlewares","Alternative Implementations"],"sub_categories":["Kafka","Integrations"],"readme":"# Jocko\n\n![ci](https://github.com/travisjeffery/jocko/workflows/Go/badge.svg)\n[![gitter](https://badges.gitter.im/travisjeffery/jocko.svg)](https://gitter.im/travisjeffery/jocko?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n[![codecov](https://codecov.io/gh/travisjeffery/jocko/branch/master/graph/badge.svg)](https://codecov.io/gh/travisjeffery/jocko)\n\nKafka/distributed commit log service in Go.\n\n## Goals of this project:\n\n- Implement Kafka in Go\n- Protocol compatible with Kafka so Kafka clients and services work with Jocko\n- Make operating simpler\n- Distribute a single binary\n- Use Serf for discovery, Raft for consensus (and remove the need to run ZooKeeper)\n- Smarter configuration settings\n    - Able to use percentages of disk space for retention policies rather than only bytes and time kept\n    - Handling size configs when you change the number of partitions or add topics\n- Learn a lot and have fun\n\n## TODO\n\n- [x] Producing\n- [x] Fetching\n- [x] Partition consensus and distribution\n- [ ] Protocol\n    - [x] Produce\n    - [x] Fetch\n    - [x] Metadata\n    - [x] Create Topics\n    - [x] Delete Topics\n    - [ ] Consumer group [current task]\n- [x] Discovery\n- [ ] API versioning [more API versions to implement]\n- [ ] Replication [first draft done - testing heavily now]\n\n## Hiatus Writing Book\n\nI’m writing a book for PragProg called Building Distributed Services with Go. [You can sign up on this mailing list and get updated when the book’s available.](http://eepurl.com/dC5-l1) It walks you through building a distributed commit log from scratch. I hope it will help Jocko contributors and people who want to work on distributed services.\n\n## Reading\n\n- [How Jocko's built-in service discovery and consensus works](https://medium.com/the-hoard/building-a-kafka-that-doesnt-depend-on-zookeeper-2c4701b6e961#.uamxtq1yz)\n- [How Jocko's (and Kafka's) storage internals work](https://medium.com/the-hoard/how-kafkas-storage-internals-work-3a29b02e026#.qfbssm978)\n\n## Project Layout\n\n```\n├── broker        broker subsystem\n├── cmd           commands\n│   └── jocko     command to run a Jocko broker and manage topics\n├── commitlog     low-level commit log implementation\n├── examples      examples running/using Jocko\n│   ├── cluster   example booting up a 3-broker Jocko cluster\n│   └── sarama    example producing/consuming with Sarama\n├── protocol      golang implementation of Kafka's protocol\n├── prometheus    wrapper around Prometheus' client lib to handle metrics\n├── server        API subsystem\n└── testutil      test utils\n    └── mock      mocks of the various subsystems\n```\n\n## Building\n\n### Local\n\n1. Clone Jocko\n\n    ```\n    $ go get github.com/travisjeffery/jocko\n    ```\n\n1. Build Jocko\n\n    ```\n    $ cd $GOPATH/src/github.com/travisjeffery/jocko\n    $ make build\n    ```\n\n    (If you see an error about `dep` not being found, ensure that\n    `$GOPATH/bin` is in your `PATH`)\n\n### Docker\n\n`docker build -t travisjeffery/jocko:latest .`\n\n## Contributing\n\nSee [CONTRIBUTING](CONTRIBUTING.md) for details on submitting patches and the contribution workflow.\n\n## License\n\nJocko is under the MIT license, see the [LICENSE](LICENSE) file for details.\n\n---\n\n- [travisjeffery.com](http://travisjeffery.com)\n- GitHub [@travisjeffery](https://github.com/travisjeffery)\n- Twitter [@travisjeffery](https://twitter.com/travisjeffery)\n\n- Medium [@travisjeffery](https://medium.com/@travisjeffery)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftravisjeffery%2Fjocko","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftravisjeffery%2Fjocko","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftravisjeffery%2Fjocko/lists"}