{"id":13783930,"url":"https://github.com/rayokota/keta","last_synced_at":"2025-04-14T03:14:45.500Z","repository":{"id":38123331,"uuid":"302472397","full_name":"rayokota/keta","owner":"rayokota","description":"A Transactional Metadata Store Backed by Apache Kafka","archived":false,"fork":false,"pushed_at":"2025-04-13T06:59:35.000Z","size":1269,"stargazers_count":23,"open_issues_count":8,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-14T03:14:31.360Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rayokota.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-10-08T22:07:02.000Z","updated_at":"2025-04-13T06:59:38.000Z","dependencies_parsed_at":"2023-10-11T06:52:44.936Z","dependency_job_id":"e4e90043-f1ae-45ac-89a5-eff0db901d29","html_url":"https://github.com/rayokota/keta","commit_stats":{"total_commits":557,"total_committers":3,"mean_commits":"185.66666666666666","dds":0.4290843806104129,"last_synced_commit":"792cc82232483170b702311701f03d76a0423b13"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rayokota%2Fketa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rayokota%2Fketa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rayokota%2Fketa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rayokota%2Fketa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rayokota","download_url":"https://codeload.github.com/rayokota/keta/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248813803,"owners_count":21165634,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T19:00:33.236Z","updated_at":"2025-04-14T03:14:45.480Z","avatar_url":"https://github.com/rayokota.png","language":"Java","funding_links":[],"categories":["Projects built on top of Kafka","\u003ca name=\"Java\"\u003e\u003c/a\u003eJava"],"sub_categories":["Integrations"],"readme":"# Keta - A Transactional Metadata Store Backed by Apache Kafka\n\n[![Build Status][github-actions-shield]][github-actions-link]\n[![Maven][maven-shield]][maven-link]\n[![Javadoc][javadoc-shield]][javadoc-link]\n\n[github-actions-shield]: https://github.com/rayokota/keta/workflows/build/badge.svg?branch=master\n[github-actions-link]: https://github.com/rayokota/keta/actions\n[maven-shield]: https://img.shields.io/maven-central/v/io.kcache/keta-core.svg\n[maven-link]: https://search.maven.org/#search%7Cga%7C1%7Cketa-core\n[javadoc-shield]: https://javadoc.io/badge/io.kcache/keta-core.svg?color=blue\n[javadoc-link]: https://javadoc.io/doc/io.kcache/keta-core\n\nKeta is a transactional metadata store backed by Apache Kafka.\n\n## Maven\n\nReleases of Keta are deployed to Maven Central.\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003eio.kcache\u003c/groupId\u003e\n    \u003cartifactId\u003eketa-core\u003c/artifactId\u003e\n    \u003cversion\u003e1.0.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## Getting Started\n\nTo run Keta, download a [release](https://github.com/rayokota/keta/releases), unpack it, and then modify `config/keta.properties` to point to an existing Kafka broker.  Then run the following:\n\n```bash\n$ bin/keta-start config/keta.properties\n```\n\nKeta can be used with any client that supports the etcd v3 APIs.  To use Keta with the `etcdctl` command line client, first download etcd [here](https://github.com/etcd-io/etcd).  Then at a separate terminal, start `etcdctl`.\n\n```\n$ etcdctl put mykey \"this is awesome\"\n$ etcdctl get mykey\n```\n\nThe etcd APIs have a concise way for expressioning transactions.\n\n```\n$ etcdctl put user1 bad\n$ etcdctl txn --interactive\n\ncompares:\nvalue(\"user1\") = \"bad\"      \n\nsuccess requests (get, put, delete):\ndel user1  \n\nfailure requests (get, put, delete):\nput user1 good\n```\n\nTo expire key-values, use a lease.\n\n```\n$ etcdctl lease grant 300\n# lease 2be7547fbc6a5afa granted with TTL(300s)\n\n$ etcdctl put sample value --lease=2be7547fbc6a5afa\n$ etcdctl get sample\n\n$ etcdctl lease keep-alive 2be7547fbc6a5afa\n$ etcdctl lease revoke 2be7547fbc6a5afa\n# or after 300 seconds\n$ etcdctl get sample\n```\n\nTo receive change notifications, use a watch.\n\n```\n$ etcdctl watch stock --prefix\n```\n\nThen at a separate terminal, enter the following:\n\n```\n$ etcdctl put stock1 10\n$ etcdctl put stock2 20\n```\n\nIf multiple Keta servers are configured with the same cluster group ID (see [Basic Configuration](#basic-configuration)), then they will form a cluster and one of them will be elected as leader, while the others will become followers (replicas).  If a follower receives a request, it will be forwarded to the leader.  If the leader fails, one of the followers will be elected as the new leader.\n\n\n\n## Basic Configuration\n\nKeta has a number of configuration properties that can be specified.  \n\n- `listeners` - List of listener URLs that include the scheme, host, and port.  Defaults to `http://0.0.0.0:2379`.  \n- `cluster.group.id` - The group ID to be used for leader election.  Defaults to `keta`.\n- `leader.eligibility` - Whether this node can participate in leader election.  Defaults to true.\n- `kafkacache.backing.cache` - The backing cache for KCache, one of `memory` (default), `bdbje`, `lmdb`, `mapdb`, or `rocksdb`.\n- `kafkacache.data.dir` - The root directory for backing cache storage.  Defaults to `/tmp`.\n- `kafkacache.bootstrap.servers` - A list of host and port pairs to use for establishing the initial connection to Kafka.\n- `kafkacache.group.id` - The group ID to use for the internal consumers, which needs to be unique for each node.  Defaults to `keta-1`.\n- `kafkacache.topic.replication.factor` - The replication factor for the internal topics created by Keta.  Defaults to 3.\n- `kafkacache.init.timeout.ms` - The timeout for initialization of the Kafka cache, including creation of internal topics.  Defaults to 300 seconds.\n- `kafkacache.timeout.ms` - The timeout for an operation on the Kafka cache.  Defaults to 60 seconds.\n\n## Security\n\n### HTTPS\n\nTo use HTTPS, first configure the `listeners` with an `https` prefix, then specify the following properties with the appropriate values.\n\n```\nssl.keystore.location=/var/private/ssl/custom.keystore\nssl.keystore.password=changeme\nssl.key.password=changeme\nssl.truststore.location=/var/private/ssl/custom.truststore\nssl.truststore.password=changeme\n```\n\n\n### Authentication and Role-Based Access Control\n\nKeta supports the same authentication and role-based access control (RBAC) APIs as etcd.  For more info, see the etcd documentation [here](https://etcd.io/docs/v3.4.0/op-guide/authentication/).\n\n\n### Kafka Authentication\n\nAuthentication to a secure Kafka cluster is described [here](https://github.com/rayokota/kcache#security).\n \n## Implementation Notes\n\n\nKeta uses seven topics to hold metadata:\n\n- `_keta_kv` - A topic that holds the key-values.\n- `_keta_leases` - A topic that holds leases.\n- `_keta_auth` - A topic that holds auth configuration.\n- `_keta_auth_users` - A topic that holds auth users.\n- `_keta_auth_roles` - A topic that holds auth roles.\n- `_keta_commits` - A topic that holds the list of committed transactions.\n- `_keta_timestamps` - A topic that stores the maximum timestamp that the transaction manager is allowed to return to clients.\n\nFor more info on Keta, see this [blog post](https://yokota.blog/2020/11/09/keta-a-metadata-store-backed-by-apache-kafka/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frayokota%2Fketa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frayokota%2Fketa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frayokota%2Fketa/lists"}