{"id":13697608,"url":"https://github.com/fgeller/kt","last_synced_at":"2025-06-12T10:11:49.284Z","repository":{"id":40693623,"uuid":"50273796","full_name":"fgeller/kt","owner":"fgeller","description":"Kafka command line tool that likes JSON","archived":false,"fork":false,"pushed_at":"2024-04-08T10:08:12.000Z","size":2297,"stargazers_count":950,"open_issues_count":11,"forks_count":102,"subscribers_count":19,"default_branch":"main","last_synced_at":"2025-05-03T20:33:50.343Z","etag":null,"topics":["apache-kafka","cli","golang","json","kafka"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fgeller.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-01-24T05:33:29.000Z","updated_at":"2025-04-08T10:14:25.000Z","dependencies_parsed_at":"2022-07-14T06:30:30.716Z","dependency_job_id":"75fdabd3-29fb-48ab-bea1-cacb8ddca876","html_url":"https://github.com/fgeller/kt","commit_stats":{"total_commits":322,"total_committers":27,"mean_commits":"11.925925925925926","dds":0.2142857142857143,"last_synced_commit":"6008ad3fa73000c07f6549c1d7743175c9480e46"},"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"purl":"pkg:github/fgeller/kt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fgeller%2Fkt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fgeller%2Fkt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fgeller%2Fkt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fgeller%2Fkt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fgeller","download_url":"https://codeload.github.com/fgeller/kt/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fgeller%2Fkt/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259444983,"owners_count":22858553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-kafka","cli","golang","json","kafka"],"created_at":"2024-08-02T18:01:00.721Z","updated_at":"2025-06-12T10:11:49.260Z","avatar_url":"https://github.com/fgeller.png","language":"Go","readme":"# kt - a Kafka tool that likes JSON [![Continuous Integration](https://github.com/fgeller/kt/actions/workflows/go.yml/badge.svg)](https://github.com/fgeller/kt/actions/workflows/go.yml)\n\nSome reasons why you might be interested:\n\n* Consume messages on specific partitions between specific offsets.\n* Display topic information (e.g., with partition offset and leader info).\n* Modify consumer group offsets (e.g., resetting or manually setting offsets per topic and per partition).\n* JSON output for easy consumption with tools like [kp](https://github.com/echojc/kp) or [jq](https://stedolan.github.io/jq/).\n* JSON input to facilitate automation via tools like [jsonify](https://github.com/fgeller/jsonify).\n* Configure brokers, topic, Kafka version and authentication via environment variables `KT_BROKERS`, `KT_TOPIC`, `KT_KAFKA_VERSION` and `KT_AUTH`.\n* Fast start up time.\n* No buffering of output.\n* Binary keys and payloads can be passed and presented in base64 or hex encoding.\n* Support for TLS authentication.\n* Basic cluster admin functions: Create \u0026 delete topics.\n\n\u003e [!NOTE]\n\u003e I'm not using kt actively myself anymore, so if you think it's lacking some feature - please let me know by creating an issue.\n\n## Examples\n\n\u003cdetails\u003e\u003csummary\u003eRead details about topics that match a regex\u003c/summary\u003e\n\n```sh\n$ kt topic -filter news -partitions\n{\n  \"name\": \"actor-news\",\n  \"partitions\": [\n    {\n      \"id\": 0,\n      \"oldest\": 0,\n      \"newest\": 0\n    }\n  ]\n}\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eProduce messages\u003c/summary\u003e\n\n```sh\n$ echo 'Alice wins Oscar' | kt produce -topic actor-news -literal\n{\n  \"count\": 1,\n  \"partition\": 0,\n  \"startOffset\": 0\n}\n$ echo 'Bob wins Oscar' | kt produce -topic actor-news -literal\n{\n  \"count\": 1,\n  \"partition\": 0,\n  \"startOffset\": 0\n}\n$ for i in {6..9} ; do echo Bourne sequel $i in production. | kt produce -topic actor-news -literal ;done\n{\n  \"count\": 1,\n  \"partition\": 0,\n  \"startOffset\": 1\n}\n{\n  \"count\": 1,\n  \"partition\": 0,\n  \"startOffset\": 2\n}\n{\n  \"count\": 1,\n  \"partition\": 0,\n  \"startOffset\": 3\n}\n{\n  \"count\": 1,\n  \"partition\": 0,\n  \"startOffset\": 4\n}\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eOr pass in JSON object to control key, value and partition\u003c/summary\u003e\n\n```sh\n$ echo '{\"value\": \"Terminator terminated\", \"key\": \"Arni\", \"partition\": 0}' | kt produce -topic actor-news\n{\n  \"count\": 1,\n  \"partition\": 0,\n  \"startOffset\": 5\n}\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eRead messages at specific offsets on specific partitions\u003c/summary\u003e\n\n```sh\n$ kt consume -topic actor-news -offsets 0=1:2\n{\n  \"partition\": 0,\n  \"offset\": 1,\n  \"key\": \"\",\n  \"value\": \"Bourne sequel 6 in production.\",\n  \"timestamp\": \"1970-01-01T00:59:59.999+01:00\"\n}\n{\n  \"partition\": 0,\n  \"offset\": 2,\n  \"key\": \"\",\n  \"value\": \"Bourne sequel 7 in production.\",\n  \"timestamp\": \"1970-01-01T00:59:59.999+01:00\"\n}\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eFollow a topic, starting relative to newest offset\u003c/summary\u003e\n\n```sh\n$ kt consume -topic actor-news -offsets all=newest-1:\n{\n  \"partition\": 0,\n  \"offset\": 4,\n  \"key\": \"\",\n  \"value\": \"Bourne sequel 9 in production.\",\n  \"timestamp\": \"1970-01-01T00:59:59.999+01:00\"\n}\n{\n  \"partition\": 0,\n  \"offset\": 5,\n  \"key\": \"Arni\",\n  \"value\": \"Terminator terminated\",\n  \"timestamp\": \"1970-01-01T00:59:59.999+01:00\"\n}\n^Creceived interrupt - shutting down\nshutting down partition consumer for partition 0\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eView offsets for a given consumer group\u003c/summary\u003e\n\n```sh\n$ kt group -group enews -topic actor-news -partitions 0\nfound 1 groups\nfound 1 topics\n{\n  \"name\": \"enews\",\n  \"topic\": \"actor-news\",\n  \"offsets\": [\n    {\n      \"partition\": 0,\n      \"offset\": 6,\n      \"lag\": 0\n    }\n  ]\n}\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eChange consumer group offset\u003c/summary\u003e\n\n```sh\n$ kt group -group enews -topic actor-news -partitions 0 -reset 1\nfound 1 groups\nfound 1 topics\n{\n  \"name\": \"enews\",\n  \"topic\": \"actor-news\",\n  \"offsets\": [\n    {\n      \"partition\": 0,\n      \"offset\": 1,\n      \"lag\": 5\n    }\n  ]\n}\n$ kt group -group enews -topic actor-news -partitions 0\nfound 1 groups\nfound 1 topics\n{\n  \"name\": \"enews\",\n  \"topic\": \"actor-news\",\n  \"offsets\": [\n    {\n      \"partition\": 0,\n      \"offset\": 1,\n      \"lag\": 5\n    }\n  ]\n}\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eCreate and delete a topic\u003c/summary\u003e\n\n```sh\n$ kt admin -createtopic morenews -topicdetail \u003c(jsonify =NumPartitions 1 =ReplicationFactor 1)\n$ kt topic -filter news\n{\n  \"name\": \"morenews\"\n}\n$ kt admin -deletetopic morenews\n$ kt topic -filter news\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eChange broker address via environment variable\u003c/summary\u003e\n\n```sh\n$ export KT_BROKERS=brokers.kafka:9092\n$ kt \u003ccommand\u003e \u003coption\u003e\n```\n\n\u003c/details\u003e\n\n## Installation\n\nYou can download kt via the [Releases](https://github.com/fgeller/kt/releases) section.\n\nAlternatively, the usual way via the go tool, for example:\n\n    $ go install github.com/fgeller/kt/v14@latest\n\nOr via Homebrew on OSX:\n\n    $ brew tap fgeller/tap\n    $ brew install kt\n\n### Docker\n\n[@Paxa](https://github.com/Paxa) maintains an image to run kt in a Docker environment - thanks!\n\nFor more information: [https://github.com/Paxa/kt](https://github.com/Paxa/kt)\n\n## Usage:\n\n    $ kt -help\n    kt is a tool for Kafka.\n\n    Usage:\n\n            kt command [arguments]\n\n    The commands are:\n\n            consume        consume messages.\n            produce        produce messages.\n            topic          topic information.\n            group          consumer group information and modification.\n            admin          basic cluster administration.\n\n    Use \"kt [command] -help\" for for information about the command.\n\n    Authentication:\n\n    Authentication with Kafka can be configured via a JSON file.\n    You can set the file name via an \"-auth\" flag to each command or\n    set it via the environment variable KT_AUTH.\n\n## Authentication / Encryption\n\nAuthentication configuration is possibly via a JSON file. You indicate the mode\nof authentication you need and provide additional information as required for\nyour mode. You pass the path to your configuration file via the `-auth` flag to\neach command individually, or set it via the environment variable `KT_AUTH`.\n\n### TLS\n\nRequired fields:\n\n - `mode`: This needs to be set to `TLS`\n - `client-certificate`: Path to your certificate\n - `client-certificate-key`: Path to your certificate key\n - `ca-certificate`: Path to your CA certificate\n\nExample for an authorization configuration that is used for the system tests:\n\n\n    {\n        \"mode\": \"TLS\",\n        \"client-certificate\": \"test-secrets/kt-test.crt\",\n        \"client-certificate-key\": \"test-secrets/kt-test.key\",\n        \"ca-certificate\": \"test-secrets/snakeoil-ca-1.crt\"\n    }\n\nIf any certificate or key path is simply the name of the file, it is assumed to\nbe in the same directory as the auth file itself. For example if the path to the\nauth file is `/some/dir/kt-auth.json` then a `\"client-certificate\":\n\"kt-test.crt\"` will be qualified to `/some/dir/kt-test.crt`.\n\n### TLS one-way\n\nRequired fields:\n\n - `mode`: This needs to be set to `TLS-1way`\n\nOptional fields:\n\n - `ca-certificate`: Path to your CA certificate\n\n\nExample:\n\n\n    {\n        \"mode\": \"TLS-1way\"\n    }\n\n### Other modes\n\nPlease create an\n[issue](https://github.com/fgeller/kt/issues/new) with details for the mode that you need.\n\n","funding_links":[],"categories":["Libraries","Go","Operations"],"sub_categories":["Kafka","Tools"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffgeller%2Fkt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffgeller%2Fkt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffgeller%2Fkt/lists"}