{"id":30508335,"url":"https://github.com/codesmell/kafkapubcli","last_synced_at":"2025-08-25T22:35:35.373Z","repository":{"id":309616401,"uuid":"663496936","full_name":"CodeSmell/KafkaPubCLI","owner":"CodeSmell","description":"CLI to monitor a directory and publish files to Kafka","archived":false,"fork":false,"pushed_at":"2025-08-12T21:57:46.000Z","size":11,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-12T22:30:57.815Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CodeSmell.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-07-07T12:34:08.000Z","updated_at":"2025-08-12T21:57:50.000Z","dependencies_parsed_at":"2025-08-12T22:31:00.377Z","dependency_job_id":"625c1ed0-917f-4948-bc57-2cd63d3184a7","html_url":"https://github.com/CodeSmell/KafkaPubCLI","commit_stats":null,"previous_names":["codesmell/kafkapubcli"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/CodeSmell/KafkaPubCLI","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSmell%2FKafkaPubCLI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSmell%2FKafkaPubCLI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSmell%2FKafkaPubCLI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSmell%2FKafkaPubCLI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CodeSmell","download_url":"https://codeload.github.com/CodeSmell/KafkaPubCLI/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CodeSmell%2FKafkaPubCLI/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272144112,"owners_count":24881139,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-25T02:00:12.092Z","response_time":1107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-25T22:35:33.507Z","updated_at":"2025-08-25T22:35:35.359Z","avatar_url":"https://github.com/CodeSmell.png","language":"Java","readme":"# Kafka Util\n\nThis utility will monitor a directory. It will publish each file in the specified directory to a Kafka topic.\nThis CLI utility was built to enable easy testing of products that consume from Kafka topics. \n\n\n## Format for the Files\nThe utility does not expect any format when publishing only a body on a Kafka message. \nThe entire contents of the file will be published as the body of the message.\nIt will not treat the EOL in any special way so this allows the content (like JSON) to be pretty printed and still be sent in its entirety. \n\nFor example a JSON that is pretty printed. \n\n```\n{\n\t\"foo\": \"bar\",\n\t\"code\": \"smell\"\n}\n```\n\nThis allows easy scanning and editing the file when testing.\n\n### Adding a Key and Headers to the Kafka message\n\nHowever, if a key and headers are needed in the Kafka message the following format can be used:\n\nAt the top of the file add the `--key` delimiter.\nEvery line above that delimiter will be expected to be the value of the key. \nIt is also expected that it will be one line and always appear first if it is needed.\n\nAt the top of the file (but below the `--key` delimiter if there is one) add the `--header` delimiter.\nEvery line above that delimiter will be expected to be a key value pair separated by a colon (:).\nAll of the content below that delimiter will be considered the body of the Kafka message.\n\n```\nfoo\n--key\nhello:world\nghost:buster\n--header\nfoobar\n```\n\nThe contents above will result in a single Kafka message.\nThe body will be `foobar` and there will be two headers. \nThe key for one of the headers will be `hello` and the value will be `world`. \nThe key for the other header will be `ghost` and the value will be `buster`. \nThe key for the Kafka payload will be `foo`.\n\n## Build the Uber JAR\n\n```\nmvn clean install spring-boot:repackage\n```\n\n### Running the Util\nThe default mode is to continually poll the directory (`messageLocation`) for files that should be published to Kafka. Once a file is published to the Kafka topic it will be deleted. \n\n``` \njava -jar target/kafka-utils-0.0.1-SNAPSHOT.jar \\\n                            -topic myTopic \\\n                            -bootstrap-server localhost:9092 \\\n                            -acks 1 \\\n                            -messageLocation /dev/myKafkaFiles\n```\n\nIf the user only wants to run the utility against the directory once then add the parameter (`runOnce`). \nIf the user doesn't want to remove the file then add the parameter (`noDeleteFiles`).\n\n``` \njava -jar target/kafka-utils-0.0.1-SNAPSHOT.jar \\\n                            -topic myTopic \\\n                            -bootstrap-server localhost:9092 \\\n                            -acks 1 \\\n                            -messageLocation /dev/myKafkaFiles \\\n                            -runOnce \\\n                            -noDeleteFiles\n```\n\n\n### Running the Util in secure mode\n\n``` \njava -jar target/kafka-utils-0.0.1-SNAPSHOT.jar \\\n                            -topic myTopic \\\n                            -bootstrap-server localhost:443 \\\n                            -acks 1 \\\n                            -messageLocation /dev/myKafkaFiles \\\n                            -isSecure \\\n                            -trustStoreType JKS \\\n                            -trustStoreLocation /dev/truststore.jks \\\n                            -trustStorePassword password \\                            \n                            -securityProtocol SASL_SSL \\\n                            -saslMechanism SCRAM-SHA-512 \\\n                            -saslJaasConfig 'org.apache.kafka.common.security.scram.ScramLoginModule required \\\n                            \tusername=\"user\" \\\n                            \tpassword=\"password\";'\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodesmell%2Fkafkapubcli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodesmell%2Fkafkapubcli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodesmell%2Fkafkapubcli/lists"}