{"id":13697652,"url":"https://github.com/dearcode/kafkabeat","last_synced_at":"2026-01-19T17:07:36.302Z","repository":{"id":57601191,"uuid":"88141733","full_name":"dearcode/kafkabeat","owner":"dearcode","description":"beats for kafka","archived":false,"fork":false,"pushed_at":"2017-09-12T08:10:21.000Z","size":8537,"stargazers_count":6,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-03T20:34:02.758Z","etag":null,"topics":["beats","elastic","elasticsearch","kafka"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dearcode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-13T08:19:16.000Z","updated_at":"2018-05-21T23:19:31.000Z","dependencies_parsed_at":"2022-08-25T17:53:26.313Z","dependency_job_id":null,"html_url":"https://github.com/dearcode/kafkabeat","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dearcode/kafkabeat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dearcode%2Fkafkabeat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dearcode%2Fkafkabeat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dearcode%2Fkafkabeat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dearcode%2Fkafkabeat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dearcode","download_url":"https://codeload.github.com/dearcode/kafkabeat/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dearcode%2Fkafkabeat/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28577228,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-19T16:29:19.148Z","status":"ssl_error","status_checked_at":"2026-01-19T16:29:17.772Z","response_time":67,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beats","elastic","elasticsearch","kafka"],"created_at":"2024-08-02T18:01:01.197Z","updated_at":"2026-01-19T17:07:36.270Z","avatar_url":"https://github.com/dearcode.png","language":"Go","funding_links":[],"categories":["Libraries"],"sub_categories":["Kafka"],"readme":"# Kafkabeat\n\nWelcome to Kafkabeat. Kafkabeat is a beat that allows you to read data from kafka and index the results into elasticsearch.  \n\nEnsure that this folder is at the following location:\n`${GOPATH}/github.com/dearcode`\n\n## Getting Started with Kafkabeat\n\n### Requirements\n\n* [Golang](https://golang.org/dl/) 1.7\n\n### Init Project\nTo get running with Kafkabeat and also install the\ndependencies, run the following command:\n\n```\nmake setup\n```\n\nIt will create a clean git history for each major step. Note that you can always rewrite the history if you wish before pushing your changes.\n\nTo push Kafkabeat in the git repository, run the following commands:\n\n```\ngit remote set-url origin https://github.com/dearcode/kafkabeat\ngit push origin master\n```\n\nFor further development, check out the [beat developer guide](https://www.elastic.co/guide/en/beats/libbeat/current/new-beat.html).\n\n### Build\n\nTo build the binary for Kafkabeat run the command below. This will generate a binary\nin the same directory with the name kafkabeat.\n\n```\nmake\n```\n\n\n### Run\n\nTo run Kafkabeat with debugging output enabled, run:\n\n```\n./kafkabeat -c kafkabeat.yml -e -d \"*\"\n```  \n\n### Configuring  \n```\nkafkabeat:\n  # brokers kafka brokers.\n  brokers: \"localhost:9092\"\n  # topics kafka topics.\n  topics: \"topic1,topic2\"\n  # group kafka consumer group.\n  group: \"kafkabeat\"\n\n```\n\n### Test\n\nTo test Kafkabeat, run the following command:\n\n```\nmake testsuite\n```\n\nalternatively:\n```\nmake unit-tests\nmake system-tests\nmake integration-tests\nmake coverage-report\n```\n\nThe test coverage is reported in the folder `./build/coverage/`\n\n### Update\n\nEach beat has a template for the mapping in elasticsearch and a documentation for the fields\nwhich is automatically generated based on `etc/fields.yml`.\nTo generate etc/kafkabeat.template.json and etc/kafkabeat.asciidoc\n\n```\nmake update\n```\n\n\n### Cleanup\n\nTo clean  Kafkabeat source code, run the following commands:\n\n```\nmake fmt\nmake simplify\n```\n\nTo clean up the build directory and generated artifacts, run:\n\n```\nmake clean\n```\n\n\n### Clone\n\nTo clone Kafkabeat from the git repository, run the following commands:\n\n```\nmkdir -p ${GOPATH}/github.com/dearcode\ncd ${GOPATH}/github.com/dearcode\ngit clone https://github.com/dearcode/kafkabeat\n```\n\n\nFor further development, check out the [beat developer guide](https://www.elastic.co/guide/en/beats/libbeat/current/new-beat.html).\n\n\n## Packaging\n\nThe beat frameworks provides tools to crosscompile and package your beat for different platforms. This requires [docker](https://www.docker.com/) and vendoring as described above. To build packages of your beat, run the following command:\n\n```\nmake package\n```\n\nThis will fetch and create all images required for the build process. The hole process to finish can take several minutes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdearcode%2Fkafkabeat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdearcode%2Fkafkabeat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdearcode%2Fkafkabeat/lists"}