{"id":14972051,"url":"https://github.com/nodefluent/prometheus-kafka-connect","last_synced_at":"2025-10-26T17:31:26.294Z","repository":{"id":42342804,"uuid":"106820329","full_name":"nodefluent/prometheus-kafka-connect","owner":"nodefluent","description":"Node.js kafka connect connector for prometheus","archived":false,"fork":false,"pushed_at":"2022-12-07T04:43:23.000Z","size":457,"stargazers_count":12,"open_issues_count":5,"forks_count":0,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-01-31T21:52:44.818Z","etag":null,"topics":["connector","grafana","kafka","metrics","nodejs","prometheus"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nodefluent.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-10-13T12:16:27.000Z","updated_at":"2024-05-27T18:34:15.000Z","dependencies_parsed_at":"2023-01-23T19:31:20.321Z","dependency_job_id":null,"html_url":"https://github.com/nodefluent/prometheus-kafka-connect","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodefluent%2Fprometheus-kafka-connect","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodefluent%2Fprometheus-kafka-connect/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodefluent%2Fprometheus-kafka-connect/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodefluent%2Fprometheus-kafka-connect/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nodefluent","download_url":"https://codeload.github.com/nodefluent/prometheus-kafka-connect/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238375212,"owners_count":19461569,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["connector","grafana","kafka","metrics","nodejs","prometheus"],"created_at":"2024-09-24T13:46:17.985Z","updated_at":"2025-10-26T17:31:20.971Z","avatar_url":"https://github.com/nodefluent.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# prometheus-kafka-connect\n\nConnector to prometheus client which only includes sink. This will be useful to stream data from Kafka to Prometheus and subsequently to Grafana. This connector depends on, [kafka-connect](https://github.com/nodefluent/kafka-connect) and [node-sinek](https://github.com/nodefluent/node-sinek) as the consumer. Please read up those dependencies to make things clearer.\n\nYou can use javascript consumer or native which depends on [librdkafka](https://github.com/edenhill/librdkafka). The example can be found in `example` directory, which there is a comment to specify where you need to do the transformation. Basically, you need to do that in ETL function, that needs to be passed to the connector. Or just like the following.\n\n```js\nconst { runSinkConnector, ConverterFactory } = require(\"prometheus-kafka-connect\");\nconst config = require(\"./config.js\");\n\nconsole.log(\"Waiting for message to be consumed...\");\n\nconst etl = (message, next) =\u003e {\n\n  // Do the transformation here\n  let record;\n  try {\n    record = {\n      metric: message.name\n      value: 1,\n      type: \"counter\" // or \"gauge\"\n    };\n  } catch(err) {\n    // Do nothing\n  }\n\n  if (record \u0026\u0026 record.metric \u0026\u0026 record.value) {\n    // Continue with the transformed record\n    return next(null, record);\n  }\n\n  // Continue without throwing error\n  return next();\n\n}\n\nconst converter = ConverterFactory.createSinkSchemaConverter(null,etl);\n\nrunSinkConnector(config, [converter], console.log.bind(console)).then(sink =\u003e {\n\n});\n\n```\n\n### Configuration\n* For the native and non-native consumer please take a look at [node-sinek](https://github.com/nodefluent/node-sinek) project.\n* There is an example of doing local setup complete with kafka, zookeeper, prometheus, and grafana in `local-setup` directory\n* Please take a look at `example/config.js` file, this is an excerpt from there.\n\n```js\n...\n    connector: {\n        options: {\n            job: \"promclient_job\",\n            additionalLabels: [\"method\"]\n        },\n    },\n...\n```\n#### job\nThis is to mark the job that needed to be consumed by prometheus, the configuration from prometheus is defined in the `prometheus.yml`, take a look at `local-setup` directory\n\n#### additionalLabels\nThe additionalLabels that are needed for the metric, the default is label, but you can add as many as you want in the array format.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodefluent%2Fprometheus-kafka-connect","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnodefluent%2Fprometheus-kafka-connect","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodefluent%2Fprometheus-kafka-connect/lists"}