{"id":30764772,"url":"https://github.com/airbnb/kafka-statsd-metrics2","last_synced_at":"2025-09-04T17:14:01.247Z","repository":{"id":19223653,"uuid":"22457986","full_name":"airbnb/kafka-statsd-metrics2","owner":"airbnb","description":"Send Kafka Metrics to StatsD.","archived":false,"fork":false,"pushed_at":"2021-06-03T04:37:01.000Z","size":205,"stargazers_count":135,"open_issues_count":22,"forks_count":48,"subscribers_count":223,"default_branch":"master","last_synced_at":"2024-05-21T03:15:26.595Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/airbnb.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-07-31T05:43:16.000Z","updated_at":"2024-04-02T09:12:56.000Z","dependencies_parsed_at":"2022-08-01T03:08:43.774Z","dependency_job_id":null,"html_url":"https://github.com/airbnb/kafka-statsd-metrics2","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/airbnb/kafka-statsd-metrics2","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbnb%2Fkafka-statsd-metrics2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbnb%2Fkafka-statsd-metrics2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbnb%2Fkafka-statsd-metrics2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbnb%2Fkafka-statsd-metrics2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/airbnb","download_url":"https://codeload.github.com/airbnb/kafka-statsd-metrics2/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airbnb%2Fkafka-statsd-metrics2/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273641916,"owners_count":25142255,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-04T02:00:08.968Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-04T17:13:57.032Z","updated_at":"2025-09-04T17:14:01.229Z","avatar_url":"https://github.com/airbnb.png","language":"Java","funding_links":[],"categories":["指标库"],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/airbnb/kafka-statsd-metrics2.png?branch=master)](https://travis-ci.org/airbnb/kafka-statsd-metrics2)\n\n# kafka-statsd-metrics2\n\nSend Kafka Metrics to StatsD.\n\n## Contact\n**Let us know!** If you fork this, or if you use it, or if it helps in anyway, we'd love to hear from you! opensource@airbnb.com\n\n## What is it about?\nKafka uses [Yammer Metrics](http://metrics.codahale.com/getting-started/) (now part of the [Dropwizard project](http://metrics.codahale.com/about/)) for [metrics reporting](https://kafka.apache.org/documentation.html#monitoring)\nin both the server and the client.\nThis can be configured to report stats using pluggable stats reporters to hook up to your monitoring system.\n\nThis project provides a simple integration between Kafka and a StatsD reporter for Metrics.\n\nMetrics can be filtered based on the metric name and the metric dimensions (min, max, percentiles, etc).\n\n## Supported Kafka versions\n\n- For Kafka `0.9.0.0` or later use `kafka-statsd-metrics2-0.5.0`\n- For Kafka `0.8.2.0` or later use `kafka-statsd-metrics2-0.4.0`\n- For Kafka `0.8.1.1` or prior use `kafka-statsd-metrics2-0.3.0`\n\n\n## Releases\n\n### 0.5.4\n- Fix metrics with different tags not reported properly\n\n### 0.5.2 / 0.5.3\n- Convert INFINITY values to 0.\n\n### 0.5.1\n- Fix metrics change log level\n\n### 0.5.0\n\n - `0.5.0` add support to report new producer/consumer metrics in kafka-0.9\n - Compatible with Kafka 0.8\n - A complete list of all the metrics supported in the metrics reporter can be found [here](http://docs.confluent.io/2.0.1/kafka/monitoring.html)\n\n\n### 0.4.0\n\n - `0.4.0` adds support for tags on metrics. See [dogstatsd extensions](http://docs.datadoghq.com/guides/dogstatsd/#tags). If your statsd server does not support tags, you can disable them in the Kafka configuration. See property `external.kafka.statsd.tag.enabled` below.\n\n - The statsd client is [`com.indeed:java-dogstatsd-client:2.0.11`](https://github.com/indeedeng/java-dogstatsd-client/tree/java-dogstatsd-client-2.0.11).\n - support new `MetricNames` introduced by kafka 0.8.2.x\n\n### 0.3.0\n\n- initial release\n\n## How to install?\n\n- [Download](https://bintray.com/airbnb/jars/kafka-statsd-metrics2/view) or build the shadow jar for `kafka-statsd-metrics`.\n- Install the jar in Kafka classpath, typically `./kafka_2.11-0.9.0.1/libs/`\n- In the Kafka config file, `server.properties`, add the following properties. Default values are in parenthesis.\n\n## How to use metrics in Kafka 0.9 / 0.8?\n### New metrics in kafka 0.9\n\n1. Add `metric.reporters` in producer.properties or consumer.properties \n```bash\n    # declare the reporter if new producer/consumer is used\n    metric.reporters=com.airbnb.kafka.kafka09.StatsdMetricsReporter\n```\n2. Run new-producer or new-consumer\n\nProducer:\n```bash\n    bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test --producer.config config/producer.properties\n    bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --new-consumer --zookeeper localhost:2181 --topic test -from-beginning --consumer.config config/consumer.properties    \n```\n\n### Old metrics in kafka 0.8\n\n1. Add `kafka.metrics.reporters` in producer.properties or consumer.properties \n```bash\n    # declare the reporter if old producer/consumer is used\n    kafka.metrics.reporters=com.airbnb.kafka.kafka08.StatsdMetricsReporter\n```\n2. Run old-producer or old-consumer\n\n```bash\n    bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test --producer.config config/producer.properties --old-producer\n    bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --zookeeper localhost:2181 --topic test -from-beginning --consumer.config config/consumer.properties\n```\n\n2. Run old-consumer\n\n\n### Configurations\n```bash\n    # declare the reporter if new producer/consumer is used\n    metric.reporters=com.airbnb.kafka.StatsdMetricsReporter\n\n    # declare the reporter if old producer/consumer is used\n    kafka.metrics.reporters=com.airbnb.kafka.kafka08.StatsdMetricsReporter\n\n    # enable the reporter, (false)\n    external.kafka.statsd.reporter.enabled=true\n\n    # the host of the StatsD server (localhost)\n    external.kafka.statsd.host=localhost\n\n    # the port of the StatsD server (8125)\n    external.kafka.statsd.port=8125\n\n    # enable the support of statsd tag extension, e.g. datadog statsd (true)\n    external.kafka.statsd.tag.enabled=true\n\n    # a prefix for all metrics names (empty)\n    external.kafka.statsd.metrics.prefix=\n\n    # note that the StatsD reporter follows the global polling interval (10)\n    # kafka.metrics.polling.interval.secs=10\n\n\n\n    # A regex to exclude some metrics\n    # Default is: (kafka\\.consumer\\.FetchRequestAndResponseMetrics.*)|(.*ReplicaFetcherThread.*)|(kafka\\.server\\.FetcherLagMetrics\\..*)|(kafka\\.log\\.Log\\..*)|(kafka\\.cluster\\.Partition\\..*)\n    #\n    # The metric name is formatted with this template: group.type.scope.name\n    #\n    # external.kafka.statsd.metrics.exclude_regex=\n\n    #\n    # Each metric provides multiple dimensions: min, max, meanRate, etc\n    # This might be too much data.\n    # It is possible to disable some metric dimensions with the following properties:\n    # By default all dimenstions are enabled.\n    #\n    # external.kafka.statsd.dimension.enabled.count=true\n    # external.kafka.statsd.dimension.enabled.meanRate=true\n    # external.kafka.statsd.dimension.enabled.rate1m=true\n    # external.kafka.statsd.dimension.enabled.rate5m=true\n    # external.kafka.statsd.dimension.enabled.rate15m=true\n    # external.kafka.statsd.dimension.enabled.min=true\n    # external.kafka.statsd.dimension.enabled.max=true\n    # external.kafka.statsd.dimension.enabled.mean=true\n    # external.kafka.statsd.dimension.enabled.stddev=true\n    # external.kafka.statsd.dimension.enabled.median=true\n    # external.kafka.statsd.dimension.enabled.p75=true\n    # external.kafka.statsd.dimension.enabled.p95=true\n    # external.kafka.statsd.dimension.enabled.p98=true\n    # external.kafka.statsd.dimension.enabled.p99=true\n    # external.kafka.statsd.dimension.enabled.p999=true\n```\n\n- finally restart the Kafka server\n\n## How to test your configuration?\n\nYou can check your configuration in different ways:\n\n- During Kafka startup, the reporter class will be instantiated and initialized. The logs should contain a message similar to:\n`\"Kafka Statsd metrics reporter is enabled\"`\n- A JMX MBean named `kafka:type=com.airbnb.kafka.kafka08.StatsdMetricsReporter` should also exist.\n- Check the logs of your StatsD server\n- Finally, on the configured StatsD host, you could listen on the configured port and check for incoming data:\n\n```bash\n    # assuming the Statsd server has been stopped...\n    $ nc -ul 8125\n\n    kafka.controller.ControllerStats.LeaderElectionRateAndTimeMs.samples:1|gkafka.controller.ControllerStats\n    .LeaderElectionRateAndTimeMs.meanRate:0.05|gkafka.controller.ControllerStats.LeaderElectionRateAndTimeMs.\n    1MinuteRate:0.17|gkafka.controller.ControllerStats.LeaderElectionRateAndTimeMs.5MinuteRate:0.19|g....\n```\n\n## Sample file of metrics output from statsd\n[new-producer-metrics.txt](https://www.dropbox.com/s/p8e4vl5moa80ikp/new-producer-metrics.txt?dl=0)\n\n[new-consumer-metrics.txt](https://www.dropbox.com/s/ab3t8qis5p58l7f/new-consumer-metrics.txt?dl=0)\n\n\n## List of metrics for Kafka 0.8.2\n\nBelow are the metrics in Kafka 0.8.2\n\n```bash\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Metrics kind | Metric Name                                                                 | Metric Tags                                                                                                 |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.ReplicaManager.LeaderCount                                     |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.ReplicaManager.PartitionCount                                  |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.ReplicaManager.UnderReplicatedPartitions                       |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.controller.KafkaController.ActiveControllerCount                      |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.controller.KafkaController.OfflinePartitionsCount                     |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.controller.KafkaController.PreferredReplicaImbalanceCount             |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.network.RequestChannel.RequestQueueSize                               |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.ReplicaFetcherManager.Replica_MaxLag                           | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.ReplicaFetcherManager.Replica_MinFetchRate                     | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.FetchRequestPurgatory.PurgatorySize                            |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.FetchRequestPurgatory.NumDelayedRequests                       |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.ProducerRequestPurgatory.PurgatorySize                         |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.ProducerRequestPurgatory.NumDelayedRequests                    |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.consumer.ConsumerFetcherManager.MaxLag                                | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.consumer.ConsumerFetcherManager.MinFetchRate                          | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.consumer.ZookeeperConsumerConnector.FetchQueueSize                    | {\"clientId\" -\u003e config.clientId, \"topic\" -\u003e topic, \"threadId\" -\u003e thread}                                     |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.network.RequestChannel.ResponseQueueSize                              | {\"Processor\" -\u003e i}                                                                                          |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Timer        | kafka.log.LogFlushStats.LogFlushRateAndTimeMs                               |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.ReplicaManager.IsrExpandsPerSec                                |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.ReplicaManager.IsrShrinksPerSec                                |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.DelayedFetchRequestMetrics.FollowerExpiresPerSecond            |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.DelayedFetchRequestMetrics.ConsumerExpiresPerSecond            |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.controller.ControllerStats.UncleanLeaderElectionsPerSec               |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Timer        | kafka.controller.ControllerStats.LeaderElectionRateAndTimeMs                |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.producer.ProducerStats.SerializationErrorsPerSec                      | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.producer.ProducerStats.ResendsPerSec                                  | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.producer.ProducerStats.FailedSendsPerSec                              | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.producer.ProducerTopicMetrics.MessagesPerSec_all                      | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.producer.ProducerTopicMetrics.BytesPerSec_all                         | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.producer.ProducerTopicMetrics.DroppedMessagesPerSec_all               | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.producer.ProducerTopicMetrics.MessagesPerSec                          | {\"clientId\" -\u003e clientId, \"topic\" -\u003e topic}                                                                  |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.producer.ProducerTopicMetrics.BytesPerSec                             | {\"clientId\" -\u003e clientId, \"topic\" -\u003e topic}                                                                  |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.producer.ProducerTopicMetrics.DroppedMessagesPerSec                   | {\"clientId\" -\u003e clientId, \"topic\" -\u003e topic}                                                                  |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.FetcherStats.RequestsPerSec                                    | {\"clientId\" -\u003e metricId.clientId, \"brokerHost\" -\u003e metricId.brokerHost, \"brokerPort\" -\u003e metricId.brokerPort} |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.FetcherStats.BytesPerSec                                       | {\"clientId\" -\u003e metricId.clientId, \"brokerHost\" -\u003e metricId.brokerHost, \"brokerPort\" -\u003e metricId.brokerPort} |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.MessagesInPerSec_all                        |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.BytesInPerSec_all                           |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.BytesOutPerSec_all                          |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.LogBytesAppendedPerSec_all                  |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.FailedProduceRequestsPerSec_all             |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.FailedFetchRequestsPerSec_all               |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.MessagesInPerSec                            | {\"topic\" -\u003e topic}                                                                                          |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.BytesInPerSec                               | {\"topic\" -\u003e topic}                                                                                          |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.BytesOutPerSec                              | {\"topic\" -\u003e topic}                                                                                          |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.LogBytesAppendedPerSec                      | {\"topic\" -\u003e topic}                                                                                          |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.FailedProduceRequestsPerSec                 | {\"topic\" -\u003e topic}                                                                                          |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.BrokerTopicMetrics.FailedFetchRequestsPerSec                   | {\"topic\" -\u003e topic}                                                                                          |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.DelayedProducerRequestMetrics.ExpiresPerSecond_all             |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.DelayedProducerRequestMetrics.ExpiresPerSecond                 | {\"topic\" -\u003e topicAndPartition.topic, \"partition\" -\u003e topicAndPartition.partition}                            |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Timer        | kafka.producer.ProducerRequestMetrics.ProducerRequestRateAndTimeMs_all      | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.producer.ProducerRequestMetrics.ProducerRequestSize_all               | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Timer        | kafka.consumer.FetchRequestAndResponseMetrics.FetchRequestRateAndTimeMs_all | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.consumer.FetchRequestAndResponseMetrics.FetchResponseSize_all         | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Timer        | kafka.producer.ProducerRequestMetrics.ProducerRequestRateAndTimeMs          | {\"clientId\" -\u003e clientId, \"brokerHost\" -\u003e brokerHost, \"brokerPort\" -\u003e brokerPort}                            |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.producer.ProducerRequestMetrics.ProducerRequestSize                   | {\"clientId\" -\u003e clientId, \"brokerHost\" -\u003e brokerHost, \"brokerPort\" -\u003e brokerPort}                            |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Timer        | kafka.consumer.FetchRequestAndResponseMetrics.FetchRequestRateAndTimeMs     | {\"clientId\" -\u003e clientId, \"brokerHost\" -\u003e brokerHost, \"brokerPort\" -\u003e brokerPort}                            |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.consumer.FetchRequestAndResponseMetrics.FetchResponseSize             | {\"clientId\" -\u003e clientId, \"brokerHost\" -\u003e brokerHost, \"brokerPort\" -\u003e brokerPort}                            |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.consumer.ConsumerTopicMetrics.MessagesPerSec_all                      | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.consumer.ConsumerTopicMetrics.BytesPerSec_all                         | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.consumer.ConsumerTopicMetrics.MessagesPerSec                          | {\"clientId\" -\u003e clientId, \"topic\" -\u003e topic}                                                                  |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.consumer.ConsumerTopicMetrics.BytesPerSec                             | {\"clientId\" -\u003e clientId, \"topic\" -\u003e topic}                                                                  |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.network.RequestMetrics.RequestsPerSec                                 | {\"request\" -\u003e name}                                                                                         |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.network.RequestMetrics.RequestQueueTimeMs                             | {\"request\" -\u003e name}                                                                                         |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.network.RequestMetrics.LocalTimeMs                                    | {\"request\" -\u003e name}                                                                                         |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.network.RequestMetrics.RemoteTimeMs                                   | {\"request\" -\u003e name}                                                                                         |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.network.RequestMetrics.ResponseQueueTimeMs                            | {\"request\" -\u003e name}                                                                                         |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.network.RequestMetrics.ResponseSendTimeMs                             | {\"request\" -\u003e name}                                                                                         |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.network.RequestMetrics.TotalTimeMs                                    | {\"request\" -\u003e name}                                                                                         |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.FetcherLagMetrics.ConsumerLag                                  | {\"clientId\" -\u003e clientId, \"topic\" -\u003e topic, \"partition\" -\u003e partitionId}                                      |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.producer.async.ProducerSendThread.ProducerQueueSize                   | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.log.Log.NumLogSegments                                                | {\"topic\" -\u003e topic, \"partition\" -\u003e partition}                                                                |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.log.Log.LogEndOffset                                                  | {\"topic\" -\u003e topic, \"partition\" -\u003e partition}                                                                |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.cluster.Partition.UnderReplicated                                     | {\"topic\" -\u003e topic, \"partition\" -\u003e partitionId}                                                              |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.network.SocketServer.ResponsesBeingSent                               |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.network.SocketServer.NetworkProcessorAvgIdlePercent                   |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.network.SocketServer.IdlePercent                                      | {\"networkProcessor\" -\u003e i.toString}                                                                          |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.OffsetManager.NumOffsets                                       |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.OffsetManager.NumGroups                                        |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.consumer.ZookeeperConsumerConnector.OwnedPartitionsCount              | {\"clientId\" -\u003e config.clientId, \"groupId\" -\u003e config.groupId}                                                |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.consumer.ZookeeperConsumerConnector.OwnedPartitionsCount              | {\"clientId\" -\u003e config.clientId, \"groupId\" -\u003e config.groupId, \"topic\" -\u003e topic}                              |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.consumer.ZookeeperConsumerConnector.KafkaCommitsPerSec                | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.consumer.ZookeeperConsumerConnector.ZooKeeperCommitsPerSec            | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.consumer.ZookeeperConsumerConnector.RebalanceRateAndTime              | {\"clientId\" -\u003e clientId}                                                                                    |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.tools.DataChannel.MirrorMaker-DataChannel-WaitOnPut                   |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.tools.DataChannel.MirrorMaker-DataChannel-WaitOnTake                  |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Histogram    | kafka.tools.DataChannel.MirrorMaker-DataChannel-Size                        |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.common.AppInfo.Version                                                |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.server.KafkaRequestHandlerPool.RequestHandlerAvgIdlePercent           |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Meter        | kafka.util.Throttler.\"\"\"a input string not with small cardinality\"\"\"        |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.log.LogCleaner.max-buffer-utilization-percent                         |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.log.LogCleaner.cleaner-recopy-percent                                 |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.log.LogCleaner.max-clean-time-secs                                    |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Timer        | other.kafka.FetchThread.fetch-thread                                        |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Timer        | other.kafka.CommitThread.commit-thread                                      |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.log.Log.LogStartOffset                                                | {\"topic\" -\u003e topic, \"partition\" -\u003e partition}                                                                |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.log.Log.Size                                                          | {\"topic\" -\u003e topic, \"partition\" -\u003e partition)                                                                |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.server.KafkaServer.BrokerState                                        |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+\n    | Gauge        | kafka.log.LogCleanerManager.max-dirty-percent                               |                                                                                                             |\n    +--------------+-----------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+```\n```\n\n## Metrics-2.x vs Metrics-3.x\nThe metrics project has two main versions: v2 and v3. Version 3 is not backward compatible.\n\nVersion [0.8.1.1](https://github.com/apache/kafka/blob/0.8.1.1/build.gradle#L217) and [0.8.2.1](https://github.com/apache/kafka/blob/0.8.2.1/build.gradle#L209), Kafka depends on [metrics-2.2.0](http://mvnrepository.com/artifact/com.yammer.metrics/metrics-core/2.2.0).\n\n*Note:*\u003cbr/\u003e\nIn a future release, Kafka [might upgrade](https://issues.apache.org/jira/browse/KAFKA-960) to Metrics-3.x.\nDue to the incompatibilities between Metrics versions, a new Statsd reporter for metrics-3 will be required.\u003cbr/\u003e\nAll contributions welcome!\n\n\n## How to build\n\nAfter cloning the repo, type\n\n```bash\n    ./gradlew shadowJar\n```\n\nThis produces a jar file in `build/libs/`.\n\nThe shallow jar is a standalone jar.\n\n\n# License \u0026 Attributions\n\nThis project is released under the Apache License Version 2.0 (APLv2).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fairbnb%2Fkafka-statsd-metrics2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fairbnb%2Fkafka-statsd-metrics2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fairbnb%2Fkafka-statsd-metrics2/lists"}