{"id":13451787,"url":"https://github.com/linkedin/cruise-control","last_synced_at":"2026-05-27T01:00:54.337Z","repository":{"id":37602791,"uuid":"100402355","full_name":"linkedin/cruise-control","owner":"linkedin","description":"Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.","archived":false,"fork":false,"pushed_at":"2025-11-06T21:10:09.000Z","size":14317,"stargazers_count":3025,"open_issues_count":274,"forks_count":646,"subscribers_count":127,"default_branch":"main","last_synced_at":"2026-05-21T16:50:56.907Z","etag":null,"topics":["cluster-management","kafka","self-healing"],"latest_commit_sha":null,"homepage":"https://github.com/linkedin/cruise-control/tags","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/linkedin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2017-08-15T17:32:10.000Z","updated_at":"2026-05-20T20:04:34.000Z","dependencies_parsed_at":"2023-02-09T03:30:45.407Z","dependency_job_id":"4c2c3da4-0619-4bf4-9a20-41000d7e95eb","html_url":"https://github.com/linkedin/cruise-control","commit_stats":{"total_commits":918,"total_committers":111,"mean_commits":8.27027027027027,"dds":0.5305010893246187,"last_synced_commit":"96c3d6bce10bafbeed63a51977b8b2daa9e94576"},"previous_names":[],"tags_count":496,"template":false,"template_full_name":null,"purl":"pkg:github/linkedin/cruise-control","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkedin%2Fcruise-control","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkedin%2Fcruise-control/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkedin%2Fcruise-control/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkedin%2Fcruise-control/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/linkedin","download_url":"https://codeload.github.com/linkedin/cruise-control/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkedin%2Fcruise-control/sbom","scorecard":{"id":590954,"data":{"date":"2025-08-11","repo":{"name":"github.com/linkedin/cruise-control","commit":"bea2bcbd50936c4b152e59dbf3803b990917395d"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.3,"checks":[{"name":"Maintained","score":2,"reason":"3 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":9,"reason":"Found 28/30 approved changesets -- score normalized to 9","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/artifactory.yaml:1","Warn: no topLevel permission defined: .github/workflows/ci.yaml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Binary-Artifacts","score":9,"reason":"binaries present in source code","details":["Warn: binary detected: gradle/wrapper/gradle-wrapper.jar:1"],"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: BSD 2-Clause \"Simplified\" License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/artifactory.yaml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/artifactory.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/artifactory.yaml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/artifactory.yaml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/artifactory.yaml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/artifactory.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yaml:23: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/ci.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yaml:26: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/ci.yaml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yaml:32: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/ci.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yaml:45: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/ci.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yaml:48: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/ci.yaml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yaml:54: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/ci.yaml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yaml:68: update your workflow using https://app.stepsecurity.io/secureworkflow/linkedin/cruise-control/ci.yaml/main?enable=pin","Info:   0 out of   7 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   3 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":5,"reason":"5 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2014-14 / GHSA-652x-xj99-gmcc","Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-9wx4-h78v-vm56","Warn: Project is vulnerable to: PYSEC-2014-13 / GHSA-cfj3-7x9c-4p3h","Warn: Project is vulnerable to: PYSEC-2018-28 / GHSA-x84v-xcm2-53pg"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 29 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-20T21:52:16.099Z","repository_id":37602791,"created_at":"2025-08-20T21:52:16.099Z","updated_at":"2025-08-20T21:52:16.099Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33545458,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"ssl_error","status_checked_at":"2026-05-26T15:22:15.568Z","response_time":63,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster-management","kafka","self-healing"],"created_at":"2024-07-31T07:01:02.511Z","updated_at":"2026-05-27T01:00:54.314Z","avatar_url":"https://github.com/linkedin.png","language":"Java","funding_links":[],"categories":["Java","Libraries","AI for *Ops","进程间通信","Management \u0026 Monitoring","Operations"],"sub_categories":["Kafka","Automation \u0026 Self-Healing","Cluster Management","Operational Utilities"],"readme":"Cruise Control for Apache Kafka\n===================\n\n[![CI](https://github.com/linkedin/cruise-control/actions/workflows/ci.yaml/badge.svg)](https://github.com/linkedin/cruise-control/actions/workflows/ci.yaml)\n[![CircleCI](https://circleci.com/gh/linkedin/cruise-control.svg?style=svg)](https://circleci.com/gh/linkedin/cruise-control)\n\n\n### Introduction ###\n  Cruise Control is a product that helps run Apache Kafka clusters at large scale. Due to the popularity of \n  Apache Kafka, many companies have increasingly large Kafka clusters with hundreds of brokers. At LinkedIn, we have 10K+ Kafka brokers, \n  which means broker deaths are an almost daily occurrence and balancing the workload of Kafka also becomes a big overhead. \n  \n  Kafka Cruise Control is designed to address this operational scalability issue.\n  \n### Features ###\n  Kafka Cruise Control provides the following features out of the box:\n  \n  * Resource utilization tracking for brokers, topics, and partitions.\n  \n  * Query the current Kafka cluster state to see the online and offline partitions, in-sync and out-of-sync replicas,\n  replicas under `min.insync.replicas`, online and offline logDirs, and distribution of replicas in the cluster.\n  \n  * Multi-goal rebalance proposal generation for:\n    * Rack-awareness\n    * Resource capacity violation checks (CPU, DISK, Network I/O)\n    * Per-broker replica count violation check\n    * Resource utilization balance (CPU, DISK, Network I/O)\n    * Leader traffic distribution\n    * Replica distribution for topics\n    * Global replica distribution\n    * Global leader replica distribution\n    * Custom goals that you wrote and plugged in\n  \n  * Anomaly detection, alerting, and self-healing for the Kafka cluster, including:\n    * Goal violation\n    * Broker failure detection\n    * Metric anomaly detection\n    * Disk failure detection (not available in `kafka_0_11_and_1_0` branch)\n    * Slow broker detection (not available in `kafka_0_11_and_1_0` branch)\n  \n  * Admin operations, including:\n    * Add brokers\n    * Remove brokers\n    * Demote brokers\n    * Rebalance the cluster\n    * Fix offline replicas (not available in `kafka_0_11_and_1_0` branch)\n    * Perform preferred leader election (PLE)\n    * Fix offline replicas\n    * Adjust replication factor\n\n### Environment Requirements ###\n* The `main` (previously `migrate_to_kafka_2_5`) branch of Cruise Control is compatible with Apache Kafka `2.5+` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.5.*`),\n  `2.6` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.5.11+`), `2.7` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.5.36+`),\n  `2.8` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.5.66+`), `3.0` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.5.85+`),\n  `3.1` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.5.85+`), `3.8` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.5.142+`),\n  `3.9` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.5.143+`), and `4.0` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.5.144+`)\n* The `migrate_to_kafka_2_4` branch of Cruise Control is compatible with Apache Kafka `2.4` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.4.*`).\n* The `kafka_2_0_to_2_3` branch (deprecated) of Cruise Control is compatible with Apache Kafka `2.0`, `2.1`, `2.2`, and `2.3` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `2.0.*`).\n* The `kafka_0_11_and_1_0` branch (deprecated) of Cruise Control is compatible with Apache Kafka `0.11.0.0`, `1.0`, and `1.1` (i.e. [Releases](https://github.com/linkedin/cruise-control/releases) with `0.1.*`).\n* `message.format.version` `0.10.0` and above is needed.\n* The `kafka_2_0_to_2_3` and `kafka_0_11_and_1_0` branches compile with `Scala 2.11`.\n* The branch `migrate_to_kafka_2_4` compiles with `Scala 2.12`.\n* The branch `migrate_to_kafka_2_5` compile with `Scala 2.13`.\n* This project requires Java 17.\n\n#### Known Compatibility Issues ####\n* Support for Apache Kafka `2.0`, `2.1`, `2.2`, and `2.3` requires [KAFKA-8875](https://issues.apache.org/jira/browse/KAFKA-8875) hotfix.\n\n### Quick Start ###\n0. Get Cruise Control\n    1. (Option-1): via `git clone`\n        * `git clone https://github.com/linkedin/cruise-control.git \u0026\u0026 cd cruise-control/`\n    2. (Option-2): via browsing the available releases:\n        * Browse `https://github.com/linkedin/cruise-control/releases` to pick a release -- e.g. `0.1.10`\n        * Get and extract the release: `wget https://github.com/linkedin/cruise-control/archive/0.1.10.tar.gz \n        \u0026\u0026 tar zxvf 0.1.10.tar.gz \u0026\u0026 cd cruise-control-0.1.10/`\n        * Initialize the local repo: `git init \u0026\u0026 git add . \u0026\u0026 git commit -m \"Init local repo.\"\n        \u0026\u0026 git tag -a 0.1.10 -m \"Init local version.\"`\n1. This step is required if `CruiseControlMetricsReporter` is used for metrics collection (i.e. the default for Cruise\nControl). The metrics reporter periodically samples the Kafka raw metrics on the broker and sends them to a Kafka topic.\n    * `./gradlew jar` (Note: This project requires Java 17)\n    * Copy `./cruise-control-metrics-reporter/build/libs/cruise-control-metrics-reporter-A.B.C.jar` (Where `A.B.C` is\n    the version of the Cruise Control) to your Kafka server dependency jar folder. For Apache Kafka, the folder would\n    be `core/build/dependant-libs-SCALA_VERSION/` (for a Kafka source checkout) or `libs/` (for a Kafka release download).\n    * Modify Kafka server configuration to set `metric.reporters` to\n    `com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter`. For Apache Kafka, server \n    properties are located at `./config/server.properties`.\n    * If `SSL` is enabled, ensure that the relevant client configurations are properly set for all brokers in \n    `./config/server.properties`. Note that `CruiseControlMetricsReporter` takes all configurations for vanilla \n    `KafkaProducer` with a prefix of `cruise.control.metrics.reporter.` -- e.g. \n    `cruise.control.metrics.reporter.ssl.truststore.password`.    \n    * If the default broker cleanup policy is `compact`, make sure that the topic to which Cruise Control metrics\n    reporter should send messages is created with the `delete` cleanup policy -- the default metrics reporter topic\n    is `__CruiseControlMetrics`.\n2. Start Kafka server ([See tutorial](https://kafka.apache.org/quickstart)), and if you're using a ZooKeeper-based Kafka cluster also start a ZooKeeper server.\n3. Modify `config/cruisecontrol.properties` of Cruise Control:\n    * (Required) fill in `bootstrap.servers` to the Kafka cluster to be monitored.\n    * (Required) update `capacity.config.file` to the path of your capacity file.  \n      * Capacity file is a JSON file that provides the capacity of the brokers\n      * You can start Cruise Control server with the default file (`config/capacityJBOD.json`), but it may not reflect the actual capacity of the brokers \n      * See [BrokerCapacityConfigurationFileResolver configurations](https://github.com/linkedin/cruise-control/wiki/Configurations#brokercapacityconfigurationfileresolver-configurations) for more information and examples\n    * (Optional) set `metric.sampler.class` to your implementation (the default sampler class is `CruiseControlMetricsReporterSampler`) \n    * (Optional) set `sample.store.class` to your implementation if you have one (the default `SampleStore` is `KafkaSampleStore`)\n4. Run the following command \n    ```\n    ./gradlew jar copyDependantLibs\n    ./kafka-cruise-control-start.sh [-jars PATH_TO_YOUR_JAR_1,PATH_TO_YOUR_JAR_2] config/cruisecontrol.properties [port]\n    ```\n    JAR files correspond to your applications and `port` enables customizing the Cruise Control port number (default: `9090`).\n    * (Note) To emit Cruise Control JMX metrics on a particular port (e.g. `56666`), `export JMX_PORT=56666` before\n    running `kafka-cruise-control-start.sh`\n5. (Verify your setup) Visit `http://localhost:9090/kafkacruisecontrol/state` (or \n`http://localhost:\\[port\\]/kafkacruisecontrol/state` if you specified the port when starting Cruise Control).\n\n**Note**: \n* Cruise Control will need some time to read the raw Kafka metrics from the cluster.\n* The metrics of a newly up broker may take a few minutes to get stable. Cruise Control will drop the inconsistent \nmetrics (e.g when topic bytes-in is higher than broker bytes-in), so first few windows may not have enough valid partitions.\n\n### REST API ###\nCruise Control provides a [REST API](https://github.com/linkedin/cruise-control/wiki/REST-APIs) for users \nto interact with. See the wiki page for more details.\n\n### How Does It Work ###\nCruise Control relies on the recent load information of replicas to optimize the cluster.\n\nCruise Control periodically collects resource utilization samples at both broker- and partition-level to \ninfer the traffic pattern of each partition. Based on the traffic characteristics and distribution of all the partitions, \nit derives the load impact of each partition over the brokers. Cruise Control then builds a workload\nmodel to simulate the workload of the Kafka cluster. The goal optimizer explores different ways to generate \ncluster workload optimization proposals based on the user-specified list of goals.\n\nCruise Control also monitors the liveness of all the brokers in the cluster.\nTo avoid the loss of redundancy, Cruise Control automatically moves replicas from failed brokers to alive ones.\n\nFor more details about how Cruise Control achieves that, see \n[these slides](https://www.slideshare.net/JiangjieQin/introduction-to-kafka-cruise-control-68180931).\n\n### Configurations for Cruise Control ###\nTo read more about the configurations. Check the \n[configurations wiki page](https://github.com/linkedin/cruise-control/wiki/Configurations).\n\n### Artifactory ###\nPublished at [Jfrog Artifactory](https://linkedin.jfrog.io/linkedin/webapp/#/artifacts/browse/tree/General/cruise-control). See [available releases](https://github.com/linkedin/cruise-control/releases). \n\n### Pluggable Components ###\nMore about pluggable components can be found in the \n[pluggable components wiki page](https://github.com/linkedin/cruise-control/wiki/Pluggable-Components).\n\n#### Metric Sampler #### \nThe metric sampler enables users to deploy Cruise Control to various environments and work with the existing metric systems.\n\nCruise Control provides a metrics reporter that can be configured in your Apache Kafka server. Metrics reporter generates\nperformance metrics to a Kafka metrics topic that can be consumed by Cruise Control.\n\n#### Sample Store ####\nThe Sample Store enables storage of collected metric samples and training samples in an external storage. \n\nMetric sampling uses derived data from the raw metrics, and the accuracy of the derived data depends on the metadata of the cluster at that point.\nHence, when we look at the old metrics, if we do not know the metadata at the point the metric was collected, the derived data would not be accurate.\nSample Store helps solving this problem by storing the derived data directly to an external storage for later loading.\n\nThe default Sample Store implementation produces metric samples back to Kafka.\n\n#### Goals ####\nThe goals in Cruise Control are pluggable with different priorities. The default goals in order of decreasing priority are:\n * **RackAwareGoal** - Ensures that all replicas of each partition are assigned in a rack aware manner -- i.e. no more than one replica of \n each partition resides in the same rack.\n * **RackAwareDistributionGoal** - A relaxed version of `RackAwareGoal`. Contrary to `RackAwareGoal`, as long as replicas of each partition\n can achieve a perfectly even distribution across the racks, this goal lets placement of multiple replicas of a partition into a single rack.\n * **MinTopicLeadersPerBrokerGoal** - Ensures that each alive broker has at least a certain number of leader replica of each topic in a configured set of topics\n * **ReplicaCapacityGoal** - Ensures that the maximum number of replicas per broker is under the specified maximum limit.\n * **DiskCapacityGoal** - Ensures that Disk space usage of each broker is below a given threshold.\n * **NetworkInboundCapacityGoal** - Ensures that inbound network utilization of each broker is below a given threshold.\n * **NetworkOutboundCapacityGoal** - Ensures that outbound network utilization of each broker is below a given threshold.\n * **CpuCapacityGoal** - Ensures that CPU utilization of each broker is below a given threshold.\n * **ReplicaDistributionGoal** - Attempts to make all the brokers in a cluster have a similar number of replicas.\n * **PotentialNwOutGoal** - Ensures that the potential network output (when all the replicas in the broker become leaders) on each of the broker do \n not exceed the broker’s network outbound bandwidth capacity.\n * **DiskUsageDistributionGoal** - Attempts to keep the Disk space usage variance among brokers within a certain range relative to the average Disk utilization.\n * **NetworkInboundUsageDistributionGoal** - Attempts to keep the inbound network utilization variance among brokers within a certain range relative to the average inbound network utilization.\n * **NetworkOutboundUsageDistributionGoal** - Attempts to keep the outbound network utilization variance among brokers within a certain range relative to the average outbound network utilization.\n * **CpuUsageDistributionGoal** - Attempts to keep the CPU usage variance among brokers within a certain range relative to the average CPU utilization.\n * **LeaderReplicaDistributionGoal** - Attempts to make all the brokers in a cluster have a similar number of leader replicas.\n * **LeaderBytesInDistributionGoal** - Attempts to equalize the leader bytes in rate on each host.\n * **TopicReplicaDistributionGoal** - Attempts to maintain an even distribution of any topic's partitions across the entire cluster.\n * **PreferredLeaderElectionGoal** - Simply move the leaders to the first replica of each partition.\n * **KafkaAssignerDiskUsageDistributionGoal** - (Kafka-assigner mode) Attempts to distribute disk usage evenly among brokers based on swap.\n * **IntraBrokerDiskCapacityGoal** - (Rebalance-disk mode, not available in `kafka_0_11_and_1_0` branch) Ensures that Disk space usage of each disk is below a given threshold.\n * **IntraBrokerDiskUsageDistributionGoal** - (Rebalance-disk mode, not available in `kafka_0_11_and_1_0` branch) Attempts to keep the Disk space usage variance among disks within a certain range relative to the average broker Disk utilization.\n\n#### Anomaly Notifier ####\nThe anomaly notifier allows users to be notified when an anomaly is detected. Anomalies include:\n * Broker failure\n * Goal violation\n * Metric anomaly\n * Disk failure (not available in `kafka_0_11_and_1_0` branch)\n * Slow brokers (not available in `kafka_0_11_and_1_0` branch)\n * Topic replication factor anomaly (not available in `kafka_0_11_and_1_0` branch)\n * Topic partition size anomaly (not available in `kafka_0_11_and_1_0` branch)\n * Maintenance Events (not available in `kafka_0_11_and_1_0` branch)\n \nIn addition to anomaly notifications, users can enable actions to be taken in response to an anomaly by turning self-healing\non for the relevant anomaly detectors. Multiple anomaly detectors work in harmony using distinct mitigation mechanisms.\nTheir actions broadly fall into the following categories:\n * **fix** - fix the problem right away (e.g. start a rebalance, fix offline replicas)\n * **check** - check the situation again after a configurable delay (e.g. adopt a grace period before fixing broker failures)\n * **ignore** - ignore the anomaly (e.g. self-healing is disabled)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinkedin%2Fcruise-control","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinkedin%2Fcruise-control","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinkedin%2Fcruise-control/lists"}