{"id":18408332,"url":"https://github.com/pingcap/monitoring","last_synced_at":"2025-04-12T22:25:14.843Z","repository":{"id":36027853,"uuid":"197427119","full_name":"pingcap/monitoring","owner":"pingcap","description":null,"archived":false,"fork":false,"pushed_at":"2025-04-11T06:06:46.000Z","size":145556,"stargazers_count":27,"open_issues_count":31,"forks_count":39,"subscribers_count":80,"default_branch":"master","last_synced_at":"2025-04-11T07:46:06.387Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pingcap.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-17T16:38:13.000Z","updated_at":"2025-04-11T06:06:49.000Z","dependencies_parsed_at":"2024-05-28T08:33:17.130Z","dependency_job_id":"f0af6224-1040-4483-8eb7-20d5d171c35f","html_url":"https://github.com/pingcap/monitoring","commit_stats":null,"previous_names":[],"tags_count":43,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Fmonitoring","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Fmonitoring/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Fmonitoring/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Fmonitoring/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pingcap","download_url":"https://codeload.github.com/pingcap/monitoring/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248638878,"owners_count":21137721,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T03:18:31.442Z","updated_at":"2025-04-12T22:25:14.810Z","avatar_url":"https://github.com/pingcap.png","language":"Shell","readme":"# Overview\nThis repo contains three functions. One support dynamic reload rules of prometheus, the other is used to support multi TiDB version.\n\n## Function1 - Automatic generation monitoring configurations for ansible deployment and operator deployment\n\nGenerate versioned monitoring data that is used by tidb-operator or tidb-ansible. It is a binary to load monitoring configurations from special component repo and it also contain platform monitoring from `platform-monitoring` directory.\n\n![monitoring](image/monitoring.png)\n\nAnsible deployment and operator deployment has different platform monitoring files. The structure of platform monitoring directory is here:\n\n```$xslt\nplatform-monitoring/\n     |-- ansbile\n     |       |---- grafana\n     |       |        | ---- machine.json\n     |       |\n     |       |---- rule\n     |                | ---- machine.rules.yaml\n     |\n     |-- operator\n             |---- grafana\n             |        |\n             |        |----- node.json\n             |---- rule\n             |        |\n             |        |---- node.rules.yaml\n             |---- datasource\n                      |\n                      |---- k8s-cluster.yaml\n                      |\n                      |---- tidb-cluster.yaml\n           \n```\n\nThe binary will automate load platform monitoring based on the above directory structure to different deployment directory.\n\n## How to use it\n\n```$xslt\ngo build -o pull-monitoring  cmd/monitoring.go\n\n./pull-monitoring\nError: required flag(s) \"config\" not set\nUsage:\n  load [flags]\n\nFlags:\n      --auto-push                        auto generate new branch from master and push auto-generate files to the branch\n      --config string                    the monitoring configuration file.\n  -h, --help                             help for load\n      --output-dir string                the base directory of the program (default \".\")\n      --tag string                       the tag of pull monitoring repo.\n\n```\n\n## Function2 - Prometheus Rule Reloader\n\nIt is a simple binary to trigger a reload when Rules are updated. It watches dirs and call `reload` API that the rules has been changed. \n\nIt provide a UI to update rules(For ease of use, the UI is similar with UI of Prometheus).\n![UI](reload/ui/static/image/ui.png)\n\nThe text editor is friendly to Yaml format. To quickly verify that the modification is successful, there is Rules UI which get rules from Prometheus (You can also verify it by Prometheus.).\n\n## How to use it\n\n```$xslt\nmake\n```\n\nThere is binary in `reload/build/{plateform}/reload`, you can run it like this\n\n```$xslt\n./reload --watch-path=/tmp/prometheus-2.8.0.darwin-amd64/rules --prometheus-url=http://127.0.0.1:9090\n```\n\n## Function3 - Cloud TiDB Monitoring\n\n## Overview\n\nGenerate versioned monitoring data that is used by tidb-operator. This project pulls TiDB monitoring data from [tidb-ansible](https://github.com/pingcap/tidb-ansible) and uses the git tag to understand the TiDB version.\n\nAll TiDB version monitoring information is automatically generated (by default it generates for TiDB version \u003e= 2.1.8). The structure of monitor directory is like this\n\n```$xslt\n monitor/\n    |── v2.1.8\n    |   ├── dashboards\n    |   │   ├─ overview.json \n    |   │   ├─ binlog.json  \n    |   │   |_ pd.json\n    |   |   |_ tikv_pull.json\n    |   |   |_ tidb.json \n    |   |   \n    |   |── rules\n    |   |   ├── tidb.rule.yml\n    |   |   ├── tikv.rule.yml\n    |   |   └── pd.rule.yml\n    |   |—— Dockerfile     \n    |   |__ init.sh\n    |\n    |── v3.0.0\n    |   ├── dashboards\n    |   │   |- overview.json \n    |   │   |- binlog.json  \n    |   │   |- pd.json\n    |   |   |- tidb.json \n    |   |   |- tikv_details.json\n    |   |   |- tikv_sumary.json\n    |   |   |_ tikv_trouble_shooting.json\n    |   |   \n    |   |── rules\n    |   |   ├── tidb.rule.yml\n    |   |   ├── tikv.rule.yml\n    |   |   └── pd.rule.yml\n    |   |—— Dockerfile     \n    |   |__ init.sh\n    |___ ...\n        \n```\n\n\n## How to use it\n\n```$xslt\nmake\n```\n\nThere will be monitoring binary, you can run it like this\n\n```$xslt\n./monitoring --path=.\n```\n\nThe program will replace some variables and the docker will receive 4 variables: \n\n```$xslt\nGF_PROVISIONING_PATH // grafana provisioning path\nTIDB_CLUSTER_NAME // TiDB cluster name\nTIDB_ENABLE_BINLOG // whether enable binlog\nPROM_CONFIG_PATH // proemtheus rules config path\n```\n\n## How to Add New Dashbords\n\nReference PR: https://github.com/pingcap/monitoring/pull/227\n\n1. Add dashbord files into the `dashbords` map in the file `pkg/operator/dashboards.go`, so that the `platform-monitoring` won't filter these files.\n   \n    ```go\n    // pkg/operator/dashboards.go\n    dashboards = map[string]string{\n\t\t  \"binlog.json\":                  \"Test-Cluster-Binlog\",\n\t\t  \"tidb.json\":                    \"Test-Cluster-TiDB\",\n\t\t  \"overview.json\":                \"Test-Cluster-Overview\",\n      // ...\n    }\n   ```\n\n   The key is the file name fetched from each github repository, and the value is the dashbord title in Grafana.\n\n2. Update the `init.sh` of the image `monitoring-initializer` to provide new dashbord files for the Grafana.\n   \n    For example:\n\n    ```bash\n    # TiDB dashboard\n    # copy dashboard files to the `$GF_PROVISIONING_PATH/dashboards` dir where Grafana loads dashboards.\n    cp /tmp/tidb*.json $GF_PROVISIONING_PATH/dashboards  \n\n    # replace some fields\n    sed -i 's/Test-Cluster-TiDB/Cluster-TiDB/g' $GF_PROVISIONING_PATH/dashboards/tidb.json\n    sed -i 's/Test-Cluster-TiDB/Cluster-TiDB/g' $GF_PROVISIONING_PATH/dashboards/tidb_runtime.json\n    sed -i 's/Test-Cluster-TiDB/Cluster-TiDB/g' $GF_PROVISIONING_PATH/dashboards/tidb_resource_control.json\n    ```\n\n3. Build the image `monitoring-initializer` and test it locally.\n\n   1. Build the binary `pull-monitoring`.\n\n      ```bash\n      make pull-monitoring\n      ```\n  \n   2. Fetch dashbords files and generate the dir used to build a new image.\n\n      ```bash\n      ./pull-monitoring --config=monitoring.yaml --tag=${tag_in_target_repo} --token=${your_github_token}\n      ```\n\n      You can see a new dir named `${tag}` created in the dir `monitor-snapshot`.\n\n      ```bash\n      $ ls ./monitor-snapshot/${tag}/operator\n      dashboards  datasources  Dockerfile  init.sh  rules\n      ```\n\n    3. Build the image `monitoring-initializer`\n   \n      ```bash\n      cd ./monitor-snapshot/master/operator \u0026\u0026  docker build -t ${image} .\n      ```\n\n    4. Follow the [doc](https://docs.pingcap.com/tidb-in-kubernetes/stable/monitor-a-tidb-cluster) and use your image to deploy monitoring.\n\n        Note that you need replace the image `pingcap/tidb-monitor-initializer` and version with your image in the `TidbMonitor` spec.\n\u003c!-- VERSION_PLACEHOLDER: v8.2.0-alpha --\u003e","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpingcap%2Fmonitoring","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpingcap%2Fmonitoring","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpingcap%2Fmonitoring/lists"}