{"id":18408345,"url":"https://github.com/pingcap/tidb-insight","last_synced_at":"2026-03-07T01:02:43.584Z","repository":{"id":31917906,"uuid":"130807250","full_name":"pingcap/tidb-insight","owner":"pingcap","description":null,"archived":false,"fork":false,"pushed_at":"2022-09-02T03:46:38.000Z","size":1627,"stargazers_count":11,"open_issues_count":1,"forks_count":9,"subscribers_count":99,"default_branch":"tiup","last_synced_at":"2025-04-07T08:36:54.456Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pingcap.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-24T06:33:09.000Z","updated_at":"2022-01-11T03:26:30.000Z","dependencies_parsed_at":"2022-07-26T04:18:32.531Z","dependency_job_id":null,"html_url":"https://github.com/pingcap/tidb-insight","commit_stats":null,"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/pingcap/tidb-insight","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Ftidb-insight","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Ftidb-insight/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Ftidb-insight/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Ftidb-insight/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pingcap","download_url":"https://codeload.github.com/pingcap/tidb-insight/tar.gz/refs/heads/tiup","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Ftidb-insight/sbom","scorecard":{"id":734337,"data":{"date":"2025-08-11","repo":{"name":"github.com/pingcap/tidb-insight","commit":"fb5ae0ddc8c11543bdb6591de2a82b09a8d02961"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.6,"checks":[{"name":"Code-Review","score":1,"reason":"Found 5/30 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/build.yaml:1","Warn: no topLevel permission defined: .github/workflows/release.yaml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/build.yaml:33: update your workflow using https://app.stepsecurity.io/secureworkflow/pingcap/tidb-insight/build.yaml/tiup?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/build.yaml:38: update your workflow using https://app.stepsecurity.io/secureworkflow/pingcap/tidb-insight/build.yaml/tiup?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:33: update your workflow using https://app.stepsecurity.io/secureworkflow/pingcap/tidb-insight/release.yaml/tiup?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:39: update your workflow using https://app.stepsecurity.io/secureworkflow/pingcap/tidb-insight/release.yaml/tiup?enable=pin","Info:   0 out of   4 GitHub-owned GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact v0.2.5 not signed: https://api.github.com/repos/pingcap/tidb-insight/releases/13386711","Warn: release artifact v0.2.4 not signed: https://api.github.com/repos/pingcap/tidb-insight/releases/12779441","Warn: release artifact v0.2.3 not signed: https://api.github.com/repos/pingcap/tidb-insight/releases/12494834","Warn: release artifact v0.2.2 not signed: https://api.github.com/repos/pingcap/tidb-insight/releases/12428283","Warn: release artifact v0.2.1.1 not signed: https://api.github.com/repos/pingcap/tidb-insight/releases/11868476","Warn: release artifact v0.2.5 does not have provenance: https://api.github.com/repos/pingcap/tidb-insight/releases/13386711","Warn: release artifact v0.2.4 does not have provenance: https://api.github.com/repos/pingcap/tidb-insight/releases/12779441","Warn: release artifact v0.2.3 does not have provenance: https://api.github.com/repos/pingcap/tidb-insight/releases/12494834","Warn: release artifact v0.2.2 does not have provenance: https://api.github.com/repos/pingcap/tidb-insight/releases/12428283","Warn: release artifact v0.2.1.1 does not have provenance: https://api.github.com/repos/pingcap/tidb-insight/releases/11868476"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during GetBranch(master): error during branchesHandler.query: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 7 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"33 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2021-358 / GHSA-2pfh-q76x-gwvm","Warn: Project is vulnerable to: PYSEC-2018-81 / GHSA-3xvg-x47j-x75w","Warn: Project is vulnerable to: GHSA-4r65-35qq-ch8j","Warn: Project is vulnerable to: PYSEC-2017-4 / GHSA-588w-w6mv-3cw5","Warn: Project is vulnerable to: PYSEC-2021-1 / GHSA-5rrg-rr89-x9mv","Warn: Project is vulnerable to: GHSA-74vq-h4q8-x6jv","Warn: Project is vulnerable to: PYSEC-2020-3 / GHSA-785x-qw4v-6872","Warn: Project is vulnerable to: PYSEC-2020-9 / GHSA-893h-35v4-mxqx","Warn: Project is vulnerable to: PYSEC-2021-124 / GHSA-8f4m-hccc-8qph","Warn: Project is vulnerable to: PYSEC-2020-11 / GHSA-923p-fr2c-g5m2","Warn: Project is vulnerable to: PYSEC-2020-10 / GHSA-f85h-23mf-2fwh","Warn: Project is vulnerable to: PYSEC-2018-43 / GHSA-fc4h-467w-46rh","Warn: Project is vulnerable to: PYSEC-2020-5 / GHSA-g4mq-6fp5-qwcf","Warn: Project is vulnerable to: PYSEC-2019-2 / GHSA-grgm-pph5-j5h7","Warn: Project is vulnerable to: PYSEC-2020-161 / GHSA-gwr8-5j83-483c","Warn: Project is vulnerable to: PYSEC-2020-6 / GHSA-h39q-95q5-9jfp","Warn: Project is vulnerable to: PYSEC-2019-171 / GHSA-h653-95qw-h2mp","Warn: Project is vulnerable to: PYSEC-2018-44 / GHSA-hwrm-63v2-42g4","Warn: Project is vulnerable to: GHSA-j569-fghw-f9rx","Warn: Project is vulnerable to: GHSA-j667-c2hm-f2wp","Warn: Project is vulnerable to: GHSA-jpvw-p8pr-9g2x","Warn: Project is vulnerable to: PYSEC-2018-42 / GHSA-jwcc-j78w-j73w","Warn: Project is vulnerable to: PYSEC-2019-4 / GHSA-pm48-cvv2-29q5","Warn: Project is vulnerable to: PYSEC-2021-105 / GHSA-r6h7-5pq2-j77h","Warn: Project is vulnerable to: PYSEC-2018-60 / GHSA-v735-2pp6-h86r","Warn: Project is vulnerable to: PYSEC-2020-12 / GHSA-vcg8-98q8-g7mj","Warn: Project is vulnerable to: GHSA-vp9j-rghq-8jhh","Warn: Project is vulnerable to: PYSEC-2021-106 / GHSA-wv5p-gmmv-wh9v","Warn: Project is vulnerable to: PYSEC-2020-210","Warn: Project is vulnerable to: PYSEC-2020-220","Warn: Project is vulnerable to: PYSEC-2020-7","Warn: Project is vulnerable to: PYSEC-2020-8","Warn: Project is vulnerable to: PYSEC-2021-126"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-22T15:18:34.361Z","repository_id":31917906,"created_at":"2025-08-22T15:18:34.361Z","updated_at":"2025-08-22T15:18:34.361Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30204452,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T19:07:06.838Z","status":"ssl_error","status_checked_at":"2026-03-06T18:57:34.882Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T03:18:38.745Z","updated_at":"2026-03-07T01:02:43.486Z","avatar_url":"https://github.com/pingcap.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TiDB Insight\n\n[![Build Status](https://travis-ci.org/pingcap/tidb-insight.svg?branch=master)](https://travis-ci.org/pingcap/tidb-insight)\n[![Go Report Card](https://goreportcard.com/badge/github.com/pingcap/tidb-insight)](https://goreportcard.com/report/github.com/pingcap/tidb-insight)\n\nTiDB Insight is a set of tools and/or scripts to collect information and metrics from all nodes of a TiDB cluster. The data is collected and archived to a tarball so that one can copy the data anywhere for further investigation.\n\nTiDB Insight is designed to help you troubleshoot and diagnose machines within a TiDB cluster when you don't have access to the cluster.\n\n## Collector\n\nThe `collector` is a simple binary tool written in Go that collects various kinds of server information, including system information, TiDB/TiKV/PD versions, etc.\n\n`collector` prints a JSON to stdout.\n\n## Scripts\n\nWIP: The scripts to call other tools (e.g., `collector`) and organize their results.\n\nThe `insight.py` file is the entrance of all scripts. It calls `collector` to collect basic information and then trigger other tools to gather more data.\n\nIt is recommended to run this script with the root privilege.\n\nThe following features have been implemented:\n\n - Record `perf` data of the whole system, TiDB modules, or specific process. (disabled by default)\n - Check the file size of TiDB modules' data directories. (enabled by default)\n - Collect and save log files of TiDB modules or system. (disabled by default)\n - Collect and save various configuration files of system. (disabled by default)\n - Query PD API to collect runtime information of the TiDB cluster. (enabled by default)\n\n### Usage\n\nFor a full list of arguments, you may refer to the output of `insight.py -h`:\n\n```\nusage: insight.py [-h] [-o OUTPUT] [-p] [--pid PID] [--tidb-proc]\n                  [--perf-exec PERF_EXEC] [--perf-freq PERF_FREQ]\n                  [--perf-time PERF_TIME] [-l] [--syslog] [--config-file]\n                  [--pd-host PD_HOST] [--pd-port PD_PORT]\n\nTiDB Insight Scripts\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -o OUTPUT, --output OUTPUT\n                        The directory to store output data of TiDB Insight.\n                        Any existing file will be overwritten without futher\n                        confirmation.\n  -p, --perf            Collect trace info using perf. Disabled by default.\n  --pid PID             PID of process to run perf on. If `-p`/`--perf` is not\n                        set, this value will not take effect. Multiple PIDs\n                        can be set by using more than one `--pid` argument.\n                        `None` by default which means the whole system.\n  --tidb-proc           Collect perf data for PD/TiDB/TiKV processes instead\n                        of the whole system.\n  --perf-exec PERF_EXEC\n                        Custom path of perf executable file.\n  --perf-freq PERF_FREQ\n                        Event sampling frequency of perf-record, in Hz.\n  --perf-time PERF_TIME\n                        Time period of perf recording, in seconds.\n  -l, --log             Collect log files in output. PD/TiDB/TiKV logs are\n                        included by default.\n  --syslog              Collect the system log in output. This may\n                        significantly increase output size. If `-l/--log` is\n                        not set, the system log will be ignored.\n  --config-file         Collect various configuration files in output,\n                        disabled by default.\n  --pd-host PD_HOST     The host of the PD server. `localhost` by default.\n  --pd-port PD_PORT     The port of PD API service, `2379` by default.\n\nNote that some arguments may decrease system performance.\n```\n\nIf `insight.py` is called with no argument provided, the following information is collected and saved to `data` directory in the current working directory:\n\n - `collector`, which provides the system information, program versions, NTP status, partition statistics and process statistics of TiDB modules (if present), placed in the `collector` sub-directory.\n - PD API output of `config`, `diagnose`, `health`, `hotspot`, `labels`, `members`, `operators`, `regions`, `regionstats`, `schedulers` and `status`, placed in the `pdctl` sub-directory in JSON format.\n\nUsing `-o` can set another location to store output data except the default `data`.\n\n### Examples\n\nTake `insight.py -l --config-file`, a common combination of arguments as an example. It is used to collect the basic system information, TiDB modules' log files and several config files.\n\nIf you want to collect runtime information of a TiDB cluster from PD on a remote host (instead of running the scripts on that server), set `--pd-host` to the IP address or a resolvable hostname of that host. Additionally, if PD is listening on non-default port, use `--pd-port` to set it.\n\nThe system log is not collectd by default, even if `-l` is set. If needed, use `-l --syslog` to enable it. Both `syslog` and `systemd-journald` logs are supported and automatically detected. Note that there is no content filter available for log files, thus enabling log file collecting may lead to a very large output size.\n\n## Integration with Ansible\n\nThe `tidb-insight` project is designed to integrate with [tidb-ansible](https://github.com/pingcap/tidb-ansible) playbooks.\n\n\u003e All collecting features will be disabled by default after the development of integration is finished, so it's highly recommended to specify all the arguments needed when using `tidb-insight` now, even if the feature is currently enabled by default.\n\nTO DO: Add documents/links to documents of using `tidb-insight` with `tidb-ansible`.\n\n## Insight\n\nTO DO: Create the analysis tool that helps process dataset.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpingcap%2Ftidb-insight","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpingcap%2Ftidb-insight","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpingcap%2Ftidb-insight/lists"}