{"id":30930252,"url":"https://github.com/yansheng836/mysql_collect_processlist","last_synced_at":"2026-05-19T10:02:28.619Z","repository":{"id":314023830,"uuid":"1052536906","full_name":"yansheng836/mysql_collect_processlist","owner":"yansheng836","description":"这是一个纯脚本工具，用于从MySQL的`information_schema.processlist`视图中定期收集数据并保存到本地日志文件。","archived":false,"fork":false,"pushed_at":"2025-09-10T04:31:43.000Z","size":26,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-10T05:50:09.490Z","etag":null,"topics":["mysql","shell","sql","tool"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yansheng836.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-08T07:36:14.000Z","updated_at":"2025-09-10T04:31:39.000Z","dependencies_parsed_at":"2025-09-10T06:03:37.466Z","dependency_job_id":null,"html_url":"https://github.com/yansheng836/mysql_collect_processlist","commit_stats":null,"previous_names":["yansheng836/mysql_collect_processlist"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/yansheng836/mysql_collect_processlist","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yansheng836%2Fmysql_collect_processlist","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yansheng836%2Fmysql_collect_processlist/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yansheng836%2Fmysql_collect_processlist/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yansheng836%2Fmysql_collect_processlist/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yansheng836","download_url":"https://codeload.github.com/yansheng836/mysql_collect_processlist/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yansheng836%2Fmysql_collect_processlist/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274448077,"owners_count":25287119,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-10T02:00:12.551Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mysql","shell","sql","tool"],"created_at":"2025-09-10T10:43:07.142Z","updated_at":"2026-05-19T10:02:23.597Z","avatar_url":"https://github.com/yansheng836.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mysql_collect_processlist\n\n![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/yansheng836/mysql_collect_processlist/shell-ci.yml?style=flat\u0026label=build%3A%20shell-ci) ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/yansheng836/mysql_collect_processlist/.github%2Fworkflows%2Fmysql-ci-8.0.43.yml?style=flat\u0026label=build%3A%20MySQL-ci-8.0.43) ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/yansheng836/mysql_collect_processlist/.github%2Fworkflows%2Fshell-ci.yml?style=flat\u0026label=build%3A%20MySQL-ci-5.7-9.4) ![GitHub commit activity](https://img.shields.io/github/commit-activity/m/yansheng836/mysql_collect_processlist) [![GitHub Issues](https://img.shields.io/github/issues/yansheng836/mysql_collect_processlist)](https://github.com/yansheng836/mysql_collect_processlist/issues) [![GitHub Pull Requests](https://img.shields.io/github/issues-pr/yansheng836/mysql_collect_processlist)](https://github.com/yansheng836/mysql_collect_processlist/pulls) [![GitHub Tag](https://img.shields.io/github/v/tag/yansheng836/mysql_collect_processlist)](https://github.com/yansheng836/mysql_collect_processlist/tags) [![GitHub Release](https://img.shields.io/github/v/release/yansheng836/mysql_collect_processlist)](https://github.com/yansheng836/mysql_collect_processlist/releases) ![GitHub Repo stars](https://img.shields.io/github/stars/yansheng836/mysql_collect_processlist) ![GitHub forks](https://img.shields.io/github/forks/yansheng836/mysql_collect_processlist) [![Codacy Badge](https://app.codacy.com/project/badge/Grade/be2e1300a69341068cc34505a1547b32)](https://app.codacy.com/gh/yansheng836/mysql_collect_processlist/dashboard?utm_source=gh\u0026utm_medium=referral\u0026utm_content=\u0026utm_campaign=Badge_grade) [![GitHub License](https://img.shields.io/github/license/yansheng836/mysql_collect_processlist)](https://github.com/yansheng836/mysql_collect_processlist/blob/main/LICENSE.txt)\n\n这是一个纯脚本工具，用于从MySQL的`information_schema.processlist`视图中定期收集数据并保存到本地日志文件。支持MYSQL5.7-9.4版本。\n\ntemplate copy from PostgreSQL version : \u003chttps://github.com/yansheng836/pg_collect_pgsa\u003e .\n\n**相关背景**：\n\n1. 某个慢SQL打满内存，导致系统kill掉mysql的进程，进而导致mysql进程重启，没有现场排查不了具体原因。（即使开启了慢SQL日志，没有执行完也不会记录到数据库日志中）\n2. 数据库连接数被打满，MYSQL相关监控数据丢失（因为也连不上数据库了），没有现场，不知道异常请求来源。\n\n**特性**：\n\n- 定期收集MySQL活动会话信息\n- 支持通过定时任务配置收集频率\n- 提供日志文件自动分割功能\n- 包含丰富的日志分析示例\n\n## 安装指南\n\n拉取代码，修改参数，设置定时任务。\n\n```shell\n# 克隆代码\ngit clone git@github.com:yansheng836/mysql_collect_processlist.git\ncd mysql_collect_processlist\n\n# 修改必要参数(均以 MYSQL_ 开头，例如：MYSQL_PATH、MYSQL_HOST 等)\nvi mysql_collect_processlist.sh\n\n# 查路径\npwd\n\n# crontab -e\n# 每分钟执行\n* * * * * sh pwd路径/mysql_collect_processlist.sh\n\n# 如果需要更频繁，比如每5秒执行（可自行调整秒数）\n* * * * * sh pwd路径/mysql_collect_processlist_gap_second.sh 5\n```\n\n## 日志文件内容\n\n测试版本：mysql  Ver 8.0.43 for Linux on x86_64 (MySQL Community Server - GPL)\n\nCI版本：MYSQL5.7-9.4\n\n输出字段为：now(),ID,USER,HOST,DB,COMMAND,TIME,STATE,INFO\n\n```plain\n2025-09-09 14:44:02|42|root|localhost|mysql|Query|0|executing|SELECT now(),ID,USER,HOST,DB,COMMAND,TIME,STATE,INFO from information_schema.processlist\n2025-09-09 14:44:02|5|event_scheduler|localhost|NULL|Daemon|8000|Waiting on empty queue|NULL\n```\n\n## 日志分析参考\n\n### 1.简单检索\n\n```shell\n# cat/more/less/grep \ngrep 'executing' processlist.log\n\n# 查找具体时间的相关日志\ngrep '2025-09-04 12:59' processlist.log\n\n# 在归档日志中，查找具体时间的相关日志\nzless logs/processlist-20250904-12.log.gz  | grep '2025-09-04 12:59'\n```\n\n### 2.统计不同状态的语句的数量\n\n```shell\n# 第8列是状态：state\nawk -F '|' '{print $8}' processlist.log | sort | uniq -c\n      1 executing\n      1 Waiting on empty queue\n```\n\n### 3.按照时间统计\n\n```shell\n# 按天统计\nawk -F '|' '{print $1}' processlist.log | cut -d ' ' -f1 | sort | uniq -c\n      2 2025-09-09\n# 按小时统计\nawk -F '[| ]' '{print $1 \" \" $2}' processlist.log | cut -d: -f1 | sort | uniq -c\n      2 2025-09-09 14\n# 按分钟统计\nawk -F '[| ]' '{print $1 \" \" $2}' processlist.log | cut -d: -f1-2 | sort | uniq -c\n      2 2025-09-09 14:44\n```\n\n## 注意事项\n\n1. 在业务繁忙的数据库上使用时，需要注意日志文件可能会快速增长，建议在特殊情况下短暂使用，并密切关注磁盘空间。\n2. `info`字段的长度受MySQL常量`PROCESS_LIST_INFO_WIDTH`限制，默认为65535（写死在源码中），超出部分会被截断。（足以满足绝大多数场景。）\n3. 账号权限问题，可不使用mysql。推荐最小权限：[创建空库，]创建普通用户，授予`PROCESS`权限即可。\n      ```sql\n      -- CREATE DATABASE processlist_db;\n      GRANT PROCESS ON *.* TO 'processlist_user'@'hostname';\n      ```\n\n## 贡献\n\n欢迎提交bug报告或功能需求：\n\nGitHub：\u003chttps://github.com/yansheng836/mysql_collect_processlist/issues\u003e\n\nGitee：\u003chttps://gitee.com/yansheng0083/mysql_collect_processlist/issues\u003e\n\n## License\n\n使用 MIT License。有关详细信息，请参阅 [LICENSE.txt](./LICENSE.txt) 文件。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyansheng836%2Fmysql_collect_processlist","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyansheng836%2Fmysql_collect_processlist","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyansheng836%2Fmysql_collect_processlist/lists"}