{"id":16812918,"url":"https://github.com/sjmudd/queryprofiler","last_synced_at":"2025-07-02T00:10:06.554Z","repository":{"id":43327212,"uuid":"48158936","full_name":"sjmudd/queryprofiler","owner":"sjmudd","description":"Capture query profiles of one or more MySQL servers using performance_schema","archived":false,"fork":false,"pushed_at":"2015-12-29T00:59:32.000Z","size":48,"stargazers_count":20,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-11T01:46:47.980Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sjmudd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-12-17T07:19:33.000Z","updated_at":"2024-04-23T07:57:44.000Z","dependencies_parsed_at":"2022-09-07T02:50:15.866Z","dependency_job_id":null,"html_url":"https://github.com/sjmudd/queryprofiler","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/sjmudd/queryprofiler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjmudd%2Fqueryprofiler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjmudd%2Fqueryprofiler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjmudd%2Fqueryprofiler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjmudd%2Fqueryprofiler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sjmudd","download_url":"https://codeload.github.com/sjmudd/queryprofiler/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjmudd%2Fqueryprofiler/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263052430,"owners_count":23406106,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T10:23:58.813Z","updated_at":"2025-07-02T00:10:06.511Z","avatar_url":"https://github.com/sjmudd.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# queryprofiler\nCompare query profiles of 2 different servers by querying performance_schema.events_statements_summary_by_digest\n# overview\nThe intention here is to connect to two servers and collect information from the digest table\nin parallel.\nThis will do the following:\n* generate n collections of query digests for each server.\n* From this data we can collect n-1 samples which are based on collection x compared against collection x-1.\n* Finally we find the top Z queries for server1\n* With each query on server1 attempt to compare each of these queries with server2, comparing metrics using the available samples.\n\nNote: this is still work in progress and not completed.\n# Usage\n\n```\nqueryprofiler [\u003coptions\u003e] \u003cdsn1\u003e [\u003cdsn2\u003e ...]\n```\n\n```\nDSN1='user:password@tcp(server1.example.com:3306)/performance_schema'\nDSN2='user:password@tcp(server2.example.com:3306)/performance_schema'\n./queryprofiler \"$DSN1\" \"$DSN2\"\n```\n\n# Concepts\n\nIn theory using P_S to profile the queries may seem quite simple,\nbut I think that to get useful values it requires a little more attention.\nThe sections below describe how queryprofiler analyses the queries\non the server.\n\n## Event\n\nEvent is the table represetation of the P_S digest table.\n\n## Collection\n\nCollection is a slice of Events together with a timestamp of when\ndata was collected.  Collections is a slice of Collection.\n\n## Sample\n\nSample is a slice of rows that come from subtracting _matching_\nvalues by Key and recording the start time and duration of the\nsample. It contains several rows for different queries.  Samples\nis a slice of Sample.  Sample metrics are normalised to metrics\nper second for consistency.\n\n## Metric\n\nThis is a slice of float64, which is the underlying numbers used\nby this program. Thus a sample really contains a named set of\nMmetric.\n\n## Key\n\nIn theory the QUERY_DIGEST might be used but this digest is not\nstable between different MySQL versions so I collect an MD5 digest\nof the DIGEST_TEXT.  That said the DIGEST is not a unique key,\nwhat's unique is a combination of query (digest) and SCHEMA_NAME,\nso the Key considered as the key of queries is based on the MD5_DIGEST\nand the SCHEMA_NAME, joined by a \".\". if SCHEMA_NAME contains a\nvalue.\n\n## Issues\n\n* events_statements_summary_by_digest may have empty DIGEST/DIGEST_TEXT. This represents lost values because the maximum number of digest values has been exceeded. You may see this empty query having quite high values because of this.\n\n* Only completed queries are shown. Any long query that is running while queryprofiler is looking for data won't be shown.\n\n* events_statements_summary_by_digest should have only one row per DIGEST_TEXT / SCHEMA_NAME. Unfortunately I've seen that this is not the case and multiple row may be present. This has been reported. See http://bugs.mysql.com/bug.php?id=79533. In the meantime if multiple rows are found with the same DIGEST_TEXT/SCHEMA_NAME the values are merged together.\n\n* events_statements_summary_by_digest has a DIGEST column which represents a unique key (with the SCHEMA_NAME) to identify queries. However, this digest may not be the same for the same query on 2 different servers due to the way the optimiser works. Consequently queryprofiler takes an MD5 checksum of the QUERY_TEXT and uses that instead. I should really file a feature requesting that the generated query digest is calculated consistently as that would avoid this extra operation.\n\n# Sample Output\n\nFor sample output look at the file of the same name: sample-output.txt\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsjmudd%2Fqueryprofiler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsjmudd%2Fqueryprofiler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsjmudd%2Fqueryprofiler/lists"}