{"id":21812178,"url":"https://github.com/cou929/sql-fingerprint-js","last_synced_at":"2026-05-07T20:03:14.332Z","repository":{"id":57689347,"uuid":"494518798","full_name":"cou929/sql-fingerprint-js","owner":"cou929","description":"Converts a SQL into a fingerprint. A JavaScript port of pt-fingerprint.","archived":false,"fork":false,"pushed_at":"2022-05-24T00:25:43.000Z","size":46,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-31T00:26:51.936Z","etag":null,"topics":["javascript","mysql","sql"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/sql-fingerprint","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cou929.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-05-20T15:36:20.000Z","updated_at":"2022-05-24T00:27:00.000Z","dependencies_parsed_at":"2022-09-26T20:53:28.987Z","dependency_job_id":null,"html_url":"https://github.com/cou929/sql-fingerprint-js","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cou929/sql-fingerprint-js","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cou929%2Fsql-fingerprint-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cou929%2Fsql-fingerprint-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cou929%2Fsql-fingerprint-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cou929%2Fsql-fingerprint-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cou929","download_url":"https://codeload.github.com/cou929/sql-fingerprint-js/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cou929%2Fsql-fingerprint-js/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32753938,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-07T02:14:30.463Z","status":"ssl_error","status_checked_at":"2026-05-07T02:14:29.405Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["javascript","mysql","sql"],"created_at":"2024-11-27T14:14:32.989Z","updated_at":"2026-05-07T20:03:14.305Z","avatar_url":"https://github.com/cou929.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sql-fingerprint\n\nConverts a SQL into a fingerprint, the abstracted form of a query, which makes it possible to classify similar queries.\n\nActually, this module is a JavaScript port of [pt-fingerprint](https://www.percona.com/doc/percona-toolkit/LATEST/pt-fingerprint.html). Therefore is primarily intended to use for MySQL queries.\n\n## Description\n\nThe fingerprint is the abstracted form of a query. What \"abstracting query\" is string conversion such that, for instance, replace values to `?`, collapse whitespaces and so on.\n\nLet's say that there are such queries:\n\n```sql\nSELECT * FROM t WHERE i = 1 ORDER BY a, b ASC, d DESC, e ASC;\n\nselect *         from t       where i = 1\n order by a, b ASC, d DESC, e asc;\n```\n\nA fingerprint of both of these will be below:\n\n```sql\nselect * from t where i = ? order by a, b, d desc, e;\n```\n\n## Usage\n\n### CLI\n\n```sh\nnpm install -g sql-fingerprint\n\nfingerprint --query=\"your query\"\n```\n\n### Module\n\n```sh\nnpm install sql-fingerprint\n```\n\n```js\nimport fingerprint from 'sql-fingerprint';\n\nconsole.log(fingerprint('SELECT * FROM users WHERE id = 1', false, false));\n```\n\n### BigQuery UDF\n\nThe initial motivation I wrote this was that classifies similar queries stored in my dataset of BigQuery. BigQuery supports [user-defined functions written in JavaScript](https://cloud.google.com/bigquery/docs/reference/standard-sql/user-defined-functions#javascript-udf-structure). Therefore the approach of pt-fingerprint, which is generate fingerprint by a set of RegExps, is more fit than other approaches such as parsing SQL, building AST and then interpreting it. RegExps approach seems a little bit harder to maintain but easier to use as UDF because of the relatively simple source code.\n\n```sql\nCREATE TEMP FUNCTION fingerprint(sql STRING, matchMD5Checksum BOOL, matchEmbeddedNumbers BOOL)\nRETURNS STRING\nLANGUAGE js AS r\"\"\"\n\n  // copy and paste fingerprint function here from fingerprint.js\n  // ...\n\n  return query;\n\"\"\";\n\nSELECT\n  fingerprint(textPayload, true, true) fp,\n  count(*) as num,\n  max(query) as raw_query_sample\nFROM\n  `your_table`\nWHERE\n  DATE(timestamp, \"Asia/Tokyo\") = \"2022-05-22\"\ngroup by\n  fp\norder by\n  num desc\n```\n\nSee also: [Standard SQL user\\-defined functions  \\|  BigQuery  \\|  Google Cloud](https://cloud.google.com/bigquery/docs/reference/standard-sql/user-defined-functions#javascript-udf-structure)\n\n## Options\n\n- `matchMD5Checksum`\n    - Replace md5 string to single `?`\n\n```sql\n-- original\nSELECT * FROM db.fbc5e685a5d3d45aa1d0347fdb7c4d35_temp where id=1\n-- fingerprint with matchMD5Checksum=true\nselect * from db.?_temp where id=?\n-- fingerprint with matchMD5Checksum=false (default)\nselect * from db.fbc?_temp where id=?\n```\n\n- `matchEmbeddedNumbers`\n    - Preserve numbers within words. Useful for the case like that table name contains a number\n\n```sql\n-- original\nSELECT * FROM prices.rt_5min WHERE id = 1\n-- fingerprint with matchEmbeddedNumbers=true\nselect * from prices.rt_5min where id = ?\n-- fingerprint with matchEmbeddedNumbers=false (default)\nselect * from prices.rt_?min where id = ?\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcou929%2Fsql-fingerprint-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcou929%2Fsql-fingerprint-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcou929%2Fsql-fingerprint-js/lists"}