{"id":13514139,"url":"https://github.com/felixge/sqlbench","last_synced_at":"2025-04-06T10:11:45.639Z","repository":{"id":44088014,"uuid":"296135221","full_name":"felixge/sqlbench","owner":"felixge","description":"sqlbench measures and compares the execution time of one or more SQL queries.","archived":false,"fork":false,"pushed_at":"2021-06-21T08:32:42.000Z","size":1205,"stargazers_count":370,"open_issues_count":2,"forks_count":11,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-30T02:07:22.365Z","etag":null,"topics":["benchmarking","performance","postgresql","sql"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/felixge.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-09-16T19:59:39.000Z","updated_at":"2025-03-06T12:59:51.000Z","dependencies_parsed_at":"2022-07-29T01:38:56.743Z","dependency_job_id":null,"html_url":"https://github.com/felixge/sqlbench","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/felixge%2Fsqlbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/felixge%2Fsqlbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/felixge%2Fsqlbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/felixge%2Fsqlbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/felixge","download_url":"https://codeload.github.com/felixge/sqlbench/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247464222,"owners_count":20942970,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking","performance","postgresql","sql"],"created_at":"2024-08-01T05:00:47.487Z","updated_at":"2025-04-06T10:11:45.619Z","avatar_url":"https://github.com/felixge.png","language":"Go","readme":"# sqlbench\n\nsqlbench measures and compares the execution time of one or more SQL queries.\n\n![screen recording](./recording/recording-min.gif)\n\nThe main use case is benchmarking simple CPU-bound query variants against each other during local development.\n\nOnly PostgreSQL is supported at this point, but pull requests for MySQL or other databases are welcome.\n\n## Install\n\nYou can download a binary from the [release page](https://github.com/felixge/sqlbench/releases).\n\nIf you have Go 1.16 or later installed, you can install or update sqlbench from source:\n\n```\n$ go get -u github.com/felixge/sqlbench\n```\n\n**Windows Users:** You may also install the [chocolatey package](https://chocolatey.org/packages/sqlbench) maintained by [picolino](https://github.com/picolino):\n\n```\n$ choco install sqlbench\n```\n\n### Install via brew\n\nIf you're macOS user and using [Homebrew](https://brew.sh/), you can install via brew command:\n\n```sh\n$ brew update\n$ brew install sqlbench\n```\n\n## Examples\n\nBelow are a few one-liners to get you started. They assume you're running sqlbench from the directory of a clone of this repo.\n\n```bash\n# Benchmark a few queries until ctrl+c is hit. Output results in realtime.\nsqlbench examples/sum/*.sql\n\n# Benchmark using client wallclock time (instead of explain) until ctrl+c.\nsqlbench -m client examples/sum/*.sql\n\n# Run for 3 seconds and only print results once at the end.\nsqlbench -t 3 -s examples/sum/*.sql\n\n# Run for 1000 iterations and only print verbose results once at the end\nsqlbench -n 1000 -s -v examples/sum/*.sql\n\n# Record the results for 1000 iterations into a csv file.\nsqlbench -n 1000 -o baseline.csv examples/sum/*.sql\n\n# Compare 1000 iterations to a baseline recording.\nsqlbench -n 1000 -i baseline.csv examples/sum/*.sql\n```\n\n## Usage\n\n```\nUsage of sqlbench:\n  -c string\n    \tConnection URL or DSN for connecting to PostgreSQL as understood by pgx [1].\n    \tE.g.: postgres://user:secret@localhost:5432/my_db?sslmode=disable\n\n    \tAlternatively you can use standard PostgreSQL environment variables [2] such as\n    \tPGHOST, PGPORT, PGPASSWORD, ... .\n\n    \t[1] https://pkg.go.dev/github.com/jackc/pgx/v4/stdlib?tab=doc\n    \t[2] https://www.postgresql.org/docs/current/libpq-envars.html\n    \t(default \"postgres://\")\n  -i string\n    \tInput path for CSV file with baseline measurements.\n  -m string\n    \tMethod for measuring the query time. One of: \"client\", \"explain\" (default \"explain\")\n  -n int\n    \tTerminate after the given number of iterations. (default -1)\n  -o string\n    \tOutput path for writing individual measurements in CSV format.\n  -p\tInclude the query planning time. For -m explain this is accomplished by adding\n    \tthe \"Planning Time\" to the measurement. For -m client this is done by not using\n    \tprepared statements.\n  -s\tSilent mode for non-interactive use, only prints stats once after terminating.\n  -t float\n    \tTerminate after the given number of seconds. (default -1)\n  -v\tVerbose output. Print the content of all SQL queries, as well as the\n    \tPostgreSQL version.\n  -version\n    \tPrint version and exit.\n```\n\n### How It Works\n\nsqlbench takes a list of SQL files and keeps executing them sequentially, measuring their execution times. By default the execution time is measured by prefixing the query with `EXPLAIN (ANALYZE, TIMING OFF)` and capturing the total `Execution Time` for it.\n\nThe query columns are ordered by mean execution time in ascending order, and the relative difference compared to the fastest query is shown in parentheses. If you provide a baseline csv via `-i`, the relative differences are comparing the corresponding queries in the baseline rather than the current queries with each other.\n\nIf the `-m client` flag is given, the time is measured using the wallclock time of sqlbench which includes network overhead.\n\nPlanning time is excluded by default, but can be included using the `-p` flag.\n\nThe filenames `init.sql` and `destroy.sql` are special, and are executed once before and after the benchmark respectively. They can be used to setup or teardown tables, indexes, etc..\n\n## Tutorial\n\nLet's say you want to compare three different queries for computing the running total of all numbers from 1 to 1000. Your first idea is to use a window function:\n\n```sql\nSELECT i, sum(i) OVER (ORDER BY i) AS sum\nFROM generate_series(1, 1000) g(i);\n```\n\nThen you decide to get fancy and implement it as a recursive CTE:\n\n```sql\nWITH RECURSIVE sums AS (\n\tSELECT 1 AS i, 1 AS sum\n\tUNION\n\tSELECT i+1, sum+i FROM sums WHERE i \u003c= 1000\n)\n\nSELECT * FROM sums;\n```\n\nAnd finally you become wise and remember that [9 year old Gauss](https://www.nctm.org/Publications/Teaching-Children-Mathematics/Blog/The-Story-of-Gauss/) could probably beat both approaches:\n\n```sql\nSELECT i, (i * (i + 1)) / 2 AS sum\nFROM generate_series(1, 1000) g(i);\n```\n\nNow that you have your queries in `window.sql`, `recursive.sql`, `gauss.sql`, you want to summarize the performance differences for your colleagues. However, you know they're a pedantic bunch, and will ask you annoying questions such as:\n\n- How many times did you run each query?\n- Were you running other stuff on your laptop in the background?\n- How can I reproduce this on my local machine?\n- What version of PostgreSQL were you running on your local machine?\n- Are you sure you're not just measuring the overhead of `EXPLAIN ANALYZE`?\n\nThis could normally be quite annoying to deal with, but luckily there is sqlbench. The command below lets you run your three queries 1000 times with `EXPLAIN ANALYZE` and report the statistics, the PostgreSQL version and even the SQL of your queries:\n\n```\n$ sqlbench -v -s -n 1000 examples/sum/*.sql | tee explain-bench.txt\n```\n\n```\n         | gauss |    window     |   recursive\n---------+-------+---------------+----------------\n  n      |  1000 |          1000 |          1000\n  min    |  0.35 | 1.31 (3.79x)  | 1.80 (5.22x)\n  max    |  4.18 | 23.76 (5.68x) | 11.41 (2.73x)\n  mean   |  0.50 | 1.94 (3.85x)  | 2.67 (5.30x)\n  stddev |  0.16 | 0.81 (4.93x)  | 0.63 (3.87x)\n  median |  0.53 | 2.02 (3.80x)  | 2.91 (5.49x)\n  p90    |  0.67 | 2.53 (3.80x)  | 3.41 (5.12x)\n  p95    |  0.68 | 2.57 (3.81x)  | 3.50 (5.18x)\n\nStopping after 1000 iterations as requested.\n\npostgres version: PostgreSQL 11.6 on x86_64-apple-darwin16.7.0, compiled by Apple LLVM version 8.1.0 (clang-802.0.42), 64-bit\nsqlbench -v -s -n 1000 examples/sum/gauss.sql examples/sum/recursive.sql examples/sum/window.sql\n\n==\u003e examples/sum/gauss.sql \u003c==\nSELECT i, (i * (i + 1)) / 2 AS sum\nFROM generate_series(1, 1000) g(i);\n\n==\u003e examples/sum/window.sql \u003c==\nSELECT i, sum(i) OVER (ORDER BY i) AS sum\nFROM generate_series(1, 1000) g(i);\n\n==\u003e examples/sum/recursive.sql \u003c==\nWITH RECURSIVE sums AS (\n\tSELECT 1 AS i, 1 AS sum\n\tUNION\n\tSELECT i+1, sum+i FROM sums WHERE i \u003c= 1000\n)\n\nSELECT * FROM sums;\n```\n\nAnd finally, you can use the `-m client` flag to measure the query times without `EXPLAIN ANALYZE` to see if that had a significant overhead:\n\n```\n$ sqlbench -s -n 1000 -m client examples/sum/*.sql | tee client-bench.txt\n```\n\n```\n         | gauss |    window    |  recursive\n---------+-------+--------------+---------------\n  n      |  1000 |         1000 |         1000\n  min    |  0.66 | 1.44 (2.18x) | 2.03 (3.08x)\n  max    |  5.66 | 7.31 (1.29x) | 4.34 (0.77x)\n  mean   |  0.83 | 1.72 (2.08x) | 2.35 (2.83x)\n  stddev |  0.23 | 0.33 (1.41x) | 0.27 (1.18x)\n  median |  0.78 | 1.65 (2.11x) | 2.26 (2.89x)\n  p90    |  0.98 | 1.98 (2.03x) | 2.68 (2.75x)\n  p95    |  1.05 | 2.13 (2.03x) | 2.89 (2.76x)\n\nStopping after 1000 iterations as requested.\n```\n\nIndeed, it appears that from the client's perspective the gauss query is a bit slower, while the others are a bit faster when measuring without `EXPLAIN ANALYZE`. Whether that's a rabbit hole worth exploring depends on you, but either way you now have a much better sense of the errors that might be contained in your measurements.\n\n## Todos\n\nBelow are a few ideas for todos that I might implement at some point or would welcome as pull requests.\n\n- [ ] Dynamically adjust unit between ms, s, etc.\n- [ ] Support specifying benchmarks using a single YAML file.\n- [ ] Support for other databases, e.g. MySQL.\n- [ ] Capture query plans for each query, ideally one close to the median execution time.\n- [ ] Provide an easy way to capture all inputs and outputs in a single tar.gz file or GitHub gist.\n- [ ] Plot query times as a histogram (made a proof of concept for this, but didn't like it enough yet to release)\n- [ ] Maybe add db name to verbose output, [see request](https://twitter.com/breinbaas1/status/1308138210606940160).\n- [x] Compare benchmark results between PG versions\n- [x] Oneliner examples for README\n- [x] Warmup phase (can be done via init.sql and pg_prewarm()\n- [x] Use `TIMING OFF` to reduce EXPLAIN overhead.\n- [x] A flag to include planning time in `-m explain` mode.\n- [x] A flag to use prepared queries in `-m client` mode.\n\n## License\n\nsqlbench is licensed under the MIT license.\n","funding_links":[],"categories":["开源类库","Go","Open source library"],"sub_categories":["测试","Test"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffelixge%2Fsqlbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffelixge%2Fsqlbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffelixge%2Fsqlbench/lists"}