{"id":46147,"url":"https://github.com/zhangysh1995/awesome-database-testing","name":"awesome-database-testing","description":"数据库测试资料 This repo is a collection of resources on testing database systems","projects_count":42,"last_synced_at":"2026-09-10T23:00:19.604Z","repository":{"id":55145499,"uuid":"229693680","full_name":"zhangysh1995/awesome-database-testing","owner":"zhangysh1995","description":"数据库测试资料 This repo is a collection of resources on testing database systems","archived":false,"fork":false,"pushed_at":"2025-06-23T05:44:43.000Z","size":82,"stargazers_count":171,"open_issues_count":0,"forks_count":18,"subscribers_count":17,"default_branch":"master","last_synced_at":"2026-08-22T05:09:36.523Z","etag":null,"topics":["awesome-list","database","database-testing","papers","query-optimization","sql","testing"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhangysh1995.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-12-23T06:48:25.000Z","updated_at":"2026-08-09T21:00:55.000Z","dependencies_parsed_at":"2024-01-14T09:15:17.253Z","dependency_job_id":"2c6200ea-47b9-4cd6-8f37-cd00036b245d","html_url":"https://github.com/zhangysh1995/awesome-database-testing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zhangysh1995/awesome-database-testing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangysh1995%2Fawesome-database-testing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangysh1995%2Fawesome-database-testing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangysh1995%2Fawesome-database-testing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangysh1995%2Fawesome-database-testing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhangysh1995","download_url":"https://codeload.github.com/zhangysh1995/awesome-database-testing/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhangysh1995%2Fawesome-database-testing/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":341189360,"owners_count":37206660,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-08-22T15:14:58.755Z","status":"online","status_checked_at":"2026-09-10T02:00:06.818Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"created_at":"2024-01-14T06:42:30.641Z","updated_at":"2026-09-10T23:00:19.605Z","primary_language":null,"list_of_lists":false,"displayable":true,"categories":["High-level Views","Input Generation","Engine Correctness","Performance","Industry Practice"],"sub_categories":["Research Papers","Others","Tools"],"readme":" \n# Awesome-Database-Testing\nThis is a curitated list of resources on *database testing*.\n\n**What covered: papers**, blogs, notes, tools and etc. for whoever wants to test a database mangement system (DBMS). \n\n**What is not**: using SQLs, learning database basic knowledge, implementing a database system.\n\n**Note**: this is NOT an exhaustive list of materials, you may want to find more accroding to references of each item.\n\n\n## High-level Views\nHere we put materials with general discussions.\n\n### Research Papers\n**NOTE:** We only include peer-reviewed and published papers here, the same also applies to other entries. You may want to find the authors' *free-version* of the papers on their personal page.\n\n*  *Understanding the query optimization*  [Query Optimization in Database Systems](https://dl.acm.org/doi/10.1145/356924.356928) [1984]\n*  *Understanding the query optimization - a more recent view*  [An Overview of Query Optimization](https://www2.cs.duke.edu/courses/fall19/compsci516/Papers/chaudhuri98.pdf) [1998]\n*  *What are the problems from the indutrial view*  [Testing SQL Server's Query Optimizer : Challenges , Techniques and Experiences](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.143.3767)\n\n*  *Optimizer Evaluation*  [How good are query optimizers, really?](https://www.vldb.org/pvldb/vol9/p204-leis.pdf)\n\n### Others\n*  *What are the problems from the academic view*  [Is Query Optimization a “Solved” Problem?](https://wp.sigmod.org/?p=1075)\n\n\n\n## Input Generation\n\n### Research Papers\n*  *Query generation with template substitution*  [Generating Thousand Benchmark Queries in Seconds](http://linkinghub.elsevier.com/retrieve/pii/B9780120884698500917)\n*  *Language for customized data generator*  [Flexible Database Generators](https://www.csd.uoc.gr/~hy460/pdf/Flexible%20Database%20Generators.pdf)\n*  *RAGS Microsoft SQL Server*  [Massive Stochastic Testing of SQL](https://www.microsoft.com/en-us/research/publication/massive-stochastic-testing-of-sql/)\n*  *Generate data regarding the query constraints*  [QAGen: Generating Query-Aware Test Databases](http://portal.acm.org/citation.cfm?doid=1247480.1247520)\n*  *How could symbolic execution help*  [Qex: Symbolic SQL Query Explorer](http://link.springer.com/10.1007/978-3-642-17511-4_24)\n*  *Data generation as a search problem*  [Search-based test data generation for SQL queries](http://dl.acm.org/citation.cfm?doid=3180155.3180202)\n*  *For more valid inputs!* [SQUIRREL: Testing Database Management Systems with Language Validity and Coverage Feedback](https://arxiv.org/abs/2006.02398) [CCS 2021]\n\n### Tools\n*  Randgen [MySQL version (not maintained)](https://launchpad.net/randgen) [doc](https://github.com/RQG/RQG-Documentation/wiki/), [MariaDB version](https://github.com/MariaDB/randgen), [PingCAP version](https://github.com/pingcap/go-randgen) \n*  [pquery](https://github.com/Percona-QA/pquery) - stress testing MySQL\n*  [Sysbench](https://dev.mysql.com/downloads/benchmarks.html) - MySQL benchmarks\n*  [SQLsmith](https://github.com/anse1/sqlsmith) - fuzzing style tool\n*  [Squirrel](https://github.com/s3team/Squirrel) - tool for CCS 2021 paper\n*  [mtr_to_sql.sh](https://github.com/Percona-QA/percona-qa/blob/master/mtr_to_sql.sh)@Percona-QA - a script to extract [mysql-test](https://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_MYSQL_TEST_RUN.html) cases as a sql file, with engine replacement and shuffles according to your need\n\n\n\n## Engine Correctness\nHere we mainly refer to the case where the engine returns wrong query result.\n\n### Research Papers\n\n#### What You Really Want to Know About\n*  *RAGS with system-differential testing*  [Massive Stochastic Testing of SQL](https://www.microsoft.com/en-us/research/publication/massive-stochastic-testing-of-sql/)\n*  *Data generation with contraints sovling*  [Query-Aware Test Generation Using a Relational Constraint Solver](http://ieeexplore.ieee.org/document/4639327/)\n*  *Differential testing for the optimizing rules*  [A framework for testing query transformation rules](http://portal.acm.org/citation.cfm?doid=1559845.1559874)\n*  *Criteria on integrtity testing*  [The Effectiveness of Test Coverage Criteria for Relational Database Schema Integrity Constraints](http://dl.acm.org/citation.cfm?doid=2852270.2818639)\n\n\n#### What You Want to Know About\n*  *Parallelism on the old machines*  [Quickly generating billion-record synthetic databases](http://portal.acm.org/citation.cfm?doid=191843.191886)\n*  *Verified database system*  [Toward a Verified Relational Database Management System](http://portal.acm.org/citation.cfm?doid=1706299.1706329)\n*  *Reply the transactions*  [Debugging transactions and tracking their provenance with reenactment]()\n\n#### What's New\n*  [Search-based test data generation for SQL queries](http://dl.acm.org/citation.cfm?doid=3180155.3180202) [2018]\n*  [Automated verification of query equivalence using satisfiability modulo theories](http://dl.acm.org/citation.cfm?doid=3342263.3360343) [2019]\n*  [Detecting Optimization Bugs in Database Engines via Non-Optimizing Reference Engine Construction](https://www.manuelrigger.at/preprints/NoREC.pdf\n) [2020]\n*  [Testing query execution engines with mutations](https://dl.acm.org/doi/10.1145/3395032.3395322) [2020]\n\n### Tools\n*  [SQLancer](https://github.com/sqlancer/sqlancer)@ETH_ZURICH\n*  [go-sqlancer](https://github.com/chaos-mesh/go-sqlancer)@PingCAP\n*  [pquery](https://github.com/Percona-QA/pquery)@Percona-QA - a tool providing the ability to combine correctness and crash recovery testing on both single-node and clustered instances\n\n\n\n## Performance\nHere we focus on detecting and debugging performance issues.\n\n### Research Papers\n\n#### What You Really Want to Know About\n*  *Very first system*  [Efficient testing of high performance transaction processing systems](http://www.vldb.org/conf/1997/P595.PDF)\n*  *Is the exact estimation possible*  [Exact Cardinality Query Optimization for Optimizer Testing](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.151.6265)\n*  *How to identify issues on plan generation*  [On the stability of plan costs and the costs of plan stability](http://dl.acm.org/doi/10.14778/1920841.1920983)\n*  *What's wrong with the models*  [Testing cardinality estimation models in SQL server](http://dl.acm.org/citation.cfm?doid=2304510.2304526)\n\n#### What You Want to Know About\n*  *Generate and compare the plans*  [Testing the accuracy of query optimizers](http://dl.acm.org/citation.cfm?doid=2304510.2304525)\n\n#### What's New\n*  [APOLLO: Automatic Detection and Diagnosis of Performance Regressions in Database Systems](https://dl.acm.org/doi/10.14778/3357377.3357382) [2020]\n\n### Tools\n*  [TPC Benchmarking](http://www.tpc.org/)\n*  [APOLLO](https://github.com/sslab-gatech/apollo)@gatech\n\n\n## Industry Practice\nWe list idea and tools which are adopted in the industry.\n\n### Tools\n* [Chaos Mesh](https://github.com/chaos-mesh/chaos-mesh) from PingCAP\n  *  Article: [Building an Automated Testing Framework Based on Chaos Mesh® and Argo](https://pingcap.com/blog/building-automated-testing-framework-based-on-chaos-mesh-and-argo)\n  *  Talk: [Testing TiDB Using Chaos Mesh® with TiPocket](https://www.youtube.com/watch?v=60uCsnwNdU0)\n\n\n## Contributors\n*  [zhangysh1995](https://github.com/zhangysh1995)\n*  [mkx22](https://github.com/mkx22)\n\n# Contributions are Super Welcome!\nWhat you can contribute:\n*  papers (only peer-reviewed papers will be accepted to this repo)\n*  talks/slides/articles\n*  tools\n\nPlease also add yourself to *Contributors* in the PR.\n\n\n","projects_url":"https://awesome.ecosyste.ms/api/v1/lists/zhangysh1995%2Fawesome-database-testing/projects"}