{"id":25363566,"url":"https://github.com/tcd93/mssql-execution-plan","last_synced_at":"2025-04-09T04:19:45.242Z","repository":{"id":137012160,"uuid":"342901449","full_name":"tcd93/MSSQL-execution-plan","owner":"tcd93","description":"An article about Microsoft SQL Server's plan optimizer","archived":false,"fork":false,"pushed_at":"2023-09-25T03:24:47.000Z","size":1097,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-14T22:38:52.975Z","etag":null,"topics":["mssqlserver","optimizer"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tcd93.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-27T16:21:11.000Z","updated_at":"2023-09-06T02:26:04.000Z","dependencies_parsed_at":"2023-09-25T04:57:57.609Z","dependency_job_id":null,"html_url":"https://github.com/tcd93/MSSQL-execution-plan","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tcd93%2FMSSQL-execution-plan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tcd93%2FMSSQL-execution-plan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tcd93%2FMSSQL-execution-plan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tcd93%2FMSSQL-execution-plan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tcd93","download_url":"https://codeload.github.com/tcd93/MSSQL-execution-plan/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247975201,"owners_count":21026829,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mssqlserver","optimizer"],"created_at":"2025-02-14T22:35:03.444Z","updated_at":"2025-04-09T04:19:45.226Z","avatar_url":"https://github.com/tcd93.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# MSSQL Plan Optimizer\nAn article is an introduction to Microsoft SQL Server's plan optimizer \u0026 common operators, and provide an more in-depth view \u0026 analysis of slow queries (with example)\n\nLink: https://tcd93.github.io/MSSQL-execution-plan/\n\n__*Disclaimer: as MSSQL is a closed-source database, I can not prove that this article is 100% correct, all is based on articles online (which I'll include links) and my own experience, so take it with a grain of salt \u0026 happy reading!*__\n\nNote: I don't provide sample data here as it's private \u0026 pretty huge, but even if you don't run these data yourself, you should have a pretty good grasp of \nSQL Server's optimizer after reading this\n\nTool: Microsoft SQL Server Management Tool (MSSM)\n\n---\n# Table of contents\n- [Execution Plan \u0026 Optimizer](#execution-plan--optimizer)\n- [Query Processor](#query-processor)\n- [Statistics](#statistics)\n    - [B-tree index](#a-little-about-btree-index)\n- [Common Operators](#common-operators)\n    - [Joins/aggregator](#joins--aggregator)\n        - [Nested loop](#nested-loop)\n        - [Hash join](#hash-join)\n        - [Merge join](#merge-join)\n    - [Making sense of parallelism](#making-sense-of-parallel-scan)\n    - [Why merge is slower than hash in parallel](#comparing-merge--hash-in-parallel-plans)\n\n---\n\n## Execution Plan \u0026 Optimizer\n### What is an Execution Plan?\n![](img/2021-03-01-09-54-56.png)\n\nAn execution plan is a set of physical operations (operators) that can be performed to produce the required result\n\nThe data flow order is from right to left, the thickness of the arrow indicate the amount of data compared to the entire plan; hovering on the icons show extra details\n\n![](img/2021-03-01-09-55-13.png)\n\n### Retrieving the estimated plan\nRetrieving the estimated plan is just telling SQL Server to return the execution plan without actually executing it, helpful in debugging\n\nFrom MSSM (Microsoft SQL Server Management tool), select an SQL block:\n- Press `CTRL + L`\n- or: Right click → Display estimated execution plan\n- or Query → Display estimated execution plan\n\n### Retrieving the actual plan\nIn the query menu, tick the “Include Actual Execution Plan” icon\n\n![](img/2021-03-01-09-53-33.png)\n\nSelect \u0026 run the query, the plan will be open on a new tab next to result tab\n\n### Estimated vs. Actual\nThey can differ in cases where the query involves parallelism, variable, hints, current CPU usage… Actual execution plan contains extra runtime information, such as the actual usage metrics (memory grant, actual rows, executions…), and any runtime warnings\n\n\u003cdiv\u003e\n    \u003cfigure style=\"display:inline-block;margin-left:0;width:40%;\"\u003e\n        \u003cimg src=\"img/2021-03-01-10-03-45.png\"\u003e\n        \u003cfigcaption style=\"font-size:80%;font-style:italic;\"\u003eEstimated\u003c/figcaption\u003e\n    \u003c/figure\u003e\n    vs\n    \u003cfigure style=\"display:inline-block;width:40%;\"\u003e\n        \u003cimg src=\"img/2021-03-01-10-04-03.png\"\u003e\n        \u003cfigcaption style=\"font-size:80%;font-style:italic;\"\u003eActual\u003c/figcaption\u003e\n    \u003c/figure\u003e\n\u003c/div\u003e\n\nActual Plan also include the number of rows processed by each thread, runtime memory allocation...\n\n\u003cdiv\u003e\n    \u003cfigure style=\"display:inline-block;margin-left:0;\"\u003e\n        \u003cimg src=\"img/2021-03-01-10-11-17.png\"\u003e\n    \u003c/figure\u003e\n    \u003cfigure style=\"display:inline-block;margin-left:0;\"\u003e\n        \u003cimg src=\"img/2021-03-01-10-11-09.png\"\u003e\n    \u003c/figure\u003e\n\u003c/div\u003e\n\n\n## Query Processor\nWhat happens when a query is submitted?\n\n\u003cimg style=\"background:white\" src=\"img/2021-03-01-10-18-10.png\"/\u003e\n\nThe `algebrizer` resolves all the names of various objects, tables, and columns referred to within the query string. It identifies at the individual column level, all the data types (varchar, datetime…) for the objects being accessed. It also determines the location of aggregates (SUM, MAX…)\n\nThe `algebrizer` outputs a binary tree which gives the optimizer knowledge of the logical query structure and the underlying tables and indexes, the output also includes a hash representing the query, the optimizer uses it to see if there is already a plan for this stored in plan cache \u0026 whether it’s still valid, if there’s one, then the process stops and __the cached plan is reused__, if not, then it'll compile out an execution plan based on __*statistics*__ \u0026 __*cost*__\n\nOnce the query is optimized, the generated execution plan may be stored in the plan cache and be executed step-by-step by the physical operators in that plan\n\n### Cost of the plan\nThe _estimated cost_ is based on a complex mathematical model, and it considers various factors, such as cardinality, row size, expected memory usage and number of sequential and random I/O operations, parallelism overhead… \n\n__*This number is meaningless outside of the query optimizer's context and should be used for comparison only*__\n\n![](img/2021-03-01-11-38-28.png)\n- _Operator Cost_: Cost taken by the operator\n- _Subtree Cost_: Cumulative cost associated with the whole subtree up to the node\n\n### Ways to select a plan\nThe query optimizer finds a number of candidate execution plans for a given query, estimates the cost of each of these plans and selects the plan with the lowest cost. \n\nFor some queries, the optimizer cannot consider every possible plan for every query, it actually has to consider both the cost of finding potential plans and the costs of plans themselves\n\n![](img/2021-03-01-12-42-44.png)\n![](img/2021-03-01-12-42-54.png)\n\n### Plan cache\nWhenever a query is run for the first time in SQL Server, it is compiled and a query plan is generated for the query. Every query requires a query plan before it is actually executed. This query plan is stored in SQL Server query plan cache, when that query is run again, SQL Server doesn’t need to create another query plan\n\nThe duration that a query plan stays in the plan cache depends upon how often a query is executed. Query plans that are used more often, stay in the query plan cache for longer durations, and vice-versa\n\n_Cache is not used when specific [hints](https://docs.microsoft.com/en-us/sql/t-sql/queries/hints-transact-sql-query?view=sql-server-ver15) are specified (RECOMPILE hint)_\n\n## Statistics\nWhy is it important?\n\n### The data of data\n- The statistics contain information about tables and indexes such as number of rows, the histogram, the density of values from a sample of data; these values are stored in system tables \n- Costs are generated based on statistics, if the stats are incorrect or out-of-date (stale), cost will be wrongly calculated, and the optimizer may choose a sub-optimal plan\n- Statistics can be updated automatically, periodically, or manually\n\n### Histogram\n_Histogram_ measures the frequency of occurrence for each distinct value in a data set\n\nTo create the histogram, SQL server split the data into different buckets (called steps) based on the value of first column of the index. Each record in the output is called as bucket or step\n\nThe maximum number of bucket is 200, this can cause problems for larger set of data, where there can be points of _skewed data distributions_, leading to un-optimized plans for special ranges\n\n\u003cblockquote style=\"font-size:85%\"\u003e\n\nFor example, customer A usually makes 5 purchases per week, but suddenly, at a special day (like Black Friday), he made over 10000 transactions, that huge spike might not get captured in the transaction bucket, and the query for that week would likely get much slower than normal as the `optimizer`'d still think he makes \nvery little purchases in that week\n\n\u003c/blockquote\u003e\n\nIn MSSM, expand _Table \u003e Statistics \u003e Double click a stat name_; some stat names are auto-generated, some are user-defined \n\n![](img/2021-03-01-13-00-42.png)\n\nThis is a sample histogram of column `MasterID` from _Customer_ table:\n\n![](img/2021-03-01-13-03-01.png)\n\nExplanation for the 4th bucket:\n* `RANGE_ROWS`: There are 2861 rows with keys from 4183 - 62833\n* `EQ_ROWS`: There are 272 rows with key 62834\n* `DISTINCT_RANGE_ROWS`: There are 47 distinct rows with keys from 4183 - 62833\n\nNow if we selects 30% of the 4th bucket (`21778` = `(62833 - 4183) * 0.3 + 4183`):\n```sql\nSELECT * FROM Customer WHERE MasterID BETWEEN 4183 AND 21778\n```\n\nThis is the generated plan:\n\n![](img/2021-03-01-13-23-21.png)\n\nThere are 2861 rows from ID 4183 - 62833, so if we’re selecting 30% of that range, it should also results in 30% of 2861 which is 858 rows, that’s the estimated number of the optimizer\n\n### Density\n_Density_ is the ratio of unique values with in the given column or a set of columns\n\n![](img/2021-03-01-13-41-06.png)\n\nLet's go with this query:\n```sql\nDECLARE @N INT = 4178\nSELECT * FROM Customer WHERE MasterID = @N\n```\n\n![](img/2021-03-01-13-44-53.png)\n\n_Histogram cannot be used when we're using parameter_, it then falls back to Density, which is estimated as `Total rows * Density` = `1357786 * 2.020488E-05` = `27.43` rows - but in actuality there is 2134 rows! (as showed in Histogram `EQ_ROWS` attribute). Optimizer failed pretty hard there 🤔\n\n### Memory Grant\n- Memory Grant value (kb) can only be seen in Actual execution mode\n- This memory is used to store temporary rows for sort, hash join \u0026 [parallelism exchange operators](###-parallelism-operators)\n- SQL Server calculates this based on statistics, lack of available memory grant causes a `tempdb` spill ([tempDB](https://docs.microsoft.com/en-us/sql/relational-databases/databases/tempdb-database?view=sql-server-ver15) is a global resource that is used to stores all temporary objects)\n\n\u003cblockquote style=\"font-size:85%\"\u003e\n\nIn SQL server 2012+, a yellow warning icon is displayed in plan explorer when the processor detects a spill (not enough RAM to store data)\n\nFor SQL server 2008R2, check the “sort warnings” event in \u003ca href=\"https://www.sqlshack.com/an-overview-of-the-sql-server-profiler/\"\u003eSQL profiler\u003c/a\u003e to detect memory spill\n\n\u003c/blockquote\u003e\n\n#### TempDB Spill\nBy adding a “order by” clause to the above example, we can produce a _sort warnings_ event in SQL Profiler\n\n![](img/2021-03-01-14-00-23.png)\n\n![](img/2021-03-01-14-03-49.png)\n\nThe engine only granted 1136 KB of memory buffer to perform sorting, but in reality the operation needed way more because actual rows are much higher than estimated returned rows, so the input data has to be split into smaller chunks in _tempDB_ to accommodate the granted space to be sorted, then extra passes are performed to merge these sorted chunks\n\nTo fix this, we can simply add the __RECOMPILE__ hint to the query, this forces the parse to replace the `@N` parameter with actual value, therefore correctly using the Histogram table\n\n---\n\n## A little about B+Tree Index\nIndex is a set of ordered values stored in 8kb pages, the pages form a B+tree structure, and the value contains pointer to the pages in the next level of the tree\n\nThe pages at the leaf nodes can be data pages (clustered index) or index pages (non-clustered index)\n\nClustered index (CI) is the table itself, 1 table can only have 1 CI; NonCI’s leaf may refer to the CI’s key, so _any changes to the CI’s key will force changes to every NonCI’s structures_\n\n![](img/2021-03-01-14-47-26.png)\n\nWith scan, we need to scan 6 pages to reach key 28, whereas going top-down (seek), we just need to read 2 index pages and 1 data page (3 logical/physical reads = 3 * 8kb = 24kb)\n\nSeek \u0026 scan can be combined, where a seek happens first to find where to start scanning, this is still displayed as an _index seek_ operator in plan view\n\n---\n\n## Common Operators\n### Sort\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003cth\u003eIcon\u003c/th\u003e\n        \u003cth\u003eName\u003c/th\u003e\n        \u003cth\u003eDescription\u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-14-14-50.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eSort\u003c/td\u003e\n        \u003ctd\u003eReads all input rows, sorts them, and then returns them in the specified order\u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\nSort is a _blocking_ operation, it has to read all data into RAM, and sort it. It is both time \u0026 memory consuming\n\nIf the data is too big for granted memory, a [spill](####-tempdb-spill) happens, making Sort less efficient\n\n### Data Retrievers\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003cth\u003eIcon\u003c/th\u003e\n        \u003cth\u003eName\u003c/th\u003e\n        \u003cth\u003eDescription\u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-14-22-15.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eIndex seek / Non-clustered index seek\u003c/td\u003e\n        \u003ctd\u003eFinds a specific row in an index, based on key value; and optionally continues to scan from there in logical (index) order\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-14-22-16.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eIndex scan / Non-clustered index scan\u003c/td\u003e\n        \u003ctd\u003eReads all data from an index, either in \u003ca href=\"https://sqlperformance.com/2015/01/t-sql-queries/allocation-order-scans\"\u003eallocation order\u003c/a\u003e or in logical (index) order\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-14-26-54.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eKey lookup\u003c/td\u003e\n        \u003ctd\u003eReads a single row from a clustered index, based on a key that was retrieved from a non-clustered index on the same table.\u003cbr\u003e\n        A Key lookup is a very expensive operation because it performs random I/O into the clustered index.\u003cbr\u003e\n        For every row of the non-clustered index, SQL Server has to go to the Clustered Index to read their data.\u003cbr\u003e\n        We can take advantage of knowing this to improve the query performance\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-14-27-28.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eTable scan\u003c/td\u003e\n        \u003ctd\u003eReads all data from a heap table, in allocation order\u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n### Joins / Aggregator\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003cth\u003eIcon\u003c/th\u003e\n        \u003cth\u003eName\u003c/th\u003e\n        \u003cth\u003eDescription\u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-14-53-57.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eHash match/aggregate\u003c/td\u003e\n        \u003ctd\u003eBuilds a hash table from its first input, then uses that hash table to either join to its second input, or produce aggregated values\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-14-54-32.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003e\n\n[Merge join](####-merge-join)\n\n\u003c/td\u003e\n        \u003ctd\u003eJoins two inputs that are ordered by the join key(s), exploiting the known sort order for optimal processing efficiency\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-14-55-06.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eStream aggregate\u003c/td\u003e\n        \u003ctd\u003eComputes aggregation results by reading a sorted input stream and returning a single row for each set of rows with the same key value\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-14-55-55.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eNested loop\u003c/td\u003e\n        \u003ctd\u003eJoins two inputs by repeatedly executing the second input for each row in the first input\u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n### Parallelism operators\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003cth\u003eIcon\u003c/th\u003e\n        \u003cth\u003eName\u003c/th\u003e\n        \u003cth\u003eDescription\u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-15-01-35.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eDistribute streams\u003c/td\u003e\n        \u003ctd rowspan=\"3\"\u003eThe parallelism operators, also known as exchange operators, manage the distribution of rows between threads in parallel plans\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-15-02-20.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eRepartition streams\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-01-15-02-50.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eGather streams\u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n### Spools\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003cth\u003eIcon\u003c/th\u003e\n        \u003cth\u003eName\u003c/th\u003e\n        \u003cth\u003eDescription\u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-08-15-45-59.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eTable spool\u003c/td\u003e\n        \u003ctd\u003eStores its input rows in an internal worktable; this worktable can then be used to re-process the same data\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003e\u003cimg src=\"img/2021-03-08-15-46-39.png\" style=\"width:50px;height:50px;\"/\u003e\u003c/td\u003e\n        \u003ctd\u003eIndex spool\u003c/td\u003e\n        \u003ctd\u003eStores its input rows in an internal, indexed worktable; this indexed worktable can then be used to re-process specific subsets of the data\u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n---\n\n#### Nested loop\n[Source](https://bertwagner.com/posts/visualizing-nested-loops-joins-and-understanding-their-implications/)\n\n\u003cdiv\u003e\n    \u003cspan style=\"display:inline-block;width:30%;\"\u003e\n        \u003cul\u003e\n            \u003cli\u003eO(n.m) / \u003ci\u003eO(nlog(m))\u003c/i\u003e*\u003c/li\u003e\n            \u003cli\u003eRequire data sorted: No\u003c/li\u003e\n            \u003cli\u003eCPU cost: Low\u003c/li\u003e\n            \u003cli\u003eMemory grant: Maybe\u003c/li\u003e\n            \u003cli\u003eSpill-able?: No\u003c/li\u003e\n            \u003cli\u003eBlocking: No / \u003ci\u003eSemi\u003c/i\u003e**\u003c/li\u003e\n            \u003cli\u003e\n                Optimal for:\n                \u003cul\u003e\n                    \u003cli\u003eSmall outer input → Small/Medium (indexed) inner input\u003c/li\u003e\n                    \u003cli\u003eLow cardinality data\u003c/li\u003e \n                    \u003cli\u003eOLTP\u003c/li\u003e\n                \u003c/ul\u003e\n            \u003c/li\u003e\n        \u003c/ul\u003e\n    \u003c/span\u003e\n    \u003cspan style=\"display:inline-block;\"\u003e\n        \u003cfigure\u003e\n            \u003cimg src=\"https://bertwagner.com/wp-content/uploads/2018/12/Nested-Loop-Join-50fps-1.gif\"\u003e\n            \u003cfigcaption style=\"font-size:80%;font-style:italic;\"\u003ethis gif demonstrates “brute-force” type of NL\u003c/figcaption\u003e\n        \u003c/figure\u003e\n    \u003c/span\u003e\n\u003c/div\u003e\n\n\u003cblockquote style=\"font-size:85%\"\u003e\n\n(*) SQL Server can use multiple ways to optimize a nested loop (to get Big O of \u003ci\u003enlog(m)\u003c/i\u003e time complexity)\n\u003cul\u003e\n    \u003cli\u003e\u003ca href=\"https://sqlserverfast.com/epr/table-spool/#:~:text=The%20Table%20Spool%20operator%20is,operators%20to%20produce%20them%20again.\"\u003eSpool\u003c/a\u003e in inner loop to maximize reusability\u003c/li\u003e\n    \u003cli\u003ePerform index seek on inner loop\u003c/li\u003e\n    \u003cli\u003ePrefetch data in inner loop\u003c/li\u003e\n\u003c/ul\u003e\n\n(**) Order inner loop implicitly to create Semi-blocking nested loop\n\n\u003c/blockquote\u003e\n\n##### Nested loop prefetching (`WithUnorderedPrefetch: True`)\nExample plan:\n\n![](img/2021-03-01-15-29-22.png)\n\nScans `IX_agent` index, for each agent, seek the corresponding customer __asynchronously__ from `IX_custid`, forward the result whenever it's available\n\n\u003cblockquote style=\"font-size:85%\"\u003e\n\nWhen \u003ccode\u003eWithUnorderedPrefetch\u003c/code\u003e is set to False, the index-seek-result result will be forwarded only when the previous ordered key is fetched \u0026 forwarded\n\n\u003c/blockquote\u003e\n\n##### Optimized nested loop (`Optimized: True`)\nExample plan:\n\n![](img/2021-03-01-15-43-34.png)\n\n![](img/2021-03-01-15-48-59.png)\n\n1. Scans `IX_tnx_type` index\n2. May implicitly perform an (partial) \"_order by_\" to create less random seeks; hence the high memory usage\n3. If memory does not fit, it’ll fill what it can, so it does not spill\n\n\u003cblockquote style=\"font-size:85%\"\u003e\n\n\u003cul\u003e\n    \u003cli\u003eAlthough getting just 10 rows, the above plan still requires 189,312 KB of sorting space\u003c/li\u003e\n    \u003cli\u003eConcurrent runs of above query cause high \u003cb\u003eRESOURCE_SEMAPHORE\u003c/b\u003e wait, leading to slower performance (fixed in 2016)\u003c/li\u003e\n    \u003cli\u003eThe sort method \u0026 memory grant algorithm is different to a normal sort operator, there’s no guarantee that it is faster than same query without optimization\u003c/li\u003e\n    \u003cli\u003eThis is treated as a “safety net” in case the statistics are out-of-date\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003c/blockquote\u003e\n\n#### Hash match\n[Source](https://bertwagner.com/posts/hash-match-join-internals/)\n\n\u003cdiv\u003e\n    \u003cspan style=\"display:inline-block;width:30%;\"\u003e\n        \u003cul\u003e\n            \u003cli\u003eO(n + m)\u003c/li\u003e\n            \u003cli\u003eRequire data sorted: No\u003c/li\u003e\n            \u003cli\u003eCPU cost: High\u003c/li\u003e\n            \u003cli\u003eMemory grant: \u003cb\u003eYes\u003c/b\u003e\u003c/li\u003e\n            \u003cli\u003eSpill-able?: Yes\u003c/li\u003e\n            \u003cli\u003eBlocking: Yes\u003c/li\u003e\n            \u003cli\u003e\n                Optimal for:\n                \u003cul\u003e\n                    \u003cli\u003eMedium build input → Medium/Large probe input\u003c/li\u003e\n                    \u003cli\u003eMedium/high cardinality data\u003c/li\u003e\n                \u003c/ul\u003e\n            \u003c/li\u003e\n            \u003cli\u003e\u003cb\u003eScales well with parallelism\u003c/b\u003e\u003c/li\u003e\n        \u003c/ul\u003e\n    \u003c/span\u003e\n    \u003cspan style=\"display:inline-block;\"\u003e\n        \u003cfigure\u003e\n            \u003cimg src=\"https://bertwagner.com/wp-content/uploads/2018/12/Hash-Match-Join-Looping-1.gif\"\u003e\n        \u003c/figure\u003e\n    \u003c/span\u003e\n\u003c/div\u003e\n\n#### Merge join\n[Source](https://bertwagner.com/posts/visualizing-merge-join-internals-and-understanding-their-implications/)\n\n\u003cdiv\u003e\n    \u003cspan style=\"display:inline-block;width:30%;\"\u003e\n        \u003cul\u003e\n            \u003cli\u003eO(n + m)\u003c/li\u003e\n            \u003cli\u003eRequire data sorted: \u003cb\u003eYes\u003c/b\u003e\u003c/li\u003e\n            \u003cli\u003eCPU cost: Low\u003c/li\u003e\n            \u003cli\u003eMemory grant: No\u003c/li\u003e\n            \u003cli\u003eSpill-able?: No\u003c/li\u003e\n            \u003cli\u003eBlocking: No\u003c/li\u003e\n            \u003cli\u003e\n                Optimal for:\n                \u003cul\u003e\n                    \u003cli\u003eEvenly sized inputs\u003c/li\u003e\n                    \u003cli\u003e\u003ca href=\"https://sqlserverfast.com/blog/hugo/2017/12/many-many-reads-many-many-merge-join/\"\u003eOne to many\u003c/a\u003e\u003c/li\u003e\n                \u003c/ul\u003e\n            \u003c/li\u003e\n            \u003cli\u003e\u003cb\u003eScales badly parallelism\u003c/b\u003e\u003c/li\u003e\n        \u003c/ul\u003e\n    \u003c/span\u003e\n    \u003cspan style=\"display:inline-block;\"\u003e\n        \u003cfigure\u003e\n            \u003cimg src=\"https://bertwagner.com/wp-content/uploads/2018/12/Merge-Join-1.gif\"\u003e\n        \u003c/figure\u003e\n    \u003c/span\u003e\n\u003c/div\u003e\n\n#### Making sense of parallel scan\n\nThis is the explain plan produced from the following query:\n```sql\nSELECT [product].id, [tnx_table].amount...\nFROM tnx_table\nINNER JOIN product\nON [tnx_table].prod_id = [product].id\n```\n\n![](img/2021-03-01-16-02-41.png)\n\nFirst, the engine scans the `IX_prod` index, in parallel, the distribution of rows among threads can be considered as “random”; each time the query runs, each thread will handle different number of rows\n\n![](img/2021-03-01-16-21-38.png)\n\nAfter scanning, SQL Server repartitions the rows in each thread, arranging them in a __deterministic__ order, rows are now distributed “correctly” among threads; _each time the query runs, each thread will handle same number of rows_\n\nThis operator requires some buffer space to do the sorting\n\nNext, it'll allocate some space to create a [bloom filter](https://en.wikipedia.org/wiki/Bloom_filter) (bitmap)\n\n![](img/2021-03-01-16-25-20.png)\n\nWhen the second index scan starts, it also include a probe action that checks on the bitmap net. If the bit is “0”, that means the key does not exists in the first index, if the bit is “1”, that means the key _may_ exists in the first index and can pass through into repartition streams\n\n![](img/2021-03-01-16-37-55.png)\n\nWith bitmap, the actual number of rows after the scan is reduced\n\nWith the two sources ready \u0026 optimized, the Hash join operation can be done quickly in parallel and finally merged together\n\nHere's a summary chart (_note that `mod % 2` \u0026 `mod % 10` are not actual MS hash function implementation_):\n\n\u003cimg src=\"img/2021-03-01-16-40-37.png\" style=\"background:white\"/\u003e\n\nIn this example, values in thread 1 \u0026 2 pass through the `bloom filter` (with hash function `mod % 10`) and only 7 bits is turned on, when thread 3 \u0026 4 come and look up on the bitmask, any value which divided by 10 returns a remainder of 1, 6 or 9 would get filtered, these are the _false negative_ matches. The rest, continue on and be filtered again by the _hash match_ function, with much less need for memory\n\n\u003cblockquote style=\"font-size:85%\"\u003e\n\n\u003cb\u003eTypes of scan:\u003c/b\u003e\n\n\u003cb\u003eUnordered scan\u003c/b\u003e (Allocation Order Scan) using using internal page allocation information\n\u003cul\u003e\n    \u003cli\u003eFavorable for \u003ci\u003eHash match\u003c/i\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cb\u003eOrdered scan\u003c/b\u003e, the engine will scan the index structure\n\u003cul\u003e\n    \u003cli\u003eFavorable for \u003ci\u003eMerge join\u003c/i\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ci\u003eDuring order-preserving re-partition exchange, it does not do any sorting\u003c/i\u003e, it just keep the order of output stream the same as the input stream\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003c/blockquote\u003e\n\n#### Comparing Merge \u0026 Hash, in parallel plans\nThis is a side-by-side comparation of a merge join \u0026 hash join, both produce same set of records\n\nThe query is simple:\n```sql\n--hash\nselect f.custid, d.Week, sum(f.Amount) \nfrom Fact f\ninner join DimDate d\non f.RptDate = d.PK_Date\n-- where d.PK_Date \u003e= '2020-01-01' (uncomment to get merge join plan)\ngroup by f.custid, d.Week\n```\n\nMerge join plan is evaluated by adding a where clause filter by date, the optimizer will now go for _index seek_ in the `DimDate` table, but `2020-01-01` is way lower than the actual data range in `Fact` table, so both queries produce same result\n\n\u003cdiv\u003e\n    \u003cfigure style=\"display:block;margin-left:0;\"\u003e\n        \u003cimg src=\"img/2021-03-01-17-05-12.png\"\u003e\n        \u003cfigcaption style=\"font-size:80%;font-style:italic;\"\u003eMerge\u003c/figcaption\u003e\n    \u003c/figure\u003e\n    vs\n    \u003cfigure style=\"display:block;margin-left:0;\"\u003e\n        \u003cimg src=\"img/2021-03-01-17-05-27.png\"\u003e\n        \u003cfigcaption style=\"font-size:80%;font-style:italic;\"\u003eHash\u003c/figcaption\u003e\n    \u003c/figure\u003e\n\u003c/div\u003e\n\nSince the _index seek_ generate an ordered result set, optimizer tries to make use of an _merge join_ plan, but data from `Fact` table's _clustered index scan_ are not yet ordered, the engine must do it implicitly in the _ordered repartition streams_ operator, thus giving very high cost compared to the _hash join_ one\n\n\u003cblockquote style=\"font-size:85%\"\u003e\n    We can keep track of these symptoms by monitoring the CXPACKET \u0026 SLEEP_TASK wait types (for SQL Server 2008)\n\u003c/blockquote\u003e\n\n**Where the fun begins**\n\nIn normal circumstances, both queries' performance is very similar (around 5s for 200k records)\n\nNow, put the system CPU under load (by running many queries at same time using __SQL Stress Test__), the _merge join_ becomes slower the more threads used, whereas _hash join_'s performance is very consistent (when CPU at 90% load, _merge_ took 13s)\n\n\u003cdiv\u003e\n    \u003cfigure style=\"display:inline-block;margin-left:0;width:40%;\"\u003e\n        \u003cimg src=\"img/2021-03-02-08-53-17.png\"\u003e\n        \u003cfigcaption style=\"font-size:80%;font-style:italic;\"\u003eMerge\u003c/figcaption\u003e\n    \u003c/figure\u003e\n    vs\n    \u003cfigure style=\"display:inline-block;margin-left:0;width:40%;\"\u003e\n        \u003cimg src=\"img/2021-03-02-08-52-07.png\"\u003e\n        \u003cfigcaption style=\"font-size:80%;font-style:italic;\"\u003eHash\u003c/figcaption\u003e\n    \u003c/figure\u003e\n\u003c/div\u003e\n\n__Why?__ \n\nIn _merge_, the order-preserving exchange operator has to run sequentially to get pages from the scan, so at this point it is actually running in _single thread_ mode, and when the CPU is under pressure, it’ll have to wait up to 4ms (a _quantum_, see [SQLOS](https://blog.sqlauthority.com/2015/11/11/sql-server-what-is-sql-server-operating-system/)) to get the next batch of pages\n\nIn _hash_, at no point the execution is done synchronously, parallel execution is used at 100% power, so it is very effective\n\n__SQLOS__\n\nWe've only touched the surface of _SQLOS_ - a operating system sitting between SQL Server \u0026 real OS to manage services such as thread, memory. There's a lot of interesting thing going on behind the scene, I'll write up another article for this at another time\n\n#### Gather streams\nConsider the SQL:\n```sql\n--get all \"above-average\" transactions by products\nselect prod_id, tran_date, amount, avg_amount\n\tfrom (\n\t\tselect prod_id, custid, tran_date, amount, \n\t\t\tavg(amount) over (partition by prod_id) avg_amount\n\t\tfrom tnx_table\n\t\twhere tran_date between ... and ...\n\t) i\nwhere stake \u003e= avg_stake\norder by prod_id\n```\n\n\u003cspan style=\"font-size:120%;font-weight:bold\"\u003e\nExecution time is 8 seconds, but with threading disabled (by adding \u003ccode\u003eoption (maxdop 1)\u003c/code\u003e), execution time drops to 1 second\n\u003c/span\u003e\u003cbr\u003e\u003cbr\u003e\n\n\n![](img/2021-03-02-11-45-55.png)\n\n\u003cspan style=\"font-size:70%\"\u003eThis is the last part of the plan\u003c/span\u003e\u003cbr\u003e\n![](img/2021-03-02-09-45-13.png)\n\nIn this example, lots of _exchange spill events_ are caught\n\nAn [exchange spill](https://www.erikdarlingdata.com/sql-server/spills-week-exchange-spill-excruciation/#:~:text=The%20Exchange%20Spill%20event%20class,plan%20has%20multiple%20range%20scans%E2%80%A6) is like a [`tempdb` spill](####-tempdb-spill), it is a buffer overflow event that happens inside of a thread\n\nHere's a [visualized version](https://forrestmcdaniel.com/2019/09/30/grokking-the-paul-white-parallel-deadlock-demo/) of the above plan:\n\n\u003cimg src=\"https://i2.wp.com/forrestmcdaniel.com/wp-content/uploads/2019/09/Parallel-Deadlock.gif?resize=350%2C220\u0026ssl=1\"\u003e\n\nBecause of the uneven distribution of data in threads (_skewed data_), the ones that have more rows (1 \u0026 4) are more likely to wait for thread 2 \u0026 3 to keep returned rows in order, while piling up their internal buffer, eventually leading to a spill\n\nTo fix this, we need to eliminate the _skewness_ by splitting up data into two parts:\n```sql\nwith [avg] as (\n\tselect prod_id, avg(amount) amount, min(tran_date) min_tdate, max(tran_date) max_tdate\n\tfrom tnx_table\n\twhere tran_date between ... and ...\n\tgroup by prod_id\n)\nselect a.prod_id, a.tran_date, a.amount, [avg].amount avg_amount\nfrom tnx_table a\ninner join [avg]\non a.amount \u003e= [avg].amount\n\tand a.prod_id = [avg].prod_id\n\tand a.tran_date between [avg].min_tdate and [avg].max_tdate\norder by prod_id\n```\n\nNow it executes instantly:\n\n![](img/2021-03-02-12-19-42.png)\n\nKeep in mind that _exchange spill_ can happen with any __blocking__ operator:\n\n![](img/2021-03-02-12-56-39.png)\n\n#### Distribute streams\n![](img/2021-03-08-15-42-57.png)\n\nDistribute rows from a single-threaded operation across multiple threads\n\nCommon types:\n- Hash (for Hash join)\n- Round-Robin (for Nested Loop join)\n- Broadcast (for small set of input)\n\n#### Spools\nA spool is created when the optimizer thinks that data reuse is important (prevent multiple seeks or scans on same index/heap)\n\n_There are two types of spool: lazy \u0026 eager_\n\n- Eager Spool catches all the rows received from another operator (ex: Index scan, concatenation...) and store these rows in TempDB (blocking)\n\n- Lazy Spool is similar to eager spool, but it only reads and stores the rows in a temporary table only when the parent operator actually asks for a row (non-blocking)\n\n**Lazy Spool**\n\nWe're using the previous plan example of _gather streams_, without parallelism\n\n![](img/2021-03-08-15-52-01.png)\n\nWe’ll see that there are 3 lazy spool operators, but it is actually just one instance (by hovering on it, they have the same primary node id)\n\nThe data flow goes as below:\n\n1. The operator scan the transaction table to continuously retrieve all data\n2. Once a segment got all records of a same customer, it copies all rows into a table spool, those rows are then used as the outer input of a nested loop _(1)_\n3. For each loop _(1)_, it scan the entire spool, calculate the average transaction amount of that customer (aggregate), the result is a single row that get passed as outer input into another nested loop _(2)_\n4. The processor scan the spool again, row-by-row to compare with the number from step 3, returning rows that are less than that number\n5. When the index scan is done for another batch of customer rows, the spool is truncated and refill with new data, repeat until all customers are done\n\n![](img/2021-03-08-15-55-09.png)\n\n\u003cblockquote style=\"font-size:85%\"\u003e\n\nThe number of customers in the transaction table is 19758\n\nThe spool is rebound 19759 times meaning it got truncated \u0026 repopulated 19758 times for each customer + 1 time on the first creation\n\n\u003c/blockquote\u003e\n\n**Eager Spool**\n\nQUERY: Add 100 to all `MasterID`s in the customer table\n\n![](img/2021-03-08-15-58-48.png)\n\nDue to MasterID being key in a non-clustered index (IX_master), updating it will physically change it’s location (move towards the right end of the b-tree), if the scan operation is from left to right, then the updated row might be reread \u0026 updated again\n\nThe eager spool is created to temporary store old rows, making sure that each row is read only once\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftcd93%2Fmssql-execution-plan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftcd93%2Fmssql-execution-plan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftcd93%2Fmssql-execution-plan/lists"}