https://github.com/peopledoc/pg-magicplan

Improve planning of EXISTS in PostgreSQL subquery
https://github.com/peopledoc/pg-magicplan

approved-public ghec-mig-migrated team-dba

Last synced: 17 days ago
JSON representation

Improve planning of EXISTS in PostgreSQL subquery

• Host: GitHub
• URL: https://github.com/peopledoc/pg-magicplan
• Owner: peopledoc
• License: 0bsd
• Created: 2019-11-21T12:10:59.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2021-02-16T10:04:48.000Z (almost 4 years ago)
• Last Synced: 2024-04-16T06:52:51.850Z (9 months ago)
• Topics: approved-public, ghec-mig-migrated, team-dba
• Language: C
• Size: 37.1 KB
• Stars: 5
• Watchers: 38
• Forks: 2
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: COPYING

Awesome Lists containing this project

README

        
# pg-magicplan

Improve planning of EXISTS in PostgreSQL subquery.

The optimizer, even in PostgreSQL 12 as of writing, does a very straight

optimization decision when executing queries that have an EXISTS sublink: it

considers that it will always be cheaper to transform the subquery into a join.

But such a decision is not always the best choice, and there is a way to tell

the optimizer not to do that, as documented in the optimizer source code (in 

function simplify_EXISTS_query), by adding an OFFSET 0 clause in the subquery.

During the development and optimization of queries in a PeopleDoc project, we

encountered a situation where such an OFFSET 0 would drastically improve 

performances for some customers, while destroying it for others, with no way

for the application to know when to add that clause.

We also realized that the PostgreSQL planner knew, in most situations, what was

indeed the best plan: when planning with an added OFFSET 0, the cost would be

lower than the pristine plan, and thus plan would indeed most of the time be a

much better plan.

The only solution we found to fix that without adding massive hacks in the

application was to add a planner hook in our database. Before executing the

query, if it matches the expected 'layout', we try to inject OFFSET 0 clauses

in every EXISTS subquery, and we execute only the best plan we found.

This solution is only a workaround. We are documenting the situation before

doing a proper bugreport along with, if possible, a way to reproduce the issue.

# Building debian package with new PG version

All these are done in the proper debian chroot.

1) make sure you've updated your postgresql-common and postgresql-server-dev-all packages

2) update the debian/pgversions file

3) `pg_buildext updatecontrol`

4) `debuild -us -uc`