{"id":19658389,"url":"https://github.com/cdaringe/slow-postgraphql-one-million-sort","last_synced_at":"2026-06-17T09:32:15.194Z","repository":{"id":142017656,"uuid":"160009989","full_name":"cdaringe/slow-postgraphql-one-million-sort","owner":"cdaringe","description":"postgraphile slow sorting research","archived":false,"fork":false,"pushed_at":"2023-12-15T02:57:48.000Z","size":23,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-11T06:21:11.071Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cdaringe.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-02T03:29:48.000Z","updated_at":"2019-02-21T02:13:48.000Z","dependencies_parsed_at":"2024-11-11T15:39:15.214Z","dependency_job_id":"4d1480d6-06c3-4607-a0c9-4abbdc556945","html_url":"https://github.com/cdaringe/slow-postgraphql-one-million-sort","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cdaringe/slow-postgraphql-one-million-sort","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cdaringe%2Fslow-postgraphql-one-million-sort","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cdaringe%2Fslow-postgraphql-one-million-sort/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cdaringe%2Fslow-postgraphql-one-million-sort/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cdaringe%2Fslow-postgraphql-one-million-sort/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cdaringe","download_url":"https://codeload.github.com/cdaringe/slow-postgraphql-one-million-sort/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cdaringe%2Fslow-postgraphql-one-million-sort/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34443232,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-17T02:00:05.408Z","response_time":127,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T15:37:26.541Z","updated_at":"2026-06-17T09:32:15.170Z","avatar_url":"https://github.com/cdaringe.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# problem :turtle:\n\n- generated sql is much, much slower than simple sql when doing basic sorting on a (quasi) large dataset\n\n# example\n\n- run `node demo.js`. this script:\n  - creates a db (docker, pg:alpine@latest)\n  - upserts a schema\n  - creates a bunch of fake data\n  - `COPY`s all records to the DB.  (these scripts are a little much, but i had the boilerplate ready!)\n  - starts postgraphile\n\n- open up graphiql\n\n```gql\n{\n  allListings(last: 10) {\n    edges {\n      node {\n        id\n      }\n    }\n  }\n}\n```\n\nnice and quick! let's run a more interesting query, albeit still pretty boring:\n\n```graphql\n{\n  allListings(last: 10, orderBy: [MLS_DESC]) {\n    edges {\n      node {\n        addr1\n        beds1\n        city\n        county\n        mls\n        priceList\n        zip\n        zone\n      }\n    }\n  }\n}\n```\n\nthis yields a 15+ second query.  hmmm. graphiql also sorted by `id`, which my query doesn't specify. :rage1:\n\n```\npostgraphile:postgres with __local_0__ as (\npostgraphile:postgres         with __local_1__ as (\npostgraphile:postgres\npostgraphile:postgres       select to_json((jsonb_build_object('@node'::text, (jsonb_build_object('addr1'::text, (__local_2__.\"addr_1\"), 'beds1'::text, (__local_2__.\"beds_1\"), 'city'::text, (__local_2__.\"city\"), 'county'::text, (__local_2__.\"county\"), 'mls'::text, (__local_2__.\"mls\"), 'priceList'::text, (__local_2__.\"price_list\"), 'zip'::text, (__local_2__.\"zip\"), 'zone'::text, (__local_2__.\"zone\")))))) as \"@edges\"\npostgraphile:postgres       from \"public\".\"listings\" as __local_2__\npostgraphile:postgres\npostgraphile:postgres       where (TRUE) and (TRUE)\npostgraphile:postgres       order by __local_2__.\"mls\" ASC,__local_2__.\"id\" DESC\npostgraphile:postgres       limit 10\npostgraphile:postgres\npostgraphile:postgres\npostgraphile:postgres         )\npostgraphile:postgres         select *\npostgraphile:postgres         from __local_1__\npostgraphile:postgres         order by (row_number() over (partition by 1)) desc\npostgraphile:postgres         ), __local_3__ as (select json_agg(to_json(__local_0__)) asdata from __local_0__) select coalesce((select __local_3__.data from __local_3__), '[]'::json) as \"data\"  +0ms\n0 error(s) in 15046.36ms :: { allListings(last: 10, orderBy: [MLS_DESC]) { edges { node { addr1 beds1 city county mls priceList zipzone } } } }\n```\n\nusing that generated sql, let's now jump just into a `psql` context.  let's _just_ take that subquery `postgraphile` generated for us:\n\n```sh\n$ ./scripts/db/psql.sh # already configured to hook up to the running container\npsql (10.5)\nType \"help\" for help.\n```\n\n```sql\n-- adding `EXPLAIN (ANALYZE, BUFFERS)`,\nEXPLAIN (ANALYZE, BUFFERS)\nselect to_json((jsonb_build_object('@node'::text, (jsonb_build_object('addr1'::text, (__local_2__.\"addr_1\"), 'beds1'::text, (__local_2__.\"beds_1\"), 'city'::text, (__local_2__.\"city\"), 'county'::text, (__local_2__.\"county\"), 'mls'::text, (__local_2__.\"mls\"), 'priceList'::text, (__local_2__.\"price_list\"), 'zip'::text, (__local_2__.\"zip\"), 'zone'::text, (__local_2__.\"zone\")))))) as \"@edges\"\nfrom \"public\".\"listings\" as __local_2__\nwhere (TRUE) and (TRUE)\norder by __local_2__.\"mls\" ASC,__local_2__.\"id\" DESC\nlimit 10;\n-- without explain analyze, ~13 seconds, repeatably\n-- with EXPLAIN (ANALYZE, BUFFERS), 31132.781 ms, repeatably\n\n-- Limit  (cost=68521.60..68521.63 rows=10 width=40) (actual time=31614.179..31614.425 rows=10 loops=1)\n--   Buffers: shared hit=2208 read=27204\n--   -\u003e  Sort  (cost=68521.60..71021.60 rows=999999 width=40) (actual time=31614.159..31614.241 rows=10 loops=1)\n--         Sort Key: mls, id DESC\n--         Sort Method: top-N heapsort  Memory: 27kB\n--         Buffers: shared hit=2208 read=27204\n--         -\u003e  Seq Scan on listings __local_2__  (cost=0.00..46911.98 rows=999999 width=40) (actual time=0.200..23706.512 rows=999999loops=1)\n```\n\nok. bummer. let's realllly simplify the generated sql:\n\n```sql\nselect *\nfrom \"public\".\"listings\"\norder by \"mls\" DESC, \"id\" DESC\nlimit 10;\n-- almost instant!\n```\n\nso, it would seem that the json operations are really hurting perf!\n\ni can do more debugging, but my pg-fu is weak, so i'd prefer to pair or be assigned homework :)\n\nfinally,\n\n`docker rm -f slowpo` to clean up the db!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcdaringe%2Fslow-postgraphql-one-million-sort","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcdaringe%2Fslow-postgraphql-one-million-sort","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcdaringe%2Fslow-postgraphql-one-million-sort/lists"}