{"id":15825025,"url":"https://github.com/karlhorky/postgresql-tricks","last_synced_at":"2025-07-30T14:39:12.600Z","repository":{"id":150431115,"uuid":"574081203","full_name":"karlhorky/postgresql-tricks","owner":"karlhorky","description":"A collection of useful tricks for PostgreSQL","archived":false,"fork":false,"pushed_at":"2024-06-14T17:22:18.000Z","size":5,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-25T23:10:33.480Z","etag":null,"topics":["postgres","postgresql"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/karlhorky.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-04T11:23:02.000Z","updated_at":"2024-10-12T14:58:07.000Z","dependencies_parsed_at":"2024-10-20T00:13:43.454Z","dependency_job_id":"c64ba6f8-4326-4f31-bec8-2d4d0b450b8e","html_url":"https://github.com/karlhorky/postgresql-tricks","commit_stats":{"total_commits":5,"total_committers":1,"mean_commits":5.0,"dds":0.0,"last_synced_commit":"a6cf055fb5d9c71c1ea53df0b8a868be6731f327"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlhorky%2Fpostgresql-tricks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlhorky%2Fpostgresql-tricks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlhorky%2Fpostgresql-tricks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlhorky%2Fpostgresql-tricks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/karlhorky","download_url":"https://codeload.github.com/karlhorky/postgresql-tricks/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246612487,"owners_count":20805355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["postgres","postgresql"],"created_at":"2024-10-05T09:02:33.580Z","updated_at":"2025-04-01T09:06:40.753Z","avatar_url":"https://github.com/karlhorky.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# PostgreSQL Tricks\n\nA collection of useful tricks for PostgreSQL\n\n## Create Records in Result Set\n\n### [DB Fiddle Demo](https://www.db-fiddle.com/f/jXbjFvW2iLg5XhFXibx1e6/0)\n\nIt can be useful to generate new hardcoded records which are not returned from data in your tables.\n\nFor example, it may be useful to have an arbitrary empty record at the end of this result set, filled with `NULL`s:\n\n```sql\nSELECT\n  id,\n  title\nFROM\n  competence_levels\nORDER BY id DESC NULLS LAST;\n\n id |        title\n----+----------------------\n  6 | Proficient\n  5 | Skilled\n  4 | Experienced\n  3 | Familiar\n  2 | Practical Experience\n  1 | Basic Understanding\n(6 rows)\n```\n\nTo create this new empty row at the bottom of the result set, the `UNION ALL` operator can be used to add an additional row:\n\n```sql\nSELECT\n  id,\n  title\nFROM\n  (\n    SELECT id, title FROM competence_levels\n    UNION ALL\n    SELECT NULL AS id, NULL AS title\n  ) AS competence_levels_with_nulls\nORDER BY id DESC NULLS LAST;\n\n id |        title\n----+----------------------\n  6 | Proficient\n  5 | Skilled\n  4 | Experienced\n  3 | Familiar\n  2 | Practical Experience\n  1 | Basic Understanding\n    |\n(7 rows)\n```\n\nTo add multiple records, another option is to use the `VALUES` keyword:\n\n```sql\nSELECT\n  id,\n  title\nFROM\n  (\n    SELECT id, title FROM competence_levels\n    UNION ALL\n    SELECT id, title FROM (VALUES\n      (NULL, NULL),\n      (1, NULL),\n      (NULL, 'Basic Understanding')\n     ) AS hardcoded_competence_levels(id, title)\n  ) AS competence_levels_with_nulls\nORDER BY id DESC NULLS LAST;\n\n id |        title\n----+----------------------\n  6 | Proficient\n  5 | Skilled\n  4 | Experienced\n  3 | Familiar\n  2 | Practical Experience\n  1 | Basic Understanding\n  1 |\n    | Basic Understanding\n    |\n(9 rows)\n```\n\n## Seed Test Fixture Data with Explicit `id`s to Generated Identity Fields\n\nWhen adding data to a database for testing purposes, it's often useful to have explicit `id` values to reference records in other tables via foreign keys.\n\nHowever, these explicit `id` values are incompatible with identity fields such as a field specified with `id PRIMARY KEY GENERATED ALWAYS AS IDENTITY` - inserting records with explicit `id` values will lead to `cannot insert a non-DEFAULT value into column` errors from PostgreSQL:\n\n```\n2025-01-06 12:49:32.107 UTC [17659] ERROR:  cannot insert a non-DEFAULT value into column \"id\"\n2025-01-06 12:49:32.107 UTC [17659] DETAIL:  Column \"id\" is an identity column defined as GENERATED ALWAYS.\n2025-01-06 12:49:32.107 UTC [17659] HINT:  Use OVERRIDING SYSTEM VALUE to override.\n2025-01-06 12:49:32.107 UTC [17659] STATEMENT:\n  INSERT INTO\n    regions (id, slug, title)\n  VALUES\n    ($1, $2, $3),\n    ($4, $5, $6)\n  ON CONFLICT (id) DO UPDATE\n  SET\n    id = excluded.id,\n    slug = excluded.slug,\n    title = excluded.title\n```\n\nTo use explicit `id` values in test fixture data while using generated identity fields, drop the identity, insert the records and add the identity back.\n\nThe following example of this approach uses:\n\n- [Postgres.js](https://github.com/porsager/postgres)\n\n`seedFixtures.ts`\n\n```ts\nimport { readdir } from 'node:fs/promises';\nimport postgres from 'postgres';\n\nif (!process.env.FEATURE_TEST_SEEDING) {\n  throw new Error('Set the environment variable FEATURE_TEST_SEEDING to seed database with test data');\n}\n\nconst sql = postgres({\n  transform: postgres.camel,\n});\n\nconst testFixtures = (await readdir('./tables', { withFileTypes: true }))\n  .filter((entry) =\u003e {\n    return entry.isFile() \u0026\u0026 /^\\d+-[^.]+\\.fixture\\.ts$/.test(entry.name);\n  })\n  .sort((a, b) =\u003e {\n    return parseInt(a.name.split('-')[0]!) - parseInt(b.name.split('-')[0]!);\n  });\n\nfor (const testFixture of testFixtures) {\n  const testFixtureModule = (await import(`../tables/${testFixture.name}`)) as {\n    [key: string]: { [key: string]: { id: number } };\n  };\n\n  for (const [exportName, fixturesObj] of Object.entries(testFixtureModule)) {\n    const tableName = camelToSnake(\n      exportName\n        .replace(/^test/, '')\n        .replace(/^[A-Z]/, (letter) =\u003e letter.toLowerCase()),\n    ) as string;\n    const fixtures = Object.values(fixturesObj);\n\n    if (fixtures.length \u003e 0) {\n      const idFieldIsIdentity =\n        (\n          await sql\u003c{ isIdentity: 'YES' | 'NO' }[]\u003e`\n            SELECT\n              is_identity\n            FROM\n              information_schema.columns\n            WHERE\n              table_name = ${tableName}\n              AND column_name = 'id'\n          `\n        )[0]!.isIdentity === 'YES';\n\n      if (idFieldIsIdentity) {\n        await sql`\n          ALTER TABLE ${sql(tableName)}\n          ALTER COLUMN id\n          DROP IDENTITY\n        `;\n      }\n\n      await sql`\n        INSERT INTO\n          ${sql(tableName)} ${sql(fixtures)}\n      `;\n\n      if (idFieldIsIdentity) {\n        // Only configure GENERATED ALWAYS, to avoid inconsistencies with GENERATED BY DEFAULT\n        await sql`\n          ALTER TABLE ${sql(tableName)}\n          ALTER COLUMN id\n          ADD GENERATED ALWAYS AS IDENTITY\n        `;\n\n        // Reset sequence to the next record id, to allow for new\n        // record inserts with the sequentially generated index\n        // (PRIMARY KEY GENERATED ALWAYS AS IDENTITY)\n        await sql`\n          SELECT\n            setval(\n              pg_get_serial_sequence(\n                ${tableName},\n                'id'\n              ),\n              (\n                SELECT\n                  max(id)\n                FROM\n                  ${sql(tableName)}\n              )\n            )\n        `;\n      }\n\n      console.log(`✔️ Inserted ${fixtures.length} records into ${tableName}`);\n    }\n  }\n}\n\nawait sql.end();\n\nconsole.log('Done syncing test seeding fixtures to database');\n\ntype CamelToSnake\u003c\n  S extends string,\n  Result extends string = '',\n\u003e = S extends `${infer First}${infer Rest}`\n  ? First extends Capitalize\u003cFirst\u003e\n    ? CamelToSnake\u003cRest, `${Result}_${Lowercase\u003cFirst\u003e}`\u003e\n    : CamelToSnake\u003cRest, `${Result}${First}`\u003e\n  : Result;\n\nfunction camelToSnake\u003cCamelCaseString extends string\u003e(\n  camelCaseString: CamelCaseString,\n): CamelToSnake\u003cCamelCaseString\u003e {\n  return camelCaseString.replace(\n    /[A-Z]/g,\n    (letter) =\u003e `_${letter.toLowerCase()}`,\n  ) as CamelToSnake\u003cCamelCaseString\u003e;\n}\n```\n\nFixture files look like this:\n\n`tables/001-regions.fixture.ts`\n\n```ts\ntype Region = {\n  id: number;\n  slug: string;\n  title: string;\n};\n\nexport const testRegions = {\n  europe: {\n    id: 1,\n    slug: 'eu',\n    title: 'Europe',\n  },\n  australia: {\n    id: 2,\n    slug: 'au',\n    title: 'Australia',\n  },\n} as const satisfies { [key: string]: Region };\n```\n\nThis allows the `id` values to be used in other tables as foreign keys, eg. `testRegions.europe.id` could be imported in `tables/002-campuses.fixture.ts` to use as a value for a foreign key field `campuses.region_id`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarlhorky%2Fpostgresql-tricks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkarlhorky%2Fpostgresql-tricks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarlhorky%2Fpostgresql-tricks/lists"}