{"id":26491609,"url":"https://github.com/edisedis777/pg-tuning-guide","last_synced_at":"2025-03-20T08:49:54.402Z","repository":{"id":282220721,"uuid":"947869720","full_name":"edisedis777/pg-tuning-guide","owner":"edisedis777","description":"A practical guide and example repository for tuning PostgreSQL parameters based on workload, hardware, and use case.","archived":false,"fork":false,"pushed_at":"2025-03-13T11:49:07.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-13T12:31:31.038Z","etag":null,"topics":["postgres","postgresql","query","sql","tuning"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/edisedis777.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-13T11:28:25.000Z","updated_at":"2025-03-13T11:49:37.000Z","dependencies_parsed_at":"2025-03-13T12:41:41.282Z","dependency_job_id":null,"html_url":"https://github.com/edisedis777/pg-tuning-guide","commit_stats":null,"previous_names":["edisedis777/pg-tuning-guide"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edisedis777%2Fpg-tuning-guide","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edisedis777%2Fpg-tuning-guide/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edisedis777%2Fpg-tuning-guide/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edisedis777%2Fpg-tuning-guide/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/edisedis777","download_url":"https://codeload.github.com/edisedis777/pg-tuning-guide/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244583179,"owners_count":20476233,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["postgres","postgresql","query","sql","tuning"],"created_at":"2025-03-20T08:49:53.699Z","updated_at":"2025-03-20T08:49:54.396Z","avatar_url":"https://github.com/edisedis777.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# PostgreSQL Tuning Guide\n\nA practical guide for tuning PostgreSQL parameters based on workload and hardware specifications. \nThis project provides example SQL scripts and recommendations for tuning key PostgreSQL parameters, including memory, parallelism, JIT, and connection settings. \n\n---\n\n## Project Overview\n\nThis file serves as a resource for tuning PostgreSQL parameters.\n\n### Disclaimer\nThe recommended values serve as a starting point. Optimal settings depend on workload type, hardware specifications, and database size. \nAlways analyze your system’s performance and adjust based on real-world usage and monitoring.\n\n---\n\n## Project Structure (Conceptual)\n\nWhile everything is in this README, here’s how the content is logically organized:\n- **Memory-Related Parameters**: Tuning `work_mem`, `shared_buffers`, etc.\n- **Parallelism-Related Parameters**: Optimizing `max_parallel_workers`, etc.\n- **JIT-Related Parameters**: Configuring `jit`, `jit_above_cost`, etc.\n- **Connection-Related Parameters**: Adjusting `max_connections`, etc.\n- **Helper Scripts**: Tools for applying settings and monitoring.\n\n---\n\n## Setup Instructions\n\n1. Ensure PostgreSQL is installed and running on your system.\n2. Copy and paste the SQL scripts below into your PostgreSQL client (e.g., `psql -U postgres`).\n3. Reload the configuration after changes with `SELECT pg_reload_conf();`.\n4. Test the settings with provided example queries.\n\n---\n\n## Memory-Related Parameters\n\nThese parameters control how PostgreSQL allocates and manages memory. Tuning them improves query performance and prevents resource bottlenecks.\n\n### work_mem\n- **Description**: Sets the maximum memory for internal operations (e.g., sorts, hashes) before writing to disk.\n- **Default**: 4MB\n- **Recommendation**: 1-2% of total system memory (e.g., 3-5 GB for 256 GB RAM).\n- **Example Scenario**: System with 256 GB RAM, complex sorting queries.\n```sql\n-- Tuning work_mem for a system with 256 GB RAM\nALTER SYSTEM SET work_mem = '4GB';\nSELECT pg_reload_conf();\n\n-- Test query to observe impact\nEXPLAIN ANALYZE SELECT * FROM large_table ORDER BY column1;\n```\n- Note: Higher values increase memory usage for sorting operations.\n\n### shared_buffers\n- **Description**: Amount of memory for caching database data.\n- **Default**: 128MB\n- **Recommendation**: 25-40% of total system memory (e.g., 64-102 GB for 256 GB RAM).\n- **Example Scenario**: System with 256 GB RAM, heavy read/write workload.\n```sql\n-- Tuning shared_buffers for a system with 256 GB RAM\nALTER SYSTEM SET shared_buffers = '80GB';\nSELECT pg_reload_conf();\n\n-- Verify setting\nSHOW shared_buffers;\n```\n\n### maintenance_work_mem\n- **Description**: Memory for maintenance operations (e.g., VACUUM, CREATE INDEX).\n- **Default**: 64MB\n- **Recommendation**: 5-10% of total system memory (e.g., 13-26 GB for 256 GB RAM).\n- **Example Scenario**: System with 256 GB RAM, frequent index rebuilds.\n```sql\n-- Tuning maintenance_work_mem for a system with 256 GB RAM\nALTER SYSTEM SET maintenance_work_mem = '20GB';\nSELECT pg_reload_conf();\n\n-- Test with VACUUM\nVACUUM VERBOSE large_table;\n```\n\n---\n\n## Parallelism-Related Parameters\nThese parameters control how tasks are divided across CPU cores, improving query performance by reducing execution time.\n\n### max_parallel_maintenance_workers\n- **Description**: Maximum parallel workers for maintenance operations (e.g., VACUUM, CREATE INDEX).\n- **Default**: 2\n- **Recommendation**: 12% of available cores (e.g., 6 cores for 48 cores).\n- **Example Scenario**: System with 48 cores, large table maintenance.\n```sql\n-- Tuning max_parallel_maintenance_workers for 48 cores\nALTER SYSTEM SET max_parallel_maintenance_workers = 6;\nSELECT pg_reload_conf();\n\n-- Test with CREATE INDEX\nCREATE INDEX idx_large_table ON large_table(column1);\n```\n\n### max_parallel_workers\n- **Description**: Maximum parallel workers for query execution (e.g., scans, joins).\n- **Default**: 8\n- **Recommendation**: 75% of available cores (e.g., 36 cores for 48 cores).\n- **Example Scenario**: System with 48 cores, complex queries.\n```sql\n-- Tuning max_parallel_workers for 48 cores\nALTER SYSTEM SET max_parallel_workers = 36;\nSELECT pg_reload_conf();\n\n-- Test parallel query\nEXPLAIN ANALYZE SELECT * FROM large_table WHERE column1 \u003e 1000;\n```\n\n### max_parallel_workers_per_gather\n- **Description**: Maximum parallel workers per query operation (e.g., JOIN, SCAN).\n- **Default**: 2\n- **Recommendation**: 1/6 of total cores (e.g., 8 cores for 48 cores).\n- **Example Scenario**: System with 48 cores, heavy JOIN operations.\n```sql\n-- Tuning max_parallel_workers_per_gather for 48 cores\nALTER SYSTEM SET max_parallel_workers_per_gather = 8;\nSELECT pg_reload_conf();\n\n-- Test with a join\nEXPLAIN ANALYZE SELECT a.* FROM large_table a JOIN another_table b ON a.id = b.id;\n```\n\n### max_worker_processes\n- **vDescription**: Maximum background worker processes for all tasks.\n- **Default**: 8\n- **Recommendation**: 100% of available cores (e.g., 48 cores for 48 cores).\n- **Example Scenario**: System with 48 cores, mixed workloads.\n```sql\n-- Tuning max_worker_processes for 48 cores\nALTER SYSTEM SET max_worker_processes = 48;\nSELECT pg_reload_conf();\n\n-- Verify\nSHOW max_worker_processes;\n```\n\n## JIT-Related Parameters\nThese parameters manage Just-In-Time (JIT) compilation to optimize query execution speed.\n\n### jit\n- **Description**: Enables/disables JIT compilation for CPU-bound queries.\n- **Default**: on\n- **Recommendation**: Enable (on) for CPU-bound workloads.\n- **Example Scenario**: System with complex, CPU-intensive queries.\n```sql\n-- Enabling JIT for CPU-bound queries\nALTER SYSTEM SET jit = 'on';\nSELECT pg_reload_conf();\n\n-- Test with a complex query\nEXPLAIN ANALYZE SELECT COUNT(*) FROM large_table WHERE complex_function(column1) \u003e 50;\n```\n- Note: Aggressive settings may compile unnecessary query parts, increasing execution time.\n\n### jit_above_cost\n- **Description**: Minimum query cost for JIT compilation.\n- **Default**: 100000\n- **Recommendation**: Increase for selective use (e.g., 150000 for CPU-bound queries).\n- **Example Scenario**: System with mixed query complexity.\n```sql\n-- Tuning jit_above_cost for selective JIT\nALTER SYSTEM SET jit_above_cost = 150000;\nSELECT pg_reload_conf();\n\n-- Test query\nEXPLAIN ANALYZE SELECT * FROM large_table WHERE expensive_calculation(column1) \u003e 100;\n```\n\n### jit_inline_above_cost\n- **Description**: Minimum cost for inlining function calls with JIT.\n- **Default**: 500000\n- **Recommendation**: Adjust based on function complexity (e.g., 500000 as starting point).\n- **Example Scenario**: System with frequent function calls.\n```sql\n-- Tuning jit_inline_above_cost\nALTER SYSTEM SET jit_inline_above_cost = 500000;\nSELECT pg_reload_conf();\n\n-- Test query\nEXPLAIN ANALYZE SELECT custom_function(column1) FROM large_table;\n```\n\n### jit_optimize_above_cost\n- **Description**: Cost threshold for additional JIT optimizations.\n- **Default**: 500000\n- **Recommendation**: Adjust for selective optimization (e.g., 500000 as starting point).\n- **Example Scenario**: System with resource-heavy queries.\n```sql\n-- Tuning jit_optimize_above_cost\nALTER SYSTEM SET jit_optimize_above_cost = 500000;\nSELECT pg_reload_conf();\n\n-- Test query\nEXPLAIN ANALYZE SELECT * FROM large_table WHERE complex_calculation(column1) \u003e 200;\n```\n\n## Connection-Related Parameters\nThese parameters manage client connections, ensuring efficient handling under varying loads.\n\n### max_connections\n- **Description**: Maximum concurrent connections allowed.\n- **Default**: 100\n- **Recommendation**: (cores * 3) * 2 (e.g., 288 for 48 cores).\n- **Example Scenario**: System with 48 cores, high connection demand.\n```sql\n-- Tuning max_connections for 48 cores\nALTER SYSTEM SET max_connections = 288;\nSELECT pg_reload_conf();\n\n-- Verify\nSHOW max_connections;\n```\n- Note: Use a connection pooler if requirements exceed this value.\n\n### idle_in_transaction_session_timeout\n- **Description**: Time (ms) a session can remain idle in a transaction before termination.\n- **Default**: 0 (disabled)\n- **Recommendation**: 30-60 seconds (30000-60000 ms).\n- **Example Scenario**: System with frequent idle transactions.\n```sql\n-- Setting timeout for idle transactions\nALTER SYSTEM SET idle_in_transaction_session_timeout = '30000';  -- 30 seconds\nSELECT pg_reload_conf();\n\n-- Test by leaving a transaction idle\nBEGIN;\n-- Wait 30+ seconds\nSELECT * FROM pg_stat_activity WHERE state = 'idle in transaction';\n```\n\n### idle_session_timeout\n- **Description**: Time (ms) a session can remain idle before termination.\n- **Default**: 0 (disabled)\n- **Recommendation**: 5-30 minutes (300000-1800000 ms).\n- **Example Scenario**: System with inactive connections.\n```sql\n-- Setting timeout for idle sessions\nALTER SYSTEM SET idle_session_timeout = '300000';  -- 5 minutes\nSELECT pg_reload_conf();\n\n-- Verify active sessions\nSELECT * FROM pg_stat_activity WHERE state = 'idle';\n```\n\n## Helper Scripts\n\n### Apply Configuration\nReloads PostgreSQL configuration after changes.\n\n```sql\n-- Reload configuration\nSELECT pg_reload_conf();\n```\n\n### Monitor Performance\nMonitors sessions and resource usage.\n\n```sql\n-- Monitor active sessions and resource usage\nSELECT pid, state, query, wait_event, pg_stat_activity.usename\nFROM pg_stat_activity\nWHERE state IS NOT NULL;\n\n-- Check memory usage\nSELECT name, setting FROM pg_settings \nWHERE name IN ('work_mem', 'shared_buffers', 'maintenance_work_mem');\n```\n\n## Additional Notes\n- Testing: Replace large_table, column1, etc., with your actual table/column names.\n- Hardware: Examples assume 256 GB RAM and 48 cores; adjust values for your system.\n- Monitoring: Use EXPLAIN ANALYZE and pg_stat_activity to measure impact.\n- Contributions: Feel free to fork this repo and submit pull requests with additional examples.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedisedis777%2Fpg-tuning-guide","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fedisedis777%2Fpg-tuning-guide","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedisedis777%2Fpg-tuning-guide/lists"}