{"id":17794250,"url":"https://github.com/pyk/postgresql-ansible","last_synced_at":"2026-02-04T00:39:10.766Z","repository":{"id":46814539,"uuid":"261207549","full_name":"pyk/postgresql-ansible","owner":"pyk","description":"Ansible playbook that I use to provision new PostgreSQL database in Ubuntu 18.04","archived":false,"fork":false,"pushed_at":"2023-01-24T23:30:29.000Z","size":20,"stargazers_count":1,"open_issues_count":4,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-07T13:25:13.749Z","etag":null,"topics":["ansible","devops","postgres"],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pyk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-05-04T14:37:51.000Z","updated_at":"2021-05-17T17:08:20.000Z","dependencies_parsed_at":"2023-02-14T02:31:14.283Z","dependency_job_id":null,"html_url":"https://github.com/pyk/postgresql-ansible","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pyk/postgresql-ansible","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyk%2Fpostgresql-ansible","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyk%2Fpostgresql-ansible/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyk%2Fpostgresql-ansible/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyk%2Fpostgresql-ansible/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pyk","download_url":"https://codeload.github.com/pyk/postgresql-ansible/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyk%2Fpostgresql-ansible/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264019980,"owners_count":23545242,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","devops","postgres"],"created_at":"2024-10-27T11:15:38.262Z","updated_at":"2026-02-04T00:39:10.732Z","avatar_url":"https://github.com/pyk.png","language":null,"readme":"# PostgreSQL Ansible\n\nThis repository contains [ansible playbook](https://docs.ansible.com/ansible/latest/user_guide/playbooks.html)\nto provision and manage PostgreSQL database in Ubuntu 18.04.\n\n\n## Best Practices\nThis is list of recommended best practices from PostgreSQL expert. This best\npractices that I wrote is only relevant for Ubuntu 18.04 64-bit.\n\nAll configurations are set in `/etc/postgresql/12/main/postgresql.conf`.\n\n1. Don't do OLAP (Online Analytical Processing) where you OLTP (Online\n   Transactional Processing).\n\n   Slow query in specific table will cause lock\n   queue, any other query for that table will wait and slow down.\n\n   It's recommended to create read-replica or build the data pipeline outside\n   postgresql database.\n\n   OLAP is where you do the agregation (SUM, COUNT, etc) from large of rows.\n   OLTP is where you insert/update/get few rows in real time.\n2. Set `listen_address = \"*\"` and then control who can and cannot connect via\n   the `pg_hba.conf` file.\n3. Set `max_connections` based on node RAM resources. To many client\n   connections will use your RAM.\n4. Set `shared_buffers` to 25% of node RAM.\n5. Set `effective_cache_size` to `50%` of node RAM.\n6. Set `default_statistics_target` to 100\n7. Set `work_mem=(50% * total_ram)/max_connections` for starter.\n   Make sure `max_connections` times `work_mem` is not larger\n   than 50% of total RAM.\n8. `log_temp_files` can be used to log sorts, hashes, and temp files which\n    can be useful in figuring out if sorts are spilling to disk instead of\n    fitting in memory. Increase `work_mem` if this log exists.\n9. Set `maintenance_work_mem` to 64MB\n10. Disable `autovacumm` when loading bulk of data.\n\n\nIdentifying slow queries and fixing them:\n1. Set `log_checkpoints = on` to tell postgresql to log checkpoints activities.\n   Query slow maybe the query happen when postgresql writing all the data to\n   the checkpoints.\n2. Set `log_connections = on` and `log_disconnections = on`. If we have so many\n   short-lived connection is very bad for performance.\n3. Set `log_lock_waits = on` to make sure that slow query is not because\n   deadlock.\n4. Set `log_min_duration=300` to tell postgresql to log all queries that\n   take more than 300ms\n5. Set `log_temp_files = 0` to find if there is any query that use temporary files.\n   Query that use temporary files tend to be slow.\n\n\nReferences:\n- [Chris Sinjakli : Lessons Learned the Hard Way / Postgres in Production at GoCardless - PGDayUK'16](https://www.youtube.com/watch?v=Tu-cf-Jki60)\n- [Tuning Your PostgreSQL Server](https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server)\n- [Postgres Open 2016 - Identifying Slow Queries and Fixing Them!](https://www.youtube.com/watch?v=yhOkob2PQFQ)\n\n\n\n## Setup\n\nClone this repository:\n\n    git clone https://github.com/pyk/postgresql-ansible.git\n    cd postgresql-ansible/\n\nCreate new python environment:\n\n    python3 -m venv venv\n\nActivate the environment:\n\n    source venv/bin/activate\n\nInstall the dependencies:\n\n    pip install -r requirements.txt\n\n\n## Inventory\n\nCreate new `hosts` file with the following content:\n\n    [nodes]\n    NODE_NAME ansible_host=IP_ADDRESS ansible_user=root ansible_python_interpreter=/usr/bin/python3\n\n## Provision\n\nRun the following command to provision new PostgreSQL database:\n\n    ansible-playbook -i hosts provision.yaml\n\nThis command will provision \u0026 configure new postgresql database in specified\n`hosts` based on the RAM of the node.\n\n\n## Configure\n\nTo configure the database, update the `postgresql.conf` and run the following\ncommand:\n\n    ansible-playbook -i hosts setup_postgresql_conf.yaml\n\nBecareful, this will restart your postgresql.\n\n\n## Create new database\n\nTo create new database, create new `databases.yaml` with the following content:\n\n    ---\n    databases:\n        - db_name: db1\n          user_name: user1\n          user_pass: pass1\n        - db_name: db2\n          user_name: user2\n          user_pass: pass2\n\nThen run the following command:\n\n    ansible-playbook -i hosts setup_databases.yaml\n\nNow you can connect the database using created user directly inside database\nnode or node in the same subnet.\n\nFor example:\n\n     psql -h PRIVATE_IP -U streamers -d streamers\n\n\n## Recommended IaaS\n\nI use DigitalOcean to host my PostgreSQL database. You can use my\n[referral](https://m.do.co/c/ddb6db36549a) to get $100 for 60 days.\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyk%2Fpostgresql-ansible","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpyk%2Fpostgresql-ansible","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyk%2Fpostgresql-ansible/lists"}