{"id":14069543,"url":"https://github.com/dbajuliano/postgres-dba-scripts","last_synced_at":"2025-07-30T05:32:26.206Z","repository":{"id":224201996,"uuid":"262558882","full_name":"dbajuliano/postgres-dba-scripts","owner":"dbajuliano","description":"Daily scripts used on multi cluster environments with many hosts","archived":false,"fork":false,"pushed_at":"2021-05-18T12:56:53.000Z","size":112,"stargazers_count":6,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-04T10:39:48.698Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dbajuliano.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-09T11:46:17.000Z","updated_at":"2024-01-26T14:06:26.000Z","dependencies_parsed_at":null,"dependency_job_id":"87c7b12a-f33a-4cca-944e-e355d4d28ad9","html_url":"https://github.com/dbajuliano/postgres-dba-scripts","commit_stats":null,"previous_names":["dbajuliano/postgres-dba-scripts"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dbajuliano/postgres-dba-scripts","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbajuliano%2Fpostgres-dba-scripts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbajuliano%2Fpostgres-dba-scripts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbajuliano%2Fpostgres-dba-scripts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbajuliano%2Fpostgres-dba-scripts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dbajuliano","download_url":"https://codeload.github.com/dbajuliano/postgres-dba-scripts/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbajuliano%2Fpostgres-dba-scripts/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267815187,"owners_count":24148356,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-13T07:07:02.247Z","updated_at":"2025-07-30T05:32:25.096Z","avatar_url":"https://github.com/dbajuliano.png","language":"Shell","funding_links":["https://www.paypal.me/julianotech"],"categories":["Shell"],"sub_categories":[],"readme":"# :elephant: [postgres-dba-scripts](README.md)\nWelcome to my public notes and scripts about Postgres\n\n\u003cbr\u003e\n\n# :closed_lock_with_key: [.pgpass](.pgpass)\n:heavy_exclamation_mark: Most of the scripts requires credentials stored on the file [.pgpass](https://www.postgresql.org/docs/11/libpq-pgpass.html) to access the databases\n\n\u003cbr\u003e\n\n# :floppy_disk: [pg_backup.sh](scripts/pg_backup.sh)\n\nCron syntax:\n```\nmm hh dd month weekday /path/to/script.sh hostname port username\n```\nThe example below runs every midnight cron user replication on 02 different hosts:\n```\n00 00 * * * /opt/bin/pg_backup.sh primary.production.localhost 5432 replication\n00 00 * * * /opt/bin/pg_backup.sh single.test.localhost 5433 replication\n```\nOr as an alternative, we can use the `/etc/crontab` like the example below exporting the `.pgpass` credentials:\n```\n00 00 * * * postgres export PGPASSFILE=/home/replication/.pgpass | /opt/bin/pg_backup.sh localhost 5432 replication\n```\n\nOutput:\n```\n[2020-05-10 - 13:12:02] : /opt/bin/pg_backup.sh : Performing base backup on new dir /backup/postgres\n[2020-05-10 - 13:24:33] : /opt/bin/pg_backup.sh : Created compressed backup directory /backup/2020.05.10-13.12\n[2020-05-10 - 13:24:33] : /opt/bin/pg_backup.sh : Compressing /backup/postgres into /backup/2020.05.10-13.12\n[2020-05-10 - 13:33:49] : /opt/bin/pg_backup.sh : Cleaning compressed backup directories older than 4 days\n[2020-05-10 - 13:33:50] : /opt/bin/pg_backup.sh : Deleting archives\n[2020-05-10 - 13:33:50] : /opt/bin/pg_backup.sh : Finished\n```\n```\nls -l /backup/\n2020.05.08-00.00\n2020.05.09-00.00\n2020.05.10-00.00\n2020.05.11-00.00\nbackup_logs/\n```\n\n\u003cbr\u003e\n\n# :mag_right: [find_user_multiple_hosts.sh](scripts/find_user_multiple_hosts.sh)\nAll you need is to provide a username as parameter to find. I.e.: `/opt/bin/find_user_multiple_hosts.sh juliano`\n\nOutput:\n```\nHost: primary.production.local Port: 5432\nuser\njuliano_readonly\njuliano_admin\n(2 rows)\n------------------------------\n\nHost: single.test.local Port: 5433\nuser\nro_juliano\nsu_juliano\n(2 rows)\n------------------------------\n\nHost: demo.local Port: 5434\nuser\n(0 rows)\n------------------------------\n```\n\n\u003cbr\u003e\n\n# :clipboard: [refresh_materialized_view.sh](scripts/refresh_materialized_view.sh)\n\nA Cron script with customized port in case of running multiple pg instances\n```\n00 2 * * * /opt/bin/refresh_materialized_view.sh localhost 5432\n00 2 * * * /opt/bin/refresh_materialized_view.sh demo.local 5434\n```\n\nOutput:\n```\n[2020-05-07 - 02:00:00] : /opt/bin/refresh_materialized_view.sh : Started\n\nREFRESHING public.mview_queries\nTiming is on.\nREFRESH MATERIALIZED VIEW\nTime: 315.307 ms\n\nREFRESHING public.mview_trx\nTiming is on.\nREFRESH MATERIALIZED VIEW\nTime: 574.408 ms\n\nREFRESHING public.mview_tests\nTiming is on.\nREFRESH MATERIALIZED VIEW\nTime: 142.555 ms\n\n[2020-05-07 - 02:00:01] : /opt/bin/refresh_materialized_view.sh : Finished\n\n-------------------------------------------------\n```\n\n\u003cbr\u003e\n\n# :watch: [idle_in_transaction.sql](scripts/idle_in_transaction.sql)\n\nJust put the psql straightly on the cron job to find queries \"idle in transaction\" (running every 2 minutes below)\n```\n*/2 * * * * psql -h localhost -U postgres -d postgres -p 5432 --no-psqlrc -q -t -A -c \"SELECT TO_CHAR(NOW(), 'DD-MM-YYYY HH:MI:SS'),pid,STATE,usename,client_addr,application_name,query FROM pg_stat_activity WHERE pid \u003c\u003e pg_backend_pid() AND STATE IN ( 'idle in transaction' ,'idle in transaction (aborted)', 'disabled' ) AND state_change \u003c current_timestamp - '2 minutes'::INTERVAL\" \u003e\u003e /var/log/pg_idle_tx_conn.log\n```\n* You can change the output symbol from \"\u003e\u003e\" to \"\u003e\" if you want to reset the log file for each entry instead of increment the file otherwise would be recommended develop a log rotate plan, just in case\n* A good strategy is to configure your alert system tool to read the log file on every change and send a notification\n* Don't forget to create the log file with the correct permissions\n\nOutput:\n```\n20-07-2020 09:52:01|14779|idle in transaction|juliano|192.168.0.1|psql|SELECT 'Juliano detection test';\n```\n\n\u003cbr\u003e\n\n# ⏩ [port-forward_k8s.sh](scripts/port-forward_k8s.sh)\nThis script maps all database instances and ports using k8s ```port-forward``` from each remote environment to your local machine (```localhost``` or ```127.0.0.1```). It can be used also to access AWS RDS Aurora Postgres and MySQL instances.\n\nIf you don’t want to use this bash script you can use [kubefwd](https://github.com/txn2/kubefwd) instead.\n\nBefore start list the service names and ports running on kubernetes using the command: ```kubectl --context=context-name-here -n dba get services```\n\n1. Edit the script with your instances addresses, they must be already configured to use ```port-forward``` on k8s as a requisite, then run the script ```./port-forward_k8s.sh```\n\nOnce the script is running you can using you prefred cli or gui on localhost.\n\n2. To stop it list the opened ports ```ps -ef | grep kubectl``` and then just kill the connection you want ```kill -9 pid_here``` or all connections ```pkill -9 kubectl```\n\n\u003cbr\u003e\n\n# ☸️ [k8s_sql_connect.sh](/scripts/k8s_sql_connect.sh)\nScript to quick automate and direct connect to an AWS RDS instance using K8S pods\nRun ```./k8s_sql_connect.sh -h``` or ```--help``` to see how it works:\n\n```\nUsage:\n     ./psql_k8s.sh -c [context]  -i [instance]\n\nE.g. ./psql_k8s.sh -c staging -i pg-staging01\n\n     -c, --context    --\u003e    -i, --instance\n     1. staging       --\u003e    pg-staging01 | pg-staging02 | mysql-staging01\n     2. production    --\u003e    pg-production01 | pg-production02 | mysql-production01\n \n     -h, --help        This message\n```\n\n# [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)\n\n# [![Buy Me Coffee](coffe.png)](https://www.paypal.me/julianotech)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdbajuliano%2Fpostgres-dba-scripts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdbajuliano%2Fpostgres-dba-scripts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdbajuliano%2Fpostgres-dba-scripts/lists"}