{"id":31661861,"url":"https://github.com/postgres-ai/postgres_ai","last_synced_at":"2025-10-07T19:03:01.659Z","repository":{"id":312855141,"uuid":"1047209888","full_name":"postgres-ai/postgres_ai","owner":"postgres-ai","description":"Expert-level Postgres monitoring tool designed for humans and AI systems","archived":false,"fork":false,"pushed_at":"2025-10-07T12:27:44.000Z","size":8650,"stargazers_count":15,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-07T12:29:56.700Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/postgres-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-29T23:45:18.000Z","updated_at":"2025-10-02T09:49:31.000Z","dependencies_parsed_at":"2025-09-02T12:49:01.589Z","dependency_job_id":"a2112564-a327-437c-a930-65f362ce140d","html_url":"https://github.com/postgres-ai/postgres_ai","commit_stats":null,"previous_names":["postgres-ai/postgres_ai"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/postgres-ai/postgres_ai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgres-ai%2Fpostgres_ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgres-ai%2Fpostgres_ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgres-ai%2Fpostgres_ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgres-ai%2Fpostgres_ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/postgres-ai","download_url":"https://codeload.github.com/postgres-ai/postgres_ai/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgres-ai%2Fpostgres_ai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278830043,"owners_count":26053223,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-07T19:01:39.950Z","updated_at":"2025-10-07T19:03:01.650Z","avatar_url":"https://github.com/postgres-ai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# postgres_ai monitoring\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![GitLab](https://img.shields.io/badge/GitLab-postgres--ai%2Fpostgres__ai-orange?logo=gitlab)](https://gitlab.com/postgres-ai/postgres_ai)\n[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-14%2B-blue?logo=postgresql)](https://www.postgresql.org/)\n\n**Expert-level Postgres monitoring tool designed for humans and AI systems**\n\nBuilt for senior DBAs, SREs, and AI systems who need rapid root cause analysis and deep performance insights. This isn't a tool for beginners — it's designed for Postgres experts who need to understand complex performance issues in minutes, not hours.\n\n**Part of [Self-Driving Postgres](https://postgres.ai/blog/20250725-self-driving-postgres)** - postgres_ai monitoring is a foundational component of PostgresAI's open-source Self-Driving Postgres (SDP) initiative, providing the advanced monitoring and intelligent root cause analysis capabilities essential for achieving higher levels of database automation.\n\n![postgres_ai monitoring](assets/postgres_ai_pic.png)\n\n## 🎯 Key highlights\n\n- **Top-down troubleshooting methodology**: Follows the Four Golden Signals approach (Latency, Traffic, Errors, Saturation)\n- **Expert-focused design**: Assumes deep Postgres knowledge and performance troubleshooting experience  \n- **Dual-purpose architecture**: Built for both human experts and AI systems requiring structured performance data\n- **Comprehensive query analysis**: Complete `pg_stat_statements` metrics with historical trends and plan variations\n- **Active Session History**: Postgres's answer to Oracle ASH and AWS RDS Performance Insights\n- **Hybrid storage**: Prometheus for metrics, Postgres for query texts — best of both worlds\n\n\u003e 📖 **Read more**: [postgres_ai monitoring v0.7 announcement](https://postgres.ai/blog/20250722-postgres-ai-v0-7-expert-level-postgresql-monitoring) - detailed technical overview and architecture decisions.\n\n## ⚠️ Important notice\n\n**This tool is NOT for beginners.** It requires extensive Postgres knowledge and assumes familiarity with:\n- Advanced Postgres internals and performance concepts\n- Query plan analysis and optimization techniques  \n- Wait event analysis and system-level troubleshooting\n- Production database operations and incident response\n\nIf you're new to Postgres, consider starting with simpler monitoring solutions before using postgres_ai.\n\n## 🚀 Live demo\n\nExperience the full monitoring solution: **https://demo.postgres.ai** (login: `demo` / password: `demo`)\n\n## 📊 Five expert dashboards\n\n1. **Troubleshooting dashboard** - Four Golden Signals with immediate incident response insights\n2. **Query performance analysis** - Top-N query workload analysis with resource consumption breakdowns  \n3. **Single query analysis** - Deep dive into individual query performance and plan variations\n4. **Wait event analysis** - Active Session History for session-level troubleshooting\n5. **Backups and DR** - WAL archiving monitoring with RPO measurements\n\n## 🏗️ Architecture\n\n- **Collection**: pgwatch v3 (by Cybertec) for metrics gathering\n- **Storage**: Prometheus for time-series data + Postgres for query texts\n- **Visualization**: Grafana with expert-designed dashboards\n- **Analysis**: Structured data output for AI system integration\n\n## 📋 Requirements\n\n**Infrastructure:**\n- **Linux machine** with Docker installed (separate from your database server)\n- **Docker access** - the user running `postgres_ai` must have Docker permissions\n- **Access (network and pg_hba)** to the Postgres database(s) you want to monitor\n\n**Database:**\n- Supports Postgres versions 14-18\n- **pg_stat_statements extension must be created** for the DB used for connection\n\n## ⚠️ Security Notice\n\n**WARNING: Security is your responsibility!**\n\nThis monitoring solution exposes several ports that **MUST** be properly firewalled:\n- **Port 3000** (Grafana) - Contains sensitive database metrics and dashboards\n- **Port 58080** (PGWatch Postgres) - Database monitoring interface  \n- **Port 58089** (PGWatch Prometheus) - Database monitoring interface\n- **Port 59090** (Prometheus) - Metrics storage and queries\n- **Port 59091** (PGWatch Prometheus endpoint) - Metrics collection\n- **Port 55000** (Flask API) - Backend API service\n- **Port 55432** (Demo DB) - When using `--demo` option\n- **Port 55433** (Metrics DB) - Postgres metrics storage\n\n**Configure your firewall to:**\n- Block public access to all monitoring ports\n- Allow access only from trusted networks/IPs\n- Use VPN or SSH tunnels for remote access\n\nFailure to secure these ports may expose sensitive database information!\n\n## 🚀 Quick start\n\nCreate a new DB user in the database to be monitored (skip this if you want to just check out `postgres_ai` monitoring with a synthetic `demo` database):\n```sql\n-- Create a user for postgres_ai monitoring\nbegin;\ncreate user postgres_ai_mon with password '\u003cpassword\u003e';\n\ngrant connect on database \u003cdatabase_name\u003e to postgres_ai_mon;\n\ngrant pg_monitor to postgres_ai_mon;\ngrant select on pg_stat_statements to postgres_ai_mon;\ngrant select on pg_stat_database to postgres_ai_mon;\ngrant select on pg_stat_user_tables to postgres_ai_mon;\n\n-- Create a public view for pg_statistic access (required for bloat metrics on user schemas)\ncreate view public.pg_statistic as\nselect \n    n.nspname as schemaname,\n    c.relname as tablename,\n    a.attname,\n    s.stanullfrac as null_frac,\n    s.stawidth as avg_width,\n    false as inherited\nfrom pg_statistic s\njoin pg_class c on c.oid = s.starelid\njoin pg_namespace n on n.oid = c.relnamespace  \njoin pg_attribute a on a.attrelid = s.starelid and a.attnum = s.staattnum\nwhere a.attnum \u003e 0 and not a.attisdropped;\n\ngrant select on public.pg_statistic to pg_monitor;\nalter user postgres_ai_mon set search_path = \"$user\", public, pg_catalog;\ncommit;\n```\n\n**One command setup:**\n\n```bash\n# Download the CLI\ncurl -o postgres_ai https://gitlab.com/postgres-ai/postgres_ai/-/raw/main/postgres_ai \\\n  \u0026\u0026 chmod +x postgres_ai\n```\n\nNow, start it and wait for a few minutes. To obtain a PostgresAI access token for your organization, visit https://console.postgres.ai (`Your org name → Manage → Access tokens`):\n\n```bash\n# Production setup with your Access token\n./postgres_ai quickstart --api-key=your_access_token\n```\n**Note:** You can also add your database instance in the same command:\n```bash\n./postgres_ai quickstart --api-key=your_access_token --add-instance=\"postgresql://user:pass@host:port/DB\"\n```\n\nOr if you want to just check out how it works:\n```bash\n# Complete setup with demo database\n./postgres_ai quickstart --demo\n```\n\nThat's it! Everything is installed, configured, and running.\n\n## 📊 What you get\n\n- **Grafana Dashboards** - Visual monitoring at http://localhost:3000\n- **Postgres Monitoring** - PGWatch with comprehensive metrics\n- **Automated Reports** - Daily performance analysis\n- **API Integration** - Automatic upload to PostgresAI\n- **Demo Database** - Ready-to-use test environment\n\n## 🎯 Use cases\n\n**For developers:**\n```bash\n./postgres_ai quickstart --demo\n```\nGet a complete monitoring setup with demo data in under 2 minutes.\n\n**For production:**\n```bash\n./postgres_ai quickstart --api-key=your_key\n# Then add your databases\n./postgres_ai add-instance \"postgresql://user:pass@host:port/DB\"\n```\n\n## 🔧 Management commands\n\n```bash\n# Instance management\n./postgres_ai add-instance \"postgresql://user:pass@host:port/DB\"\n./postgres_ai list-instances\n./postgres_ai test-instance my-DB\n\n# Service management  \n./postgres_ai status\n./postgres_ai logs\n./postgres_ai restart\n\n# Health check\n./postgres_ai health\n```\n\n## 🌐 Access points\n\nAfter running quickstart:\n\n- **🚀 MAIN: Grafana Dashboard**: http://localhost:3000 (login: `monitoring`; password is shown at the end of quickstart)\n\nTechnical URLs (for advanced users):\n- **Demo DB**: postgresql://postgres:postgres@localhost:55432/target_database\n- **Monitoring**: http://localhost:58080 (PGWatch)\n- **Metrics**: http://localhost:59090 (Prometheus)\n\n## 📖 Help\n\n```bash\n./postgres_ai help\n```\n\n## 🔑 PostgresAI access token\nGet your access token at [PostgresAI](https://postgres.ai) for automated report uploads and advanced analysis.\n\n## 🛣️ Roadmap\n\n- Host stats for on-premise and managed Postgres setups\n- `pg_wait_sampling` and `pg_stat_kcache` extension support\n- Additional expert dashboards: autovacuum, checkpointer, lock analysis\n- Query plan analysis and automated recommendations\n- Enhanced AI integration capabilities\n\n## 🤝 Contributing\n\nWe welcome contributions from Postgres experts! Please check our [GitLab repository](https://gitlab.com/postgres-ai/postgres_ai) for:\n- Code standards and review process\n- Dashboard design principles\n- Testing requirements for monitoring components\n\n## 📄 License\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.\n\n## 🏢 About PostgresAI\n\npostgres_ai monitoring is developed by [PostgresAI](https://postgres.ai), bringing years of Postgres expertise into automated monitoring and analysis tools. We provide enterprise consulting and advanced Postgres solutions for fast-growing companies.\n\n## 📞 Support \u0026 community\n\n- 💬 [Get support](https://postgres.ai/contact)\n- 📺 [Postgres.TV (YouTube)](https://postgres.tv)\n- 🎙️ [Postgres FM Podcast](https://postgres.fm)\n- 🐛 [Report issues](https://gitlab.com/postgres-ai/postgres_ai/-/issues)\n- 📧 [Enterprise support](https://postgres.ai/consulting)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpostgres-ai%2Fpostgres_ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpostgres-ai%2Fpostgres_ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpostgres-ai%2Fpostgres_ai/lists"}