{"id":42850276,"url":"https://github.com/ivanrododendro/aiqo-report","last_synced_at":"2026-05-23T08:00:36.948Z","repository":{"id":324530735,"uuid":"1097542417","full_name":"ivanrododendro/aiqo-report","owner":"ivanrododendro","description":"AI Query Optimizer Report","archived":false,"fork":false,"pushed_at":"2026-05-21T19:36:57.000Z","size":6879,"stargazers_count":2,"open_issues_count":6,"forks_count":0,"subscribers_count":0,"default_branch":"dev","last_synced_at":"2026-05-22T04:44:33.154Z","etag":null,"topics":["ai","sql","sql-optimization","sql-tuning"],"latest_commit_sha":null,"homepage":"https://aiqueryoptimizer.com/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ivanrododendro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-11-16T11:37:35.000Z","updated_at":"2026-05-21T19:35:43.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ivanrododendro/aiqo-report","commit_stats":null,"previous_names":["ivanrododendro/aiqo-report"],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/ivanrododendro/aiqo-report","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivanrododendro%2Faiqo-report","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivanrododendro%2Faiqo-report/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivanrododendro%2Faiqo-report/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivanrododendro%2Faiqo-report/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ivanrododendro","download_url":"https://codeload.github.com/ivanrododendro/aiqo-report/tar.gz/refs/heads/dev","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ivanrododendro%2Faiqo-report/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33387656,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-23T04:15:53.637Z","status":"ssl_error","status_checked_at":"2026-05-23T04:15:53.242Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","sql","sql-optimization","sql-tuning"],"created_at":"2026-01-30T12:02:19.595Z","updated_at":"2026-05-23T08:00:36.933Z","avatar_url":"https://github.com/ivanrododendro.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AIQO PostgreSQL AI Report Generator\n\nThe AIQO PostgreSQL AI Report Generator is a powerful command-line tool designed to analyze PostgreSQL `auto_explain` logs using Artificial Intelligence. It provides query tracking over time and actionable insights and optimization suggestions to improve database performance, presenting its findings in a comprehensive HTML report.\n\nIt is well suited to scenarios where complexity, data volume and fragmentation make it difficult or impossible to reproduce the execution plan outside the production environment.\n\nDownload the [sample HTML report](https://github.com/ivanrododendro/aiqo-report/releases/download/1.4.2/sample_report.zip) or the [target query report for query 421865](https://github.com/ivanrododendro/aiqo-report/releases/download/1.4.2/target_query_report_421865.zip) for a randomized dataset created using explain.dalibo.com sample plans. AI hint is at day may 4, real reports are more meaningful due to detailed context you can provide to AI analisys.\n\n![](docs/infographic.webp)\n\n## Features\n\n*   **Comprehensive HTML Reports**: Generates detailed, easy-to-read HTML reports summarizing performance metrics, queries timelines, AI findings and optimization opportunities.\n*   **Query Code Tracking**: Generates a unique \"query code\" (hash) for each normalized SQL query, enabling consistent tracking and application of query-specific optimizations across different log executions.\n*   **AI-Powered Analysis with Customizable AI Models**: Leverages large language models (LLMs) to analyze `EXPLAIN` plans from PostgreSQL logs and identify performance bottlenecks. Provides concrete recommendations for query, server, and infrastructure optimizations based on AI analysis. By default, AI analysis is executed only once for each query code. Subsequent occurrences are kept in the report but marked as already analyzed instead of triggering a new AI call. Supports various AI providers and models (e.g. ChatGPT, Gemini, DeepSeek, ...) via `litellm`. Supports provider-side prompt caching for compatible LLM providers, with an option to disable it when required.\n*   **Exceution plan compare**: The report allows to compare execution plans for two occurencies of a same query to analyze the evolution over time.\n*   **Flexible Contextualization**: Allows users to provide DDL, server configuration, infrastructure details, and custom prompts to enhance AI analysis accuracy.   \n*   **Native Builds Available**: Standalone native executables are available for Linux, macOS (Apple Silicon), and Windows.\n*   **Multilingual Output**: Supports generating reports in different languages.\n*   **Reproducible Outputs**: When using OpenAI models, analyses can be reproduced by providing the same input and context, ensuring consistent results across runs.\n*   **Privacy-First Anonymization**: Before any data is sent to an AI provider, all PostgreSQL object names (tables, schemas, indexes, columns, custom functions) are replaced with deterministic aliases (e.g. `tbl_3a1f9c`). The AI response is de-anonymized before it appears in the report, so the HTML output always shows the real names. Anonymization is enabled by default and can be disabled with `--no-anonymize`.\n\n## API Keys and LiteLLM Configuration\n\nThis tool uses `litellm` to interface with various AI providers. You must configure your API keys as environment variables.\n\nFor example, for OpenAI models:\n\n```bash\nset OPENAI_API_KEY=\"your_openai_api_key_here\"\n```\n\nFor Gemini models:\n\n```bash\nset GEMINI_API_KEY=\"your_gemini_api_key_here\"\n```\n\nYou can also use a generic `LITELLM_API_KEY` if you are using a provider that `litellm` supports via this generic key. Refer to the [LiteLLM documentation](https://litellm.ai/docs/providers) for specific provider configurations.\n\nTo route analysis through a LiteLLM Proxy Server, use the `litellm_proxy/\u003cmodel-or-alias\u003e` model prefix and configure the proxy endpoint:\n\n```bash\nset LITELLM_API_BASE=\"http://localhost:4000\"\nset LITELLM_API_KEY=\"your_litellm_proxy_key_here\"\n\n./pg_aiqo_report /path/to/your/postgresql.log --model \"litellm_proxy/pg-report-model\"\n```\n\nThe proxy must expose a model or alias matching the value after `litellm_proxy/`. You can also pass the proxy settings explicitly with `--litellm-api-base` and `--litellm-api-key`.\n\n## Basic Usage\n\nTo analyze a PostgreSQL log file with default settings:\n\n```bash\n./pg_aiqo_report /path/to/your/postgresql.log\n```\n\nThis will generate an HTML report in the current working directory (or `output/` if it exists), named similarly to `pg-ai-report_\u003ctimestamp\u003e.html`.\n\nBy default, the AI analysis is executed only on the first occurrence of each query code. Subsequent occurrences are reported with a message indicating that the same query was already analyzed earlier. Use `--analyze-all-queries` if you want an independent AI analysis for every matching log entry.\n\n## PostgreSQL Configuration\n\nTo generate the necessary log files for analysis, your PostgreSQL instance must be configured to use `auto_explain`. Here's a typical configuration to add to your `postgresql.conf`:\n\n```ini\n# auto_explain settings\nshared_preload_libraries = 'auto_explain'\nauto_explain.log_min_duration = 60000 # Log queries that exceed duration target\nauto_explain.log_analyze = on # Include EXPLAIN ANALYZE output\nauto_explain.log_buffers = on # Include buffer usage\nauto_explain.log_timing = off # Exclude unecessary detailed timing information\nauto_explain.log_nested_pages = on # For nested queries\nauto_explain.log_verbose = on # For verbose output\nauto_explain.log_format = text # Or json\n```\n\n\u003e **_NOTE:_** when auto_explain.log_timing parameter is on, per-plan-node timing occurs for all statements executed, whether or not they run long enough to actually get logged. This can have an extremely negative impact on performance. Turning off auto_explain.log_timing ameliorates the performance cost, at the price of obtaining less information. See https://www.postgresql.org/docs/current/auto-explain.html\n\nAfter modifying `postgresql.conf`, restart your PostgreSQL server for the changes to take effect. The tool expects standard text-based PostgreSQL log files.\n\n## Context Files \u0026 Location\n\nThe tool supports two distinct context sources:\n\n1. **Built-in AI instruction prompts** loaded from `src/aiqo_pg_ai_report/prompts/`:\n   - `SYSTEM.txt`\n   - `FORMAT.txt`\n   - `GENERAL_HINTS_SYNTHESIS.txt`\n   - `TARGET_QUERY_SYSTEM.txt`\n   - `TARGET_QUERY_FORMAT.txt`\n\n   These files are required by the application and are always loaded from the package prompts directory.\n\n2. **Application contexts and optimization history** loaded from a context folder:\n   - `DDL.txt`\n   - `CONFIG.txt`\n   - `PROJECT.txt`\n   - `SERVER.txt`\n   - `EVENTS.txt`\n   - `QUERIES/\u003cquery_code_prefix\u003e.txt`\n\n   Where `\u003cquery_code_prefix\u003e` is the first 6 characters of the normalized query hash.\n\n### Default context folder behavior\n\nIf `--context-folder` is **not** provided, the default application context folder is:\n\n- **Single log file mode**: `CONTEXT/` under the directory containing the log file.\n- **Directory mode** (when a directory is analyzed): `CONTEXT/` inside that analyzed directory.\n\nExamples:\n\n- Log file: `/var/log/postgresql/postgresql.log` -\u003e default context folder: `/var/log/postgresql/CONTEXT/`\n- Log directory: `/var/log/postgresql/` -\u003e default context folder: `/var/log/postgresql/CONTEXT/`\n\nIf `--context-folder` is provided, that path is used as-is and overrides the default behavior.\n\nTo use custom contexts, create a folder (e.g., `my_custom_contexts/`) and place your context files within it according to the following structure:\n\n```\nmy_custom_contexts/\n├── DDL.txt\n├── CONFIG.txt\n├── PROJECT.txt\n├── SERVER.txt\n├── EVENTS.txt\n└── QUERIES/\n    ├── \u003cquery_code_prefix_1\u003e.txt\n    ├── \u003cquery_code_prefix_2\u003e.txt\n    └── ...\n```\n\nTo specify your custom context folder, use the `--context-folder` argument. For example:\n\n```bash\n./pg_aiqo_report \\\n    --context-folder \"./my_custom_contexts\" \\\n    /path/to/your/postgresql.log\n```\n\n**Default Model**:\nThe default model is `gemini-3.1-flash-lite-preview`. You can specify a different model using the `--model` CLI argument.\n\n## Advanced Usage\n\nYou can customize the analysis using various command-line arguments:\n\n```bash\n./pg_aiqo_report \\\n    --model \"gpt-4o-mini\" \\\n    --language \"en\" \\\n    --format \"json\" \\\n    --context-folder \"./my_custom_contexts\" \\\n    --custom-prompt \"Focus on index usage.\" \\\n    --only-seq-scan-ai-analysis \\\n    --filter \"SELECT * FROM users\" \\\n    --limit-ai-calls 5 \\\n    --ai-call-timeout 120 \\\n    --analyze-all-queries \\\n    --disable-provider-cache \\\n    /path/to/your/postgresql.log\n```\n\n### Full CLI Parameters \n\n*   **`log_file_path`** (positional argument):\n    *   The full path to the PostgreSQL `auto_explain` log file to be analyzed.\n    *   Example: `/var/log/postgresql/postgresql-2023-01-01.log`\n\n*   **`--model \u003cMODEL\u003e`** (`-m`):\n    *   Specify the AI model to use for analysis (e.g., `gpt-4o`, `gemini-1.5-pro`, `o1-mini`, `litellm_proxy/pg-report-model`).\n    *   Default: `gemini-3.1-flash-lite-preview`\n\n*   **`--format \u003cFORMAT\u003e`** (`-fmt`):\n    *   Specify the log format to parse: `text` (default), `json` (supported), `yaml` (currently unsupported).\n    *   Default: `text`\n\n*   **`--limit-ai-calls \u003cNUMBER\u003e`** (`-l`):\n    *   Limits the maximum number of AI calls made during the analysis. Use `-1` for unlimited calls.\n    *   In target query mode (`--target-query`), limits the number of matching query occurrences included in the aggregated analysis instead of limiting AI calls.\n    *   Default: `-1` (unlimited)\n\n*   **`--analyze-all-queries`** (`-aaa`):\n    *   Forces an independent AI analysis for every query occurrence found in the logs.\n    *   By default, the tool analyzes only the first occurrence of each query code and marks the following occurrences as already analyzed.\n    *   This is a flag, no value needed.\n    *   Default: `False`\n\n*   **`--ai-call-timeout \u003cSECONDS\u003e`**:\n    *   Sets the timeout duration for each individual AI call in seconds.\n    *   Default: `90` seconds\n\n*   **`--disable-provider-cache`**:\n    *   Disable provider-side prompt caching for all supported LLM providers.\n    *   When enabled, OpenAI requests are sent without `prompt_cache_key`, and Gemini/Claude requests are sent without provider cache markers.\n    *   This is a flag, no value needed.\n    *   Default: `False`\n\n*   **`--litellm-api-base \u003cURL\u003e`**:\n    *   Sets the LiteLLM Proxy Server API base URL for models using the `litellm_proxy/` prefix.\n    *   Falls back to the `LITELLM_API_BASE` environment variable.\n\n*   **`--litellm-api-key \u003cKEY\u003e`**:\n    *   Sets the LiteLLM Proxy Server API key for models using the `litellm_proxy/` prefix.\n    *   Falls back to the `LITELLM_API_KEY` environment variable.\n    *   Optional for proxy deployments that do not require authentication.\n\n*   **`--language \u003cLANG\u003e`**:\n    *   Set the output language for the generated report and AI analysis.\n    *   Default: `fr` (French)\n    *   Example: `--language en` for English output.\n\n*   **`--skip-ai-analysis`** (`-s`):\n    *   If set, the AI analysis step will be skipped entirely. A report will still be generated, but without AI-driven insights.\n    *   This is a flag, no value needed.\n    *   Default: `False`\n\n*   **`--only-seq-scan-ai-analysis`** (`-o`):\n    *   If set, the AI analysis will only be performed on queries that are identified as performing sequential scans. This helps focus the AI's efforts on specific performance issues.\n    *   This is a flag, no value needed.\n    *   Default: `False`\n\n*   **`--filter \u003cSTRING\u003e`** (`-f`):\n    *   Filter log entries. Only log entries containing the specified string in the query name, job name, SQL text, or query code will be included in the report and processed for AI analysis. Can be specified multiple times. Case-insensitive.\n    *   Example: `--filter \"public.users\"` or `--filter \"2a3b4c\"` (to filter by a query code)\n    *   Default: `None` (no filter)\n\n*   **`--custom-prompt \u003cPROMPT\u003e`** (`-c`):\n    *   Provide an additional custom prompt or instruction to the AI for its analysis. This prompt will be appended to the standard prompts.\n    *   Example: `--custom-prompt \"Pay special attention to JOIN operations.\"`\n\n*   **`--version`** (`-v`):\n    *   Show the current tool version (from git tags) and the detected `litellm` version, then exit.\n\n*   **`--supported-models`** (`-sm`):\n    *   Show the models reported by `litellm.models()` with provider information, then exit.\n    *   Default: `None`\n\n*   **`--report-filename \u003cPATH\u003e`** (`-r`):\n    *   Override the HTML report filename.\n    *   Example: `--report-filename my_custom_report.html`\n    *   Default: Automatically generated based on log filename\n\n*   **`--context-folder \u003cPATH\u003e`** (`-cf`):\n    *   Path to a directory containing application context files and optimization history (`DDL.txt`, `CONFIG.txt`, `PROJECT.txt`, `SERVER.txt`, `EVENTS.txt`, and `QUERIES/*.txt`).\n    *   Example: `--context-folder /home/user/my_db_contexts`\n    *   Default: `CONTEXT/` under the analyzed path (for a log file: the log file's parent directory; for directory mode: the analyzed directory itself)\n\n*   **`--no-anonymize`**:\n    *   Disable the anonymization layer. DB object names (tables, schemas, indexes, columns, custom functions) will be sent to the AI provider as-is.\n    *   By default, all names are replaced with deterministic aliases before each AI call and restored in the response, so the HTML report always shows real names.\n    *   This is a flag, no value needed.\n    *   Default: `False` (anonymization is ON)\n\n*   **`--debug`** (`-d`):\n    *   Enable debug logging.\n    *   This is a flag, no value needed.\n    *   Default: `False`\n\n## Output Report\n\nThe tool generates a single, self-contained HTML report. This report typically includes:\n\n*   **Summary Dashboard**: Overview of analyzed queries, AI calls, and total costs.\n*   **Daily Statistics**: Breakdown of query activity and AI analysis per day.\n*   **Query Statistics**: Aggregated information about normalized queries, including frequency and average execution times.\n*   **Detailed Query Analysis**: For each significant query (especially those identified with issues like sequential scans or based on AI analysis), a dedicated section will provide:\n    *   The original SQL query.\n    *   The `EXPLAIN ANALYZE` plan.\n    *   The AI's summary of the plan's performance characteristics.\n    *   AI-generated optimization recommendations specific to that query.\n    *   PEV2 Query visualizer\n    *   Graph tracking qeury statistics over time\n\n## Build Dependencies \u0026 Requirements\n\n*   Python \u003e= 3.11 \u003c= 3.13\n*   The project uses `poetry` for dependency management.\n*   Key Python packages: `litellm`, `sqlparse`, `Jinja2`.\n\nTo install the required dependencies, navigate to the project root and run:\n\n```bash\npoetry install\n```\n\n## Native Binaries\n\nStandalone binaries are produced with the same base name on every platform:\n\n- Linux: `dist/linux/pg_aiqo_report`\n- macOS (Apple Silicon): `dist/macos-silicon/pg_aiqo_report`\n- Windows: `dist/windows/pg_aiqo_report.exe`\n\nBuild them with:\n\n```bash\npoetry install --with dev\n./scripts/build_nuitka.sh \u003clinux|macos-silicon|windows\u003e\n```\n\nThe GitHub Actions release workflow persists Nuitka caches between runs by restoring `${GITHUB_WORKSPACE}/.github-cache/nuitka/\u003ctarget\u003e` through `actions/cache`. When `ccache` is available, the build script points Nuitka to that cached compiler store automatically.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivanrododendro%2Faiqo-report","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fivanrododendro%2Faiqo-report","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fivanrododendro%2Faiqo-report/lists"}