{"id":43189839,"url":"https://github.com/fbz-tec/pgxport","last_synced_at":"2026-02-01T04:37:30.025Z","repository":{"id":319858669,"uuid":"1079773867","full_name":"fbz-tec/pgxport","owner":"fbz-tec","description":"A CLI tool to export PostgreSQL query results to CSV, JSON, YAML, XML, SQL, XLSX and other formats.","archived":false,"fork":false,"pushed_at":"2025-11-27T16:20:45.000Z","size":2419,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-28T04:51:38.083Z","etag":null,"topics":["cli","cobra-cli","csv","data-export","golang","json","pgx-v5","postgresql","sql","xml","xslx","yaml"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fbz-tec.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-20T11:15:46.000Z","updated_at":"2025-11-27T16:20:48.000Z","dependencies_parsed_at":"2025-10-23T17:40:54.799Z","dependency_job_id":null,"html_url":"https://github.com/fbz-tec/pgxport","commit_stats":null,"previous_names":["fbz-tec/pgexport","fbz-tec/pgxport"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/fbz-tec/pgxport","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbz-tec%2Fpgxport","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbz-tec%2Fpgxport/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbz-tec%2Fpgxport/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbz-tec%2Fpgxport/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fbz-tec","download_url":"https://codeload.github.com/fbz-tec/pgxport/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fbz-tec%2Fpgxport/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28968051,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T03:46:10.227Z","status":"ssl_error","status_checked_at":"2026-02-01T03:46:01.693Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","cobra-cli","csv","data-export","golang","json","pgx-v5","postgresql","sql","xml","xslx","yaml"],"created_at":"2026-02-01T04:37:29.061Z","updated_at":"2026-02-01T04:37:30.015Z","avatar_url":"https://github.com/fbz-tec.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"logo.png\" alt=\"pgXport\" width=\"60%\"/\u003e\n\u003c/p\u003e\n\n[![CI - Build, Test \u0026 Release](https://github.com/fbz-tec/pgxport/actions/workflows/ci.yml/badge.svg)](https://github.com/fbz-tec/pgxport/actions/workflows/ci.yml)\n[![Go Report Card](https://goreportcard.com/badge/github.com/fbz-tec/pgxport)](https://goreportcard.com/report/github.com/fbz-tec/pgxport)\n[![License](https://img.shields.io/github/license/fbz-tec/pgxport.svg)](LICENSE)\n\nA simple, powerful and efficient CLI tool to export PostgreSQL query results to various formats (CSV, XML, JSON ,YAML ,XLSX ,SQL, template).\n\n---\n\n## 📚 Table of Contents\n- [✨ Features](#-features)\n- [📦 Installation](#-installation)\n- [⚙️ Configuration](#️-configuration)\n- [📖 Usage](#-usage)\n- [📊 Output Formats](#-output-formats)\n- [🔍 Verbose Mode](#-verbose-mode)\n- [📄 Format Details](#-format-details)\n- [🛠️ Development](#️-development)\n- [🔒 Security](#-security)\n- [🚨 Error Handling](#-error-handling)\n- [🤝 Contributing](#-contributing)\n- [📄 License](#-license)\n- [🗺️ Roadmap](#️-roadmap)\n- [💬 Support](#-support)\n- [🙏 Acknowledgments](#-acknowledgments)\n- [⭐ Show Your Support](#-show-your-support)\n\n---\n\n## ✨ Features\n\n- 🚀 Execute SQL queries directly from command line\n- 📄 Run SQL queries from files\n- 📊 Export to **CSV**, **JSON**, **XML**, **YAML** ,  **SQL** , **Microsoft Excel (XLSX)** and **Template** for custom output formats\n- ⚡ High-performance CSV export using PostgreSQL native **COPY** mode (`--with-copy`)\n- 🔧 Customizable CSV delimiter and header\n- 🗜️ Compression: **gzip** / **zip** / **zstd** / **lz4**\n- ⚙️ Simple configuration via environment variables or `.env` file\n- 🔗 DSN connection string support (`--dsn`)\n- 🔗 **Individual connection flags** for maximum flexibility\n- 🛡️ Robust error handling and validation\n- ⚠️ Fail on empty results (`--fail-on-empty`) for scripts \u0026 pipelines\n- 🔍 Verbose mode for detailed logging\n- ⚡ Optimized for performance with buffered I/O\n- 🔄 Batch INSERT statements for SQL exports (`--insert-batch`) for improved import performance\n- 🎯 Built with [Cobra](https://github.com/spf13/cobra)\n\n## 📦 Installation\n\n### Prerequisites\n\n- Go 1.20 or higher\n- PostgreSQL database access\n\n### Option 1: Install via `go install` (Recommended)\n\n```bash\ngo install github.com/fbz-tec/pgxport@latest\n```\n\nVerify installation:\n```bash\npgxport version\n```\n\n### Option 2: Download pre-built binaries\n\nDownload from [GitHub Releases](https://github.com/fbz-tec/pgxport/releases/latest)\n\n\n### Option 3: Build from source\n\n```bash\ngit clone https://github.com/fbz-tec/pgxport.git\ncd pgxport\ngo build -o pgxport\n\n# (Optional) Install to your PATH\nsudo cp pgxport /usr/local/bin/\n```\n\n## ⚙️ Configuration\n\n### Option 1: Using `.env` file (recommended)\n\n```env\nDB_USER=myuser\nDB_PASS=mypassword\nDB_HOST=localhost\nDB_PORT=5432\nDB_NAME=mydb\n```\n\n**Advantages:**\n- ✅ Automatically loaded by pgxport\n- ✅ Keeps credentials local \u0026 secure\n\n### Option 2: Using environment variables\n\nConfigure database connection using environment variables:\n\n```bash\nexport DB_USER=your_username\nexport DB_PASS=your_password\nexport DB_HOST=localhost\nexport DB_PORT=5432\nexport DB_NAME=your_database\n```\n\n### Option 3: Using `--dsn` flag (Quick override)\n\nPass the connection string directly via command line:\n\n```bash\npgxport --dsn \"postgres://user:pass@host:port/dbname\" -s \"SELECT * FROM users\" -o users.csv\n```\n### Option 4: Using Individual Connection Flags\n\nFor maximum flexibility, specify each connection parameter individually:\n```bash\npgxport --user postgres --host localhost --port 5432 --database mydb --password secret \\\n        -s \"SELECT * FROM users\" -o users.csv\n```\n\n**Available flags:**\n- `--host` : Database host\n- `--port` : Database port\n- `--user` : Database username  \n- `--database` : Database name\n- `--password` : Database password\n\n**Advantages:**\n- ✅ Mix with `.env` file (override only what you need)\n- ✅ Compatible with shell variables\n- ✅ Fine-grained control over each parameter\n- ✅ Perfect for CI/CD and scripting\n\n### Configuration Priority\n\nThe system uses the following priority order:\n\n1. **Individual connection flags** (`--host`, `--port`, `--user`, `--database`, `--password`)\n2. **`--dsn` flag**\n3. **Environment variables** (`DB_HOST`, `DB_PORT`, `DB_USER`, `DB_NAME`, `DB_PASS`)\n4. **`.env` file**\n5. **Defaults**\n\n## 📖 Usage\n\n```bash\npgxport [command] [flags]\n```\n\n### Commands\n\n| Command | Description |\n|---------|-------------|\n| `pgxport` | Execute query and export results |\n| `pgxport version` | Show version information |\n| `pgxport --help` | Show help message |\n\n### Flags\n\n| Flag | Short | Description | Default | Required |\n|------|-------|-------------|---------|----------|\n| `--sql` | `-s` | SQL query to execute | - | * |\n| `--sqlfile` | `-F` | Path to SQL file | - | * |\n| `--output` | `-o` | Output file path | - | ✓ |\n| `--format` | `-f` | Output format (csv, json, yaml, xml, sql) | `csv` | No |\n| `--time-format` | `-T` | Custom date/time format | `yyyy-MM-dd HH:mm:ss` | No |\n| `--time-zone` | `-Z` | Time zone for date/time conversion | Local | No |\n| `--delimiter` | `-D` | CSV delimiter character | `,` | No |\n| `--no-header` | `-n` | Skip header row in output (CSV and XLSX) | `false` | No |\n| `--with-copy` | - | Use PostgreSQL native COPY for CSV export (faster for large datasets) | `false` | No |\n| `--xml-root-tag` | - | Sets the root element name for XML exports | `results` | No |\n| `--xml-row-tag` | - | Sets the row element name for XML exports | `row` | No |\n| `--tpl-file`         | -      | Path to full template file (non-streaming mode)                 | -        | No |\n| `--tpl-header`       | -      | Header template (streaming mode only)                           | -        | No       |\n| `--tpl-row`          | -      | Row template (streaming mode only) **⚠️ Required in streaming** | -        | Yes (streaming mode)  |\n| `--tpl-footer`       | -      | Footer template (streaming mode only)  | -        | No       |\n| `--fail-on-empty` | `-x` | Exit with error if query returns 0 rows | `false` | No |\n| `--table` | `-t` | Table name for SQL INSERT exports (supports schema.table) | - | For SQL format |\n| `--insert-batch` | - | Number of rows per INSERT statement for SQL exports | `1` | No |\n| `--compression` | `-z` | Compression (none, gzip, zip, zstd, lz4) | `none` | No |\n| `--dsn` | - | Database connection string | - | No |\n| `--verbose` | `-v` | Enable verbose output with detailed debug information | `false` | No |\n| `--quiet` | `-q` | Suppress all output except errors | `false` | No |\n| `--help` | `-h` | Show help message | - | No |\n| `--host` |`-H` | Database host | `localhost` | No* |\n| `--port` |`-P` | Database port | `5432` | No* |\n| `--user` |`-u`| Database username | - | No* |\n| `--database` |`-d` | Database name | - | No* |\n| `--password` |`-p` | Database password | - | No* |\n| `--progress` | - | Show a live spinner during export | `false` | No* |\n\n_* Either `--sql` or `--sqlfile` must be provided (but not both)_\n\n## 📊 Output Formats\n\n### Format Capabilities\n\n| Format | Compression | Timezone Support | COPY Mode |\n|---------|------------|------------------|-----------|\n| CSV | ✅ | ✅ | ✅ |\n| JSON | ✅ | ✅ | ❌ |\n| XML | ✅ | ✅ | ❌ |\n| YAML | ✅ | ✅ | ❌ |\n| SQL | ✅ | ✅ | ❌ |\n| XLSX | ✅ | ❌ | ❌ |\n| TEMPLATE | ✅ | ✅ | ❌ |\n\n### Common Flags (All Formats)\n- `--compression` - Enable compression (gzip/zip/zstd/lz4)\n- `--time-format` - Custom date/time format\n- `--time-zone` - Timezone conversion\n- `--fail-on-empty` - Fail if query returns 0 rows\n- `--verbose` - Detailed logging\n- `--quiet` - Suppress all output except errors\n- `--progress` – Show a live spinner during export (streaming formats only)\n\n### Format-Specific Flags\n\n| Format | Specific Flags | Description |\n|---------|----------------|-------------|\n| **CSV** | `--delimiter`\u003cbr\u003e`--no-header`\u003cbr\u003e`--with-copy` | Set delimiter character\u003cbr\u003eSkip header row\u003cbr\u003eUse PostgreSQL COPY mode |\n| **XML** | `--xml-root-tag`\u003cbr\u003e`--xml-row-tag` | Customize root element name\u003cbr\u003eCustomize row element name |\n| **SQL** | `--table`\u003cbr\u003e`--insert-batch` | Target table name (required)\u003cbr\u003eRows per INSERT statement |\n| **TEMPLATE** | `--tpl-file`\u003cbr\u003e`--tpl-header`\u003cbr\u003e`--tpl-row`\u003cbr\u003e`--tpl-footer` | Full mode template file\u003cbr\u003eStreaming header template\u003cbr\u003eStreaming row template (required)\u003cbr\u003eStreaming footer template |\n| **JSON** | *(none)* | Uses only common flags |\n| **YAML** | *(none)* | Uses only common flags |\n| **XLSX** | `--no-header` | Skip header row |\n\n### Examples\n\n#### Basic Examples\n\n```bash\n# Simple query export (uses .env file)\npgxport -s \"SELECT * FROM users WHERE active = true\" -o users.csv\n\n# Export with semicolon delimiter\npgxport -s \"SELECT id, name, email FROM users\" -o users.csv -D ';'\n\n# Skip header row with --no-header\npgxport -s \"SELECT id, name, email FROM users\" -o users.csv -f csv --no-header\n\n# Execute query from a SQL file\npgxport -F queries/monthly_report.sql -o report.csv\n\n# Show progress spinner during export\npgxport -s \"SELECT * FROM big_table\" -o big.csv --progress\n\n# Use the high-performance COPY mode for large CSV exports\npgxport -s \"SELECT * FROM big_table\" -o big_table.csv -f csv --with-copy\n\n# Export to JSON format\npgxport -s \"SELECT * FROM products\" -o products.json -f json\n\n# Export to XML format\npgxport -s \"SELECT * FROM orders\" -o orders.xml -f xml\n\n# Export to XML format with custom root and row tags\npgxport -s \"SELECT * FROM orders\" -o orders.xml -f xml --xml-root-tag=\"data\" --xml-row-tag=\"record\"\n\n# Export to SQL INSERT statements\npgxport -s \"SELECT * FROM products\" -o products.sql -f sql -t products_backup\n\n# Export to SQL INSERT statements with schema\npgxport -s \"SELECT * FROM products\" -o products.sql -f sql -t public.products_backup\n\n# Export to YAML format\npgxport -s \"SELECT * FROM products\" -o products.yaml -f yaml\n\n# Export with gzip compression\npgxport -s \"SELECT * FROM logs\" -o logs.csv -f csv -z gzip\n\n# Export with zip compression (creates logs.zip containing logs.csv)\npgxport -s \"SELECT * FROM logs\" -o logs.csv -f csv -z zip\n\n# Export with zstd compression\npgxport -s \"SELECT * FROM logs\" -o logs.csv -f csv -z zstd\n\n# Export to Excel XLSX format\npgxport -s \"SELECT * FROM products\" -o products.xlsx -f xlsx\n\n\n# Export using custom template (full mode)\npgxport -s \"SELECT * FROM users\" -o report.html -f template --tpl-file template.html\n\n# Export using streaming template mode\npgxport -s \"SELECT * FROM logs\" -o output.txt -f template \\\n        --tpl-header header.tpl --tpl-row row.tpl --tpl-footer footer.tpl\n\n# Generate Markdown documentation from database\npgxport -s \"SELECT table_name, column_name, data_type FROM information_schema.columns\" \\\n        -o schema.md -f template --tpl-file schema_doc.tpl\n\n# Create custom JSON format with template\npgxport -s \"SELECT * FROM products\" -o custom.json -f template --tpl-row jsonl.tpl\n\n# Check version\npgxport version\n```\n\n#### Handling Empty Results\n\nThe `--fail-on-empty` flag is useful for scripting and automation when you want to ensure your query returns data.\n\n```bash\n# Default behavior: Warning message but exit code 0\npgxport -s \"SELECT * FROM users WHERE 1=0\" -o empty.csv\n# Output: Warning: Query returned 0 rows. File created at empty.csv but contains no data rows.\n# Exit code: 0\n\n# Strict mode: Error and exit code 1\npgxport -s \"SELECT * FROM users WHERE 1=0\" -o empty.csv --fail-on-empty\n# Output: Error: export failed: query returned 0 rows\n# Exit code: 1\n\n# Use in shell scripts for validation\nif ! pgxport -s \"SELECT * FROM critical_data WHERE date = CURRENT_DATE\" \\\n             -o daily_export.csv --fail-on-empty; then\n    echo \"❌ Export failed or returned no data!\"\n    # Send alert, log error, etc.\n    exit 1\nfi\necho \"✅ Export successful with data\"\n\n# Combine with other flags\npgxport -s \"SELECT * FROM orders WHERE status = 'pending'\" \\\n        -o pending_orders.csv \\\n        --fail-on-empty \\\n        -z gzip\n\n# Use in CI/CD pipelines\npgxport -F validate_data.sql -o validation.csv --fail-on-empty || exit 1\n```\n\n**When to use `--fail-on-empty`:**\n- ✅ Data validation scripts\n- ✅ ETL pipelines where empty results indicate a problem\n- ✅ Automated reporting where no data is an error condition\n- ✅ CI/CD data quality checks\n- ✅ Scheduled exports that must contain data\n\n**When NOT to use `--fail-on-empty`:**\n- ❌ Exploratory queries where empty results are acceptable\n- ❌ Optional data exports\n- ❌ Queries with filters that may legitimately return no results\n\n#### Date/Time Formatting Examples\n\n```bash\n# Export with custom date format (European style)\npgxport -s \"SELECT * FROM events\" -o events.csv -T \"dd/MM/yyyy HH:mm:ss\"\n\n# Export with ISO 8601 format with milliseconds\npgxport -s \"SELECT * FROM logs\" -o logs.csv -T \"yyyy-MM-ddTHH:mm:ss.SSS\"\n\n# Export with US date format\npgxport -s \"SELECT * FROM orders\" -o orders.csv -T \"MM/dd/yyyy HH:mm:ss\"\n\n# Export with timezone conversion to UTC\npgxport -s \"SELECT * FROM events\" -o events.csv -Z \"UTC\"\n\n# Export with timezone conversion to America/New_York\npgxport -s \"SELECT * FROM events\" -o events.csv -Z \"America/New_York\"\n\n# Combine custom format and timezone\npgxport -s \"SELECT created_at FROM users\" -o users.csv \\\n  -T \"dd/MM/yyyy HH:mm:ss\" -Z \"Europe/Paris\"\n\n# Export to JSON with custom date format and timezone\npgxport -s \"SELECT * FROM products\" -o products.json -f json \\\n  -T \"yyyy-MM-dd HH:mm:ss\" -Z \"America/Los_Angeles\"\n```\n\n#### Time Format Tokens\n\nThe `--time-format` flag accepts the following tokens:\n\n| Token | Description | Example |\n|-------|-------------|---------|\n| `yyyy` | 4-digit year | 2025 |\n| `yy` | 2-digit year | 24 |\n| `MM` | Month (01-12) | 03 |\n| `dd` | Day (01-31) | 15 |\n| `HH` | Hour 24h (00-23) | 14 |\n| `mm` | Minute (00-59) | 30 |\n| `ss` | Second (00-59) | 45 |\n| `SSS` | Milliseconds (3 digits) | 123 |\n| `SS` | Centiseconds (2 digits) | 12 |\n| `S` | Deciseconds (1 digit) | 6 |\n\n**Common Format Examples:**\n- ISO 8601: `yyyy-MM-ddTHH:mm:ss.SSS`\n- European: `dd/MM/yyyy HH:mm:ss`\n- US: `MM/dd/yyyy HH:mm:ss`\n- Date only: `yyyy-MM-dd`\n- Time only: `HH:mm:ss`\n\n#### Timezone Support\n\nThe `--time-zone` flag accepts standard IANA timezone names:\n\n**Common Timezones:**\n- `UTC` - Coordinated Universal Time\n- `America/New_York` - US Eastern Time\n- `America/Los_Angeles` - US Pacific Time\n- `America/Chicago` - US Central Time\n- `Europe/London` - UK Time\n- `Europe/Paris` - Central European Time\n- `Asia/Tokyo` - Japan Standard Time\n- `Australia/Sydney` - Australian Eastern Time\n\n**Default Behavior:**\n- If `--time-zone` is not specified, the local system timezone is used\n- If an invalid timezone is provided, a warning is displayed and local timezone is used\n\n**Full timezone list:** [IANA Time Zone Database](https://www.iana.org/time-zones)\n\n#### Advanced Examples\n\n```bash\n# Complex query with joins\npgxport -s \"\nSELECT \n  u.id, \n  u.username, \n  COUNT(o.id) as order_count,\n  SUM(o.total) as total_revenue\nFROM users u \nLEFT JOIN orders o ON u.id = o.user_id \nGROUP BY u.id, u.username \nHAVING COUNT(o.id) \u003e 0\nORDER BY total_revenue DESC\n\" -o user_stats.csv -d ','\n\n# Export with timestamp in filename\npgxport -s \"SELECT * FROM logs WHERE created_at \u003e NOW() - INTERVAL '24 hours'\" \\\n         -o \"logs_$(date +%Y%m%d).csv\"\n\n# Using long-form flags\npgxport --sql \"SELECT * FROM stations ORDER BY name\" \\\n         --output stations.csv \\\n         --format csv \\\n         --delimiter ';'\n```\n\n#### Batch Processing Examples\n\n```bash\n# Process multiple queries with a script\nfor table in users orders products; do\n  pgxport -s \"SELECT * FROM $table\" -o \"${table}_export.csv\"\ndone\n\n# Export with error handling\nif pgxport -F complex_query.sql -o output.csv; then\n  echo \"Export successful!\"\nelse\n  echo \"Export failed!\"\n  exit 1\nfi\n\n# Connect to different environments\npgxport --dsn \"$DEV_DATABASE_URL\" -s \"SELECT * FROM users\" -o dev_users.csv\npgxport --dsn \"$PROD_DATABASE_URL\" -s \"SELECT * FROM users\" -o prod_users.csv\n\n# Export same data in different formats\npgxport -s \"SELECT * FROM products\" -o products.csv -f csv\npgxport -s \"SELECT * FROM products\" -o products.json -f json\npgxport -s \"SELECT * FROM products\" -o products.yaml -f yaml\npgxport -s \"SELECT * FROM products\" -o products.xml -f xml\npgxport -s \"SELECT * FROM products\" -o products.sql -f sql -t products_backup\n\n# Automated validation script\n#!/bin/bash\nset -e\n\necho \"Exporting daily metrics...\"\nif ! pgxport -s \"SELECT * FROM daily_metrics WHERE date = CURRENT_DATE\" \\\n             -o metrics.csv --fail-on-empty; then\n    echo \"ERROR: No metrics found for today!\"\n    # Send notification\n    exit 1\nfi\n\necho \"✅ Export completed successfully\"\n```\n\n## 🔍 Verbose Mode\n\nEnable detailed logging for troubleshooting with the `--verbose` (or `-v`) flag:\n\n```bash\n# Normal output\npgxport -s \"SELECT * FROM users\" -o users.csv\n\n# Detailed output with timestamps and debug information\npgxport -s \"SELECT * FROM users\" -o users.csv --verbose\n```\n\n**Verbose mode shows:**\n- Configuration details (host, port, database)\n- Connection steps and timing\n- Query execution time\n- Export progress (every 10,000 rows)\n- Performance metrics\n\n**Additional diagnostics (CSV format only):**\n- Tracks average row fetch time and overall throughput (rows/s)\n- Detects slow PostgreSQL streaming when queries stream data gradually\n- Displays a performance summary at the end of the export\n\n**Use cases:**\n- 🔍 Debugging connection or query issues\n- 📊 Analyzing export performance\n- 🐛 Troubleshooting errors\n\n**Example output:**\n```bash\n$ pgxport -s \"SELECT * FROM users LIMIT 5\" -o users.csv -v\n\n[2025-01-15 14:23:45.258] 🔍 Configuration loaded: host=localhost port=5432 database=mydb\n[2025-01-15 14:23:45.258] ℹ Connecting to database...\n[2025-01-15 14:23:45.307] 🔍 Connection established, verifying connectivity (ping)...\n[2025-01-15 14:23:45.307] ✓ Database connection established\n[2025-01-15 14:23:45.308] ℹ Executing query...\n[2025-01-15 14:23:45.311] 🔍 Query: SELECT * FROM users LIMIT 5\n[2025-01-15 14:23:46.314] 🔍 Query executed successfully in 145ms\n[2025-01-15 14:23:46.315] 🔍 CSV export completed successfully: 5 rows written in 120ms\n[2025-01-15 14:23:46.315] ✓ Export completed: 5 rows → users.csv\n```\n\n**Note:** Sensitive information (passwords) is automatically masked in logs.\n\n## 🔄 Progress Indicator (`--progress`)\nEnable a live spinner during export:\n\n```bash\npgxport -s \"SELECT * FROM big_table\" -o output.csv --progress\n```\n**Example output:**\n```log\n⠙ Processing rows... 1717965 rows [5s]\n✓ Completed!\n```\n\n## 📄 Format Details\n\n### CSV\n\n- **Default delimiter**: `,` (comma)\n- Headers included automatically\n- **Default timestamp format**: `yyyy-MM-dd HH:mm:ss` (customizable with `--time-format`)\n- **Timezone**: Local system time (customizable with `--time-zone`)\n- NULL values exported as empty strings\n- Buffered I/O for optimal performance\n\n**Example output:**\n```csv\nid,name,email,created_at\n1,John Doe,john@example.com,2024-01-15 10:30:00\n2,Jane Smith,jane@example.com,2024-01-16 14:22:15\n```\n\n### ⚙️ COPY Mode (High-Performance CSV Export)\n\nThe `--with-copy` flag enables PostgreSQL's native COPY TO STDOUT mechanism for CSV exports.\nThis mode streams data directly from the database server, reducing CPU and memory usage.\n\n**Benefits:**\n- 🚀 Up to 10× faster than row-by-row export for large datasets\n- 💾 Low memory footprint\n- 🗜️ Compatible with compression (gzip, zip, zstd, lz4)\n- 📄 Identical CSV output format\n\n**Limitations:**\n- ⚠️ **Ignores `--time-format` and `--time-zone` options**\n- ⚠️ Uses PostgreSQL's default date/time formatting\n- Only works with CSV format\n\n**When to use:**\n- Large datasets (\u003e100k rows)\n- Performance is critical\n- Default date format is acceptable\n\n**When NOT to use:**\n- Need custom date/time formatting\n- Need specific timezone conversion\n- Working with small datasets (\u003c10k rows)\n\nExample usage:\n```bash\npgxport -s \"SELECT * FROM analytics_data\" -o analytics.csv -f csv --with-copy\n```\n\n**Note:** When using `--with-copy`, PostgreSQL handles type serialization. Date and timestamp formats may differ from standard CSV export.\n\n### XLSX\n\n- **Excel spreadsheet format** with native Excel compatibility\n- **Headers in bold** for better readability (can be skipped with `--no-header`)\n- **Streaming export** for optimal memory usage with large datasets\n- **Native Excel date/time handling** - uses Excel's internal date format\n- NULL values exported as empty cells\n\n**XLSX Format Features:**\n- ✅ **Native Excel format**: Directly openable in Microsoft Excel, LibreOffice, Google Sheets\n- ✅ **Professional styling**: Column headers automatically formatted in bold\n- ✅ **Streaming architecture**: Handles large datasets efficiently without memory issues\n- ✅ **All PostgreSQL data types supported**: integers, floats, strings, booleans, timestamps, NULL\n- ✅ **Native date handling**: Dates and timestamps use Excel's native date format for proper Excel compatibility\n- ✅ Automatic multi-sheet support: exports exceeding Excel’s 1,048,576-row limit are seamlessly split across Sheet2, Sheet3, etc.\n\n**Note:** XLSX format uses Excel's native date/time handling. The `--time-format` and `--time-zone` options are not applied to maintain proper Excel compatibility.\n\n**Use cases:**\n- 📊 Business reports and dashboards\n- 🔄 Data sharing with non-technical users\n- 📈 Financial data exports\n- 🎯 Presentations and visual analysis\n\n### TEMPLATE\n\n- **Custom output format** using Go templates\n- **Two modes available**: Full mode (loads all data) or Streaming mode (row-by-row) for large dataset\n- **Rich template functions** for data transformation\n- Default timestamp format: `yyyy-MM-dd HH:mm:ss` (customizable with `--time-format`)\n- Timezone: Local system time (customizable with `--time-zone`)\n\n#### Template Modes\n\n**Full Mode** (`--tpl-file`):\n- Loads all rows into memory\n- Access to complete dataset via `.Rows` array\n- Best for: small to medium datasets, reports requiring totals/aggregations\n- Template has access to: `.Rows`, `.Columns`, `.Count`, `.GeneratedAt`\n\n**Streaming Mode** (`--tpl-row` required, `--tpl-header` and `--tpl-footer` optional):\n- Processes rows one by one\n- Low memory footprint\n- Best for: large datasets, continuous processing\n- Each template processes rows independently\n\n#### Template Data Access\n\nTemplates use ordered maps to preserve SQL column order. Use the `get` helper to access values:\n\n```go\n{{get . \"column_name\"}}\n```\n#### Available Template Functions\n\n| Function | Description | Example |\n|----------|-------------|---------|\n| `get` | Access column value from ordered map | `{{get . \"id\"}}` |\n| `upper` | Convert to uppercase | `{{upper (get . \"name\")}}` |\n| `lower` | Convert to lowercase | `{{lower (get . \"email\")}}` |\n| `title` | Convert to title case | `{{title (get . \"name\")}}` |\n| `trim` | Remove whitespace | `{{trim (get . \"text\")}}` |\n| `replace` | Replace substring | `{{replace (get . \"text\") \"old\" \"new\"}}` |\n| `join` | Join string array | `{{join .Columns \", \"}}` |\n| `split` | Split string | `{{split (get . \"tags\") \",\"}}` |\n| `contains` | Check substring | `{{if contains (get . \"email\") \"@\"}}...{{end}}` |\n| `hasPrefix` | Check prefix | `{{hasPrefix (get . \"code\") \"US\"}}` |\n| `hasSuffix` | Check suffix | `{{hasSuffix (get . \"file\") \".txt\"}}` |\n| `printf` | Format string | `{{printf \"ID: %d\" (get . \"id\")}}` |\n| `json` | Convert to JSON | `{{json .}}` |\n| `jsonPretty` | Pretty JSON | `{{jsonPretty .}}` |\n| `formatTime` | Format time | `{{formatTime (get . \"created\") \"yyyy-MM-dd\"}}` |\n| `now` | Current time | `{{now}}` |\n| `eq` | Equal comparison | `{{if eq (get . \"status\") \"active\"}}...{{end}}` |\n| `ne` | Not equal | `{{if ne (get . \"count\") 0}}...{{end}}` |\n| `add` | Addition | `{{add 1 2}}` |\n| `sub` | Subtraction | `{{sub 10 3}}` |\n| `mul` | Multiplication | `{{mul 5 4}}` |\n| `div` | Division | `{{div 20 4}}` |\n\n#### Examples\nTemplate file (`report.html`):\n```html\nUser Report\n\n  Total Users: {{.Count}}\n  Generated: {{.GeneratedAt}}\n  \n    {{range .Columns}}{{.}}{{end}}\n    {{range .Rows}}\n    \n      {{get . \"id\"}}\n      {{get . \"name\"}}\n      {{get . \"email\"}}\n    \n    {{end}}\n```\n\nCommand:\n```bash\npgxport -s \"SELECT id, name, email FROM users\" \\\n        -o report.html \\\n        -f template \\\n        --tpl-file report.html\n```\n\n**Full Mode - Generate Markdown:**\n\nTemplate file (`users.md`):\n```markdown\n# User Export Report\n\nGenerated: {{.GeneratedAt}}\nTotal Records: {{.Count}}\n\n## Users\n\n{{range .Rows}}\n### {{get . \"name\"}}\n- **ID**: {{get . \"id\"}}\n- **Email**: {{get . \"email\"}}\n- **Status**: {{upper (get . \"status\")}}\n{{end}}\n```\n\nCommand:\n```bash\npgxport -s \"SELECT * FROM users WHERE active = true\" \\\n        -o users.md \\\n        -f template \\\n        --tpl-file users.md\n```\n\n**Streaming Mode - Generate CSV-like format:**\n\nHeader template (`header.tpl`):\n\n```go\n{{range i, $col := .Columns}}{{if $i}},{{end}}{{$col}}{{end}}\n```\nRow template (`row.tpl`):\n```go\n{{get . \"id\"}},{{get . \"name\"}},{{get . \"email\"}}\n```\nCommand:\n```bash\npgxport -s \"SELECT id, name, email FROM users\" \\\n        -o output.txt \\\n        -f template \\\n        --tpl-header header.tpl \\\n        --tpl-row row.tpl\n```\nTemplate sales report sample:\n```go\nGenerated: {{.GeneratedAt}}\n{{range .Rows}}\nOrder #{{get . \"order_id\"}} - {{upper (get . \"customer_name\")}}\nAmount: ${{printf \"%.2f\" (get . \"amount\")}}\nStatus: {{if eq (get . \"status\") \"paid\"}}✓ PAID{{else}}⚠ PENDING{{end}}\n{{end}}\nTotal Orders: {{.Count}}\n```\n\n**Use cases:**\n- 📄 Custom report generation (HTML, PDF-ready, Markdown)\n- 📊 Business intelligence exports\n- 🔄 Data transformation for external systems\n- 📋 Configuration file generation\n- 📧 Email template generation\n- 🎯 API response formatting\n- 📝 Documentation generation from database\n\n### JSON\n\n- Pretty-printed with 2-space indentation\n- Array of objects format\n- **Default timestamp format**: `yyyy-MM-dd HH:mm:ss` (customizable with `--time-format`)\n- **Timezone**: Local system time (customizable with `--time-zone`)\n- NULL values preserved as `null`\n- Optimized encoding with buffered I/O\n\n**Example output:**\n```json\n[\n  {\n    \"id\": 1,\n    \"name\": \"John Doe\",\n    \"email\": \"john@example.com\",\n    \"created_at\": \"2024-01-15 10:30:00\"\n  },\n  {\n    \"id\": 2,\n    \"name\": \"Jane Smith\",\n    \"email\": \"jane@example.com\",\n    \"created_at\": \"2024-01-16 14:22:15\"\n  }\n]\n```\n### YAML\n\n- Pretty-printed with 2-space indentation\n- Array format with `-` list items\n- **Default timestamp format**: `yyyy-MM-dd HH:mm:ss` (customizable with `--time-format`)\n- **Timezone**: Local system time (customizable with `--time-zone`)\n- NULL values preserved as `null`\n\n**Example output:**\n```yaml\n- id: 1\n  name: John Doe\n  email: john@example.com\n  created_at: \"2024-01-15 10:30:00\"\n- id: 2\n  name: Jane Smith\n  email: jane@example.com\n  created_at: \"2024-01-16 14:22:15\"\n```\n**YAML Format Features:**\n- ✅ **Human-readable**: Clean, indented structure easy to read and edit\n- ✅ **Configuration-friendly**: Ideal for configuration files and data interchange\n- ✅ **Preserves column order**: Maintains the exact order of columns from the query\n- ✅ **Type preservation**: Numbers, booleans, strings, and nulls are properly typed\n- ✅ **All PostgreSQL data types supported**: integers, floats, strings, booleans, timestamps, NULL\n- ✅ **Automatic quoting**: Strings that need quoting are automatically wrapped\n- ✅ **Null handling**: NULL values exported as YAML `null`\n\n**Use cases:**\n- 📋 Configuration files\n- 📊 Data interchange between systems\n- 🔍 Human-readable data exports\n- 🧪 Test fixtures and mock data\n\n### XML\n\n- Pretty-printed with 2-space indentation\n- **Customizable tags** using:\n  - `--xml-root-tag` (default: `results`)\n  - `--xml-row-tag` (default: `row`)\n- Each column becomes a direct XML element (e.g., `\u003cid\u003e`, `\u003cname\u003e`, `\u003cemail\u003e`)\n- **Default timestamp format**: `yyyy-MM-dd HH:mm:ss` (customizable with `--time-format`)\n- **Timezone**: Local system time (customizable with `--time-zone`)\n- NULL values exported as empty strings\n- Buffered I/O for optimal performance\n\n**Example output:**\n```xml\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cresults\u003e\n  \u003crow\u003e\n    \u003cid\u003e1\u003c/id\u003e\n    \u003cname\u003eJohn Doe\u003c/name\u003e\n    \u003cemail\u003ejohn@example.com\u003c/email\u003e\n    \u003ccreated_at\u003e2024-01-15 10:30:00\u003c/created_at\u003e\n  \u003c/row\u003e\n  \u003crow\u003e\n    \u003cid\u003e2\u003c/id\u003e\n    \u003cname\u003eJane Smith\u003c/name\u003e\n    \u003cemail\u003ejane@example.com\u003c/email\u003e\n    \u003ccreated_at\u003e2024-01-16 14:22:15\u003c/created_at\u003e\n  \u003c/row\u003e\n\u003c/results\u003e\n```\n\n### SQL\n\n- INSERT statements format for easy data migration\n- Buffered I/O for optimal performance\n- **Requires `--table` / `-t` parameter to specify target table name**\n- **Batch INSERT support** with `--insert-batch` flag for improved import performance\n\n**Example output:**\n```sql\nINSERT INTO \"users\" (\"id\", \"name\", \"email\", \"created_at\") VALUES (1, 'John Doe', 'john@example.com', '2024-01-15 10:30:00');\nINSERT INTO \"users\" (\"id\", \"name\", \"email\", \"created_at\") VALUES (2, 'Jane Smith', 'jane@example.com', '2024-01-16 14:22:15');\nINSERT INTO \"users\" (\"id\", \"name\", \"email\", \"created_at\") VALUES (3, 'Bob O''Brien', NULL, '2024-01-17 09:15:30');\n\n-- Batch insert example (with --insert-batch flag)\nINSERT INTO \"users\" (\"id\", \"name\", \"email\", \"created_at\") VALUES\n\t(1, 'John Doe', 'john@example.com', '2024-01-15 10:30:00'),\n\t(2, 'Jane Smith', 'jane@example.com', '2024-01-16 14:22:15'),\n\t(3, 'Bob O''Brien', NULL, '2024-01-17 09:15:30');\n```\n\n**SQL Format Features:**\n- ✅ **Schema-qualified table names**: Supports `schema.table` notation for cross-schema exports\n- ✅ **Batch INSERT support**: Use `--insert-batch` to group multiple rows in a single INSERT statement for significantly faster imports\n- ✅ **All PostgreSQL data types supported**: integers, floats, strings, booleans, timestamps, NULL, bytea\n- ✅ **Automatic escaping**: Single quotes in strings are properly escaped (e.g., `O'Brien` → `'O''Brien'`)\n- ✅ **Identifier quoting**: Properly quotes table and column names to handle special characters\n- ✅ **Type-aware formatting**: Numbers and booleans without quotes, strings and dates with quotes\n- ✅ **NULL handling**: NULL values exported as SQL `NULL` keyword\n- ✅ **Ready to import**: Generated SQL can be directly executed on any PostgreSQL database\n\n\n## 🛠️ Development\n\nThis section is for developers who want to contribute to pgxport.\n\n### Setting up your development environment\n\n**1. Clone the repository**\n\n```bash\ngit clone https://github.com/fbz-tec/pgxport.git\ncd pgxport\n```\n\n**2. Install dependencies**\n\nThe project uses the following main dependencies:\n\n- [pgx/v5](https://github.com/jackc/pgx) - PostgreSQL driver and toolkit\n- [cobra](https://github.com/spf13/cobra) - Modern CLI framework\n- [godotenv](https://github.com/joho/godotenv) - Load environment variables from `.env` file\n\n```bash\ngo mod download\n```\n\nThe project structure follows clean architecture principles:\n- `cmd/` - CLI commands and flags\n- `core/` - Business logic (exporter, database, config, validation)\n- `internal/` - Private utilities (logger, version)\n\n**3. Configure your database**\n\nCreate a `.env` file:\n\n```bash\ncat \u003e .env \u003c\u003c EOF\nDB_USER=postgres\nDB_PASS=your_local_password\nDB_HOST=localhost\nDB_PORT=5432\nDB_NAME=testdb\nEOF\n```\n\n**4. Verify your setup**\n\n```bash\ngo build -o pgxport\n./pgxport -s \"SELECT version()\" -o version.csv\n```\n\n### Building\n\n```bash\n# Build for current platform\ngo build -o pgxport\n\n# Build with version information\nVERSION=$(git describe --tags --always --dirty 2\u003e/dev/null || echo \"dev\")\nBUILD_TIME=$(date -u +'%Y-%m-%dT%H:%M:%SZ')\nGIT_COMMIT=$(git rev-parse --short HEAD 2\u003e/dev/null || echo \"unknown\")\n\ngo build -ldflags=\"-X github.com/fbz-tec/pgxport/internal/version.AppVersion=${VERSION} \\\n                   -X github.com/fbz-tec/pgxport/internal/version.BuildTime=${BUILD_TIME} \\\n                   -X github.com/fbz-tec/pgxport/internal/version.GitCommit=${GIT_COMMIT}\" \\\n         -o pgxport\n\n# Using Taskfile (recommended)\ntask build\n\n# Cross-platform builds\nGOOS=linux GOARCH=amd64 go build -o pgxport-linux\nGOOS=darwin GOARCH=amd64 go build -o pgxport-macos\nGOOS=windows GOARCH=amd64 go build -o pgxport.exe\n```\n\n### Testing\n\n```bash\n# Run all tests\ngo test ./...\n\n# Run tests with coverage\ngo test -cover ./...\n\n# Generate coverage report\ngo test -coverprofile=coverage.out ./...\ngo tool cover -html=coverage.out\n\n# Run tests with verbose output\ngo test -v ./...\n\n# Run specific test\ngo test -run TestValidateExportParams ./...\n\n# Run tests with race detection\ngo test -race ./...\n```\n\n### Code Quality\n\n```bash\n# Format code\ngo fmt ./...\n\n# Run linter (if golangci-lint is installed)\ngolangci-lint run\n\n# Vet code\ngo vet ./...\n```\n\n## 🔒 Security\n\n1. **Never commit credentials**:\n   - `.env` is already in `.gitignore`\n   - Use `.env.example` for documentation\n   - For production, use environment variables or secrets management\n\n2. **Avoid passwords in command line**:\n   - ❌ Bad: Password visible in process list\n   pgxport --password mysecret --user user --host db.com --database mydb ...\n   - ❌ Bad: `pgxport --dsn \"postgres://user:password123@host/db\" ...` (visible in history)\n   - ✅ Good: Use environment variable\n   export PGPASSWORD=mysecret\n   - ✅ Good: Use `.env` file or environment variables\n   - ✅ Good: Store DSN in environment: `export DATABASE_URL=\"...\"` then use `pgxport --dsn \"$DATABASE_URL\" ...`\n\n3. **Use parameterized queries**: When using dynamic SQL, be aware of SQL injection risks\n\n4. **Limit database permissions**: Use a database user with minimal required privileges (SELECT only for exports)\n\n5. **Secure your output files**: Be careful with sensitive data in exported files\n\n6. **Review queries**: Always review SQL files before execution\n\n7. **Verbose mode security**: Remember that `--verbose` logs queries and configuration. Avoid logging sensitive data.\n\n## 🚨 Error Handling\n\nThe tool provides clear error messages for common issues:\n\n- **Connection errors**: Check database credentials and network connectivity\n- **SQL errors**: Verify your query syntax\n- **File errors**: Ensure write permissions for output directory\n- **Configuration errors**: Validate all required environment variables\n- **Format errors**: Ensure format is one of: csv, json, xml, sql\n- **SQL format errors**: Ensure `--table` flag is provided when using SQL format\n- **Empty result errors**: Use `--fail-on-empty` to treat 0 rows as an error\n\n**Example error output:**\n```\nError: Invalid format 'txt'. Valid formats are: csv, json, xml, sql\nError: --table (-t) is required when using SQL format\nError: Configuration error: DB_PORT must be a valid port number (1-65535)\nError: export failed: query returned 0 rows\n```\n\n## 🤝 Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add some amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n### Code Style\n\n- Follow Go conventions and use `gofmt`\n- Add comments for exported functions\n- Keep functions small and focused (single responsibility principle)\n- Follow the layered architecture:\n  - `cmd/` - CLI logic only\n  - `core/` - Business logic\n  - `internal/` - Reusable utilities\n- New export formats should implement the `Exporter` interface and register via `registry.go`\n- Write tests for new features (`*_test.go` files alongside source)\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## 🗺️ Roadmap\n\n### ✅ Completed\n\n#### Core Features\n- `.env` configuration  \n- `--dsn` flag  \n- Individual connection flags  \n- Quiet mode  \n- Fail-on-empty mode  \n- COPY mode  \n\n#### Exporters\n- CSV (streaming)  \n- JSON (streaming)  \n- XML (streaming)  \n- SQL (INSERT statements)  \n- YAML exporter  \n- XLSX exporter (auto multi-sheet) \n- Template exporter  \n\n#### Performance\n- Streaming + compression  \n- Batch SQL inserts  \n- ZSTD compression support (fast)  \n- LZ4 compression support (fast)\n\n#### User Experience\n- Progress indicator (`--progress`) using a lightweight spinner\n\n### 🚧 Planned\n- [ ] Interactive password prompt  \n- [ ] Pagination for large queries \n- [ ] Data preview before export \n\n## 💬 Support\n\nIf you encounter any issues or have questions:\n\n- 🐛 [Open an issue](https://github.com/fbz-tec/pgxport/issues) on GitHub\n- 💡 [Start a discussion](https://github.com/fbz-tec/pgxport/discussions) for feature requests\n\n## 🙏 Acknowledgments\n\n- Built with [Cobra](https://github.com/spf13/cobra) for CLI framework\n- PostgreSQL driver: [pgx](https://github.com/jackc/pgx)\n- Environment variables: [godotenv](https://github.com/joho/godotenv)\n- XLSX library: [Excelize](https://github.com/xuri/excelize)\n- Inspired by the need for simple, reliable data exports\n\n## ⭐ Show Your Support\n\nIf you find **pgxport** useful:  \n⭐ Star the repo \u0026 share it with your team!\n\n---\n\n**Made with ❤️ for the PostgreSQL community**","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffbz-tec%2Fpgxport","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffbz-tec%2Fpgxport","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffbz-tec%2Fpgxport/lists"}