{"id":45348226,"url":"https://github.com/HelgeSverre/sql-splitter","last_synced_at":"2026-03-06T12:01:27.940Z","repository":{"id":329773095,"uuid":"1067478404","full_name":"HelgeSverre/sql-splitter","owner":"HelgeSverre","description":"Split large SQL dump files into individual table files. Fast, memory-efficient, multi-dialect.","archived":false,"fork":false,"pushed_at":"2026-02-18T20:41:38.000Z","size":2154,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-18T22:45:40.662Z","etag":null,"topics":["cli","database","devtools","mysql","mysqldump","postgresql","rust","sql","sql-dump","sqlite"],"latest_commit_sha":null,"homepage":"https://sql-splitter.dev/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HelgeSverre.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"docs/ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-09-30T23:15:55.000Z","updated_at":"2026-02-18T20:41:42.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/HelgeSverre/sql-splitter","commit_stats":null,"previous_names":["helgesverre/sql-splitter"],"tags_count":26,"template":false,"template_full_name":null,"purl":"pkg:github/HelgeSverre/sql-splitter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelgeSverre%2Fsql-splitter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelgeSverre%2Fsql-splitter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelgeSverre%2Fsql-splitter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelgeSverre%2Fsql-splitter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HelgeSverre","download_url":"https://codeload.github.com/HelgeSverre/sql-splitter/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelgeSverre%2Fsql-splitter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30175866,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T11:48:51.886Z","status":"ssl_error","status_checked_at":"2026-03-06T11:48:51.460Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","database","devtools","mysql","mysqldump","postgresql","rust","sql","sql-dump","sqlite"],"created_at":"2026-02-21T12:00:27.532Z","updated_at":"2026-03-06T12:01:27.906Z","avatar_url":"https://github.com/HelgeSverre.png","language":"Rust","readme":"# sql-splitter\n\n[![Crates.io](https://img.shields.io/crates/v/sql-splitter.svg)](https://crates.io/crates/sql-splitter)\n[![Downloads](https://img.shields.io/crates/d/sql-splitter.svg)](https://crates.io/crates/sql-splitter)\n[![Build](https://github.com/helgesverre/sql-splitter/actions/workflows/test.yml/badge.svg)](https://github.com/helgesverre/sql-splitter/actions)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE.md)\n[![Amp](https://img.shields.io/badge/Amp%20Code-191C19.svg?logo=data:image/svg%2bxml;base64,PHN2ZyB3aWR0aD0iMjEiIGhlaWdodD0iMjEiIHZpZXdCb3g9IjAgMCAyMSAyMSIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZD0iTTMuNzY4NzkgMTguMzAxNUw4LjQ5ODM5IDEzLjUwNUwxMC4yMTk2IDIwLjAzOTlMMTIuNzIgMTkuMzU2MUwxMC4yMjg4IDkuODY3NDlMMC44OTA4NzYgNy4zMzg0NEwwLjIyNTk0IDkuODkzMzFMNi42NTEzNCAxMS42Mzg4TDEuOTQxMzggMTYuNDI4MkwzLjc2ODc5IDE4LjMwMTVaIiBmaWxsPSIjRjM0RTNGIi8+CjxwYXRoIGQ9Ik0xNy40MDc0IDEyLjc0MTRMMTkuOTA3OCAxMi4wNTc1TDE3LjQxNjcgMi41Njg5N0w4LjA3ODczIDAuMDM5OTI0Nkw3LjQxMzggMi41OTQ4TDE1LjI5OTIgNC43MzY4NUwxNy40MDc0IDEyLjc0MTRaIiBmaWxsPSIjRjM0RTNGIi8+CjxwYXRoIGQ9Ik0xMy44MTg0IDE2LjM4ODNMMTYuMzE4OCAxNS43MDQ0TDEzLjgyNzYgNi4yMTU4OEw0LjQ4OTcxIDMuNjg2ODNMMy44MjQ3NyA2LjI0MTcxTDExLjcxMDEgOC4zODM3NkwxMy44MTg0IDE2LjM4ODNaIiBmaWxsPSIjRjM0RTNGIi8+Cjwvc3ZnPg==\u0026style=flat)](https://ampcode.com/@helgesverre)\n\nSplit large SQL dump files into individual table files. Fast, memory-efficient, multi-dialect.\n\n- **600+ MB/s** throughput on modern hardware\n- **MySQL, PostgreSQL, SQLite, MSSQL** support (including `COPY FROM stdin`, `GO` batches)\n- **Compressed files** — gzip, bzip2, xz, zstd auto-detected\n- **Streaming architecture** — handles files larger than RAM\n- **5x faster** than shell-based alternatives\n\n## Installation\n\n### From crates.io\n\n```bash\ncargo install sql-splitter\n```\n\n### From source\n\n```bash\ngit clone https://github.com/helgesverre/sql-splitter\ncd sql-splitter\nmake install  # Installs binary + shell completions + man pages\n```\n\nOr download pre-built binaries from [GitHub Releases](https://github.com/helgesverre/sql-splitter/releases).\n\n### Man pages (optional)\n\nAfter installation, view documentation with `man sql-splitter` or `man sql-splitter-diff`.\n\nFor `cargo install` users, install man pages manually:\n\n```bash\ngit clone https://github.com/helgesverre/sql-splitter\ncd sql-splitter\nmake install-man\n```\n\n## Usage\n\n```bash\n# MySQL/MariaDB dump (default)\nsql-splitter split dump.sql -o tables/\n\n# PostgreSQL pg_dump\nsql-splitter split pg_dump.sql -o tables/ --dialect=postgres\n\n# SQLite dump\nsql-splitter split sqlite.sql -o tables/ --dialect=sqlite\n\n# MSSQL/T-SQL dump (SSMS \"Generate Scripts\", sqlcmd)\nsql-splitter split mssql_dump.sql -o tables/ --dialect=mssql\n\n# Compressed files (auto-detected)\nsql-splitter split backup.sql.gz -o tables/\nsql-splitter split backup.sql.zst -o tables/\n\n# Split specific tables only\nsql-splitter split dump.sql --tables users,posts,orders\n\n# Schema only (CREATE TABLE, indexes, etc.)\nsql-splitter split dump.sql -o schema/ --schema-only\n\n# Data only (INSERT/COPY statements)\nsql-splitter split dump.sql -o data/ --data-only\n\n# Merge split files back into single dump\nsql-splitter merge tables/ -o restored.sql\n\n# Merge specific tables only\nsql-splitter merge tables/ -o partial.sql --tables users,orders\n\n# Merge with transaction wrapper\nsql-splitter merge tables/ -o restored.sql --transaction\n\n# Analyze without splitting\nsql-splitter analyze dump.sql\n\n# Convert between SQL dialects\nsql-splitter convert mysql_dump.sql -o postgres_dump.sql --to postgres\nsql-splitter convert pg_dump.sql -o mysql_dump.sql --to mysql\nsql-splitter convert dump.sql -o sqlite_dump.sql --to sqlite\nsql-splitter convert mssql_dump.sql -o mysql_dump.sql --to mysql\n\n# Convert with explicit source dialect\nsql-splitter convert dump.sql --from postgres --to mysql -o output.sql\nsql-splitter convert dump.sql --from mssql --to postgres -o output.sql\n\n# Validate SQL dump integrity\nsql-splitter validate dump.sql\n\n# Validate with strict mode (warnings = errors)\nsql-splitter validate dump.sql --strict\n\n# Validate with JSON output for CI\nsql-splitter validate dump.sql --json\n\n# Batch operations with glob patterns\nsql-splitter validate \"dumps/*.sql\" --fail-fast\nsql-splitter analyze \"**/*.sql\"\nsql-splitter split \"*.sql\" -o output/\nsql-splitter convert \"*.sql\" --to postgres -o converted/\n\n# Compare two SQL dumps for changes\nsql-splitter diff old.sql new.sql\n\n# Diff with schema-only or data-only\nsql-splitter diff old.sql new.sql --schema-only\nsql-splitter diff old.sql new.sql --data-only\n\n# Diff with JSON or SQL migration output\nsql-splitter diff old.sql new.sql --format json -o diff.json\nsql-splitter diff old.sql new.sql --format sql -o migration.sql\n\n# Diff with verbose PK samples and ignore timestamp columns\nsql-splitter diff old.sql new.sql --verbose --ignore-columns \"*.updated_at,*.created_at\"\n\n# Override primary key for tables without PK\nsql-splitter diff old.sql new.sql --primary-key logs:timestamp+message\n\n# Redact sensitive data using inline patterns\nsql-splitter redact dump.sql -o safe.sql --null \"*.ssn\" --hash \"*.email\" --fake \"*.name\"\n\n# Redact using YAML config file\nsql-splitter redact dump.sql -o safe.sql --config redact.yaml\n\n# Generate redaction config by analyzing input file\nsql-splitter redact dump.sql --generate-config -o redact.yaml\n\n# Reproducible redaction with seed\nsql-splitter redact dump.sql -o safe.sql --null \"*.password\" --seed 42\n\n# Generate ERD (Entity-Relationship Diagram)\nsql-splitter graph dump.sql -o schema.html        # Interactive HTML (default)\nsql-splitter graph dump.sql -o schema.dot         # Graphviz DOT format\nsql-splitter graph dump.sql -o schema.mmd         # Mermaid erDiagram\nsql-splitter graph dump.sql -o schema.json        # JSON with full schema details\n\n# Graph with filtering\nsql-splitter graph dump.sql --tables \"user*,order*\" -o filtered.html\nsql-splitter graph dump.sql --exclude \"log*,audit*\" -o clean.html\nsql-splitter graph dump.sql --cycles-only         # Only tables in circular dependencies\n\n# Focus on specific table and its relationships\nsql-splitter graph dump.sql --table orders --transitive  # Show all dependencies\nsql-splitter graph dump.sql --table users --reverse      # Show all dependents\n\n# Reorder SQL dump in topological FK order\nsql-splitter order dump.sql -o ordered.sql        # Safe import order\nsql-splitter order dump.sql --check               # Check for cycles\nsql-splitter order dump.sql --reverse             # Reverse (for DROP operations)\n\n# Query SQL dumps with DuckDB analytics engine\nsql-splitter query dump.sql \"SELECT COUNT(*) FROM users\"\nsql-splitter query dump.sql \"SELECT * FROM orders WHERE total \u003e 100\" -f json\nsql-splitter query dump.sql \"SELECT * FROM users LIMIT 10\" -o results.csv -f csv\nsql-splitter query dump.sql --interactive         # Start REPL session\nsql-splitter query huge.sql \"SELECT ...\" --disk   # Use disk mode for large files\n\n# Query with caching for repeated queries\nsql-splitter query dump.sql \"SELECT ...\" --cache  # Cache imported database\nsql-splitter query --list-cache                   # Show cached databases\nsql-splitter query --clear-cache                  # Clear all cached databases\n\n# Generate shell completions (auto-installed with make install)\nsql-splitter completions bash \u003e\u003e ~/.bashrc\nsql-splitter completions zsh \u003e\u003e ~/.zshrc\nsql-splitter completions fish \u003e\u003e ~/.config/fish/completions/sql-splitter.fish\n```\n\n### Shell Completions\n\nShell completions are automatically installed when using `make install`. For manual installation:\n\n```bash\n# Install for current shell only\nmake install-completions\n\n# Install for all shells (bash, zsh, fish)\nmake install-completions-all\n```\n\n## Why sql-splitter?\n\nsql-splitter is a **dump-first, CLI-first** tool designed for automation and CI/CD pipelines.\n\n**What it's optimized for**\n\n| Strength                       | Description                                                                        |\n| ------------------------------ | ---------------------------------------------------------------------------------- |\n| **One tool for the workflow**  | Split → sample → shard → convert → merge in a single binary                        |\n| **Works on dump files**        | No running database or JDBC connection needed (unlike mydumper, Jailer, Condenser) |\n| **Streaming architecture**     | 10GB+ dumps with constant memory, 600+ MB/s throughput                             |\n| **Multi-dialect + conversion** | MySQL, PostgreSQL, SQLite including `COPY FROM stdin` → INSERT                     |\n| **FK-aware operations**        | Sampling and tenant sharding preserve referential integrity                        |\n\n**When another tool might be better**\n\n- **[mydumper](https://github.com/mydumper/mydumper)** — Parallel snapshots from live MySQL/MariaDB databases\n- **[Jailer](https://github.com/Wisser/Jailer)** — Rich GUI-based FK subsetting with JDBC across 12+ databases\n- **[sqlglot](https://github.com/tobymao/sqlglot)** — Query-level transpilation and AST manipulation (31 dialects)\n- **[DuckDB](https://github.com/duckdb/duckdb)** — Complex analytical queries over SQL/CSV/JSON/Parquet\n\nSee [docs/COMPETITIVE_ANALYSIS.md](docs/COMPETITIVE_ANALYSIS.md) for detailed comparisons.\n\n## Options\n\n### Split Options\n\n| Flag             | Description                                         | Default     |\n| ---------------- | --------------------------------------------------- | ----------- |\n| `-o, --output`   | Output directory                                    | `output`    |\n| `-d, --dialect`  | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql` | auto-detect |\n| `-t, --tables`   | Only split these tables (comma-separated)           | —           |\n| `-p, --progress` | Show progress bar                                   | —           |\n| `--dry-run`      | Preview without writing files                       | —           |\n| `--schema-only`  | Only DDL statements (CREATE, ALTER, DROP)           | —           |\n| `--data-only`    | Only DML statements (INSERT, COPY)                  | —           |\n| `--fail-fast`    | Stop on first error (for glob patterns)             | —           |\n| `--json`         | Output results as JSON                              | —           |\n\nInput can be a file path or glob pattern (e.g., `*.sql`, `dumps/**/*.sql`).\n\n### Merge Options\n\n| Flag             | Description                               | Default |\n| ---------------- | ----------------------------------------- | ------- |\n| `-o, --output`   | Output SQL file                           | stdout  |\n| `-d, --dialect`  | SQL dialect for headers/footers           | `mysql` |\n| `-t, --tables`   | Only merge these tables (comma-separated) | all     |\n| `-e, --exclude`  | Exclude these tables (comma-separated)    | —       |\n| `--transaction`  | Wrap in BEGIN/COMMIT transaction          | —       |\n| `--no-header`    | Skip header comments                      | —       |\n| `-p, --progress` | Show progress bar                         | —       |\n| `--dry-run`      | Preview without writing files             | —       |\n| `--json`         | Output results as JSON                    | —       |\n\n### Analyze Options\n\n| Flag             | Description                                         | Default     |\n| ---------------- | --------------------------------------------------- | ----------- |\n| `-d, --dialect`  | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql` | auto-detect |\n| `-p, --progress` | Show progress bar                                   | —           |\n| `--fail-fast`    | Stop on first error (for glob patterns)             | —           |\n| `--json`         | Output results as JSON                              | —           |\n\nInput can be a file path or glob pattern (e.g., `*.sql`, `dumps/**/*.sql`).\n\n### Convert Options\n\n| Flag             | Description                                            | Default     |\n| ---------------- | ------------------------------------------------------ | ----------- |\n| `-o, --output`   | Output SQL file or directory (required for glob)       | stdout      |\n| `--from`         | Source dialect: `mysql`, `postgres`, `sqlite`, `mssql` | auto-detect |\n| `--to`           | Target dialect: `mysql`, `postgres`, `sqlite`, `mssql` | required    |\n| `--strict`       | Fail on any unsupported feature                        | —           |\n| `-p, --progress` | Show progress bar                                      | —           |\n| `--dry-run`      | Preview without writing files                          | —           |\n| `--fail-fast`    | Stop on first error (for glob patterns)                | —           |\n| `--json`         | Output results as JSON                                 | —           |\n\nInput can be a file path or glob pattern (e.g., `*.sql`, `dumps/**/*.sql`).\n\n**Supported conversions (12 pairs):**\n\n- MySQL ↔ PostgreSQL (including COPY → INSERT)\n- MySQL ↔ SQLite\n- MySQL ↔ MSSQL\n- PostgreSQL ↔ SQLite\n- PostgreSQL ↔ MSSQL\n- SQLite ↔ MSSQL\n\n**Features:**\n\n- 50+ data type mappings\n- AUTO_INCREMENT ↔ SERIAL ↔ INTEGER PRIMARY KEY ↔ IDENTITY\n- PostgreSQL COPY → INSERT with NULL and escape handling\n- Session command stripping\n- Warnings for unsupported features (ENUM, arrays, triggers)\n\n### Validate Options\n\n| Flag                   | Description                                         | Default     |\n| ---------------------- | --------------------------------------------------- | ----------- |\n| `-d, --dialect`        | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql` | auto-detect |\n| `--strict`             | Treat warnings as errors (exit 1)                   | —           |\n| `--json`               | Output results as JSON                              | —           |\n| `--max-rows-per-table` | Max rows per table for PK/FK checks (0 = no limit)  | 1,000,000   |\n| `--no-limit`           | Disable row limit for PK/FK checks                  | —           |\n| `--no-fk-checks`       | Skip PK/FK data integrity checks                    | —           |\n| `-p, --progress`       | Show progress bar                                   | —           |\n| `--fail-fast`          | Stop on first error (for glob patterns)             | —           |\n\nInput can be a file path or glob pattern (e.g., `*.sql`, `dumps/**/*.sql`).\n\n**Validation checks:**\n\n- SQL syntax validation (parser errors)\n- DDL/DML consistency (INSERTs reference existing tables)\n- Encoding validation (UTF-8)\n- Duplicate primary key detection (all dialects)\n- FK referential integrity (all dialects)\n\n### Sample Options\n\n| Flag                   | Description                                         | Default     |\n| ---------------------- | --------------------------------------------------- | ----------- |\n| `-o, --output`         | Output SQL file                                     | stdout      |\n| `-d, --dialect`        | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql` | auto-detect |\n| `--percent`            | Sample percentage (1-100)                           | —           |\n| `--rows`               | Sample fixed number of rows per table               | —           |\n| `--preserve-relations` | Preserve FK relationships                           | —           |\n| `-t, --tables`         | Only sample these tables (comma-separated)          | all         |\n| `-e, --exclude`        | Exclude these tables (comma-separated)              | —           |\n| `--root-tables`        | Explicit root tables for sampling                   | —           |\n| `--include-global`     | Global table handling: `none`, `lookups`, `all`     | `lookups`   |\n| `--seed`               | Random seed for reproducibility                     | random      |\n| `-c, --config`         | YAML config file for per-table settings             | —           |\n| `--max-total-rows`     | Maximum total rows to sample (0 = no limit)         | —           |\n| `--no-limit`           | Disable row limit                                   | —           |\n| `--strict-fk`          | Fail if any FK integrity issues detected            | —           |\n| `--no-schema`          | Exclude CREATE TABLE statements from output         | —           |\n| `-p, --progress`       | Show progress bar                                   | —           |\n| `--dry-run`            | Preview without writing files                       | —           |\n| `--json`               | Output results as JSON                              | —           |\n\n### Shard Options\n\n| Flag                  | Description                                         | Default     |\n| --------------------- | --------------------------------------------------- | ----------- |\n| `-o, --output`        | Output SQL file or directory                        | stdout      |\n| `-d, --dialect`       | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql` | auto-detect |\n| `--tenant-column`     | Column name for tenant identification               | auto-detect |\n| `--tenant-value`      | Single tenant value to extract                      | —           |\n| `--tenant-values`     | Multiple tenant values (comma-separated)            | —           |\n| `--root-tables`       | Explicit root tables with tenant column             | —           |\n| `--include-global`    | Global table handling: `none`, `lookups`, `all`     | `lookups`   |\n| `-c, --config`        | YAML config file for table classification           | —           |\n| `--max-selected-rows` | Maximum rows to select (0 = no limit)               | —           |\n| `--no-limit`          | Disable row limit                                   | —           |\n| `--strict-fk`         | Fail if any FK integrity issues detected            | —           |\n| `--no-schema`         | Exclude CREATE TABLE statements from output         | —           |\n| `-p, --progress`      | Show progress bar                                   | —           |\n| `--dry-run`           | Preview without writing files                       | —           |\n| `--json`              | Output results as JSON                              | —           |\n\n### Diff Options\n\n| Flag               | Description                                                  | Default     |\n| ------------------ | ------------------------------------------------------------ | ----------- |\n| `-o, --output`     | Output file (default: stdout)                                | stdout      |\n| `-d, --dialect`    | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql`          | auto-detect |\n| `--schema-only`    | Compare schema only, skip data                               | —           |\n| `--data-only`      | Compare data only, skip schema                               | —           |\n| `--format`         | Output format: `text`, `json`, `sql`                         | `text`      |\n| `-t, --tables`     | Only compare these tables (comma-separated)                  | all         |\n| `-e, --exclude`    | Exclude these tables (comma-separated)                       | —           |\n| `--max-pk-entries` | Max PK entries to track (0 = no limit)                       | 10,000,000  |\n| `-v, --verbose`    | Show sample PK values for added/removed/modified rows        | —           |\n| `--primary-key`    | Override PK for tables (format: `table:col1+col2`)           | auto-detect |\n| `--ignore-order`   | Ignore column order differences in schema comparison         | —           |\n| `--ignore-columns` | Ignore columns matching glob patterns (e.g., `*.updated_at`) | —           |\n| `--allow-no-pk`    | Compare tables without PK using all columns as key           | —           |\n| `-p, --progress`   | Show progress bar                                            | —           |\n\n**What diff detects:**\n\n- Tables added/removed/modified (columns, types, nullability)\n- Primary key changes\n- Foreign key changes\n- Index changes (CREATE INDEX, inline INDEX/KEY)\n- Rows added/removed/modified (via PK-based comparison)\n\n**Output formats:**\n\n- `text`: Human-readable summary with optional PK samples\n- `json`: Structured data for automation (includes warnings)\n- `sql`: Migration script with ALTER/CREATE INDEX/DROP INDEX statements\n\n### Graph Options\n\n| Flag             | Description                                            | Default     |\n| ---------------- | ------------------------------------------------------ | ----------- |\n| `-o, --output`   | Output file (html, dot, mmd, json, png, svg, pdf)      | stdout      |\n| `--format`       | Output format: `html`, `dot`, `mermaid`, `json`        | auto-detect |\n| `-d, --dialect`  | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql`    | auto-detect |\n| `--layout`       | Layout direction: `lr` (left-right), `tb` (top-bottom) | `lr`        |\n| `-t, --tables`   | Only include tables matching glob patterns             | all         |\n| `-e, --exclude`  | Exclude tables matching glob patterns                  | —           |\n| `--table`        | Focus on a specific table                              | —           |\n| `--transitive`   | Show all dependencies of focused table                 | —           |\n| `--reverse`      | Show all tables that depend on focused table           | —           |\n| `--max-depth`    | Limit traversal depth                                  | unlimited   |\n| `--cycles-only`  | Only show tables in circular dependencies              | —           |\n| `--render`       | Render DOT to PNG/SVG/PDF using Graphviz               | —           |\n| `-p, --progress` | Show progress bar                                      | —           |\n| `--json`         | Output as JSON                                         | —           |\n\n**Output formats:**\n\n- `html`: Interactive diagram with dark/light theme, copy Mermaid button\n- `dot`: Graphviz DOT with ERD-style tables (columns, types, PK/FK markers)\n- `mermaid`: Mermaid erDiagram syntax\n- `json`: Full schema with tables, columns, relationships, and stats\n- `png`/`svg`/`pdf`: Rendered image (requires Graphviz `dot` command)\n\n### Order Options\n\n| Flag            | Description                                         | Default     |\n| --------------- | --------------------------------------------------- | ----------- |\n| `-o, --output`  | Output SQL file                                     | stdout      |\n| `-d, --dialect` | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql` | auto-detect |\n| `--check`       | Check for cycles and report order (don't write)     | —           |\n| `--dry-run`     | Show topological order without writing              | —           |\n| `--reverse`     | Reverse order (children before parents, for DROP)   | —           |\n\n### Redact Options\n\n| Flag                | Description                                             | Default     |\n| ------------------- | ------------------------------------------------------- | ----------- |\n| `-o, --output`      | Output SQL file                                         | stdout      |\n| `-d, --dialect`     | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql`     | auto-detect |\n| `-c, --config`      | YAML config file for redaction rules                    | —           |\n| `--generate-config` | Analyze input and generate annotated YAML config        | —           |\n| `--null`            | Columns to set to NULL (glob patterns, comma-separated) | —           |\n| `--hash`            | Columns to hash with SHA256 (glob patterns)             | —           |\n| `--fake`            | Columns to replace with fake data (glob patterns)       | —           |\n| `--mask`            | Columns to partially mask (format: `pattern=column`)    | —           |\n| `--constant`        | Column=value pairs for constant replacement             | —           |\n| `--seed`            | Random seed for reproducible redaction                  | random      |\n| `--locale`          | Locale for fake data (en, de_de, fr_fr, etc.)           | `en`        |\n| `-t, --tables`      | Only redact specific tables (comma-separated)           | all         |\n| `-e, --exclude`     | Exclude specific tables (comma-separated)               | —           |\n| `--strict`          | Fail on warnings (e.g., unsupported locale)             | —           |\n| `-p, --progress`    | Show progress bar                                       | —           |\n| `--dry-run`         | Preview without writing files                           | —           |\n| `--json`            | Output results as JSON                                  | —           |\n| `--validate`        | Validate config only, don't process                     | —           |\n\n**Redaction strategies:**\n\n- `null`: Replace value with NULL\n- `constant`: Replace with fixed value\n- `hash`: SHA256 hash (deterministic, preserves FK relationships)\n- `mask`: Partial masking with pattern (`*`=asterisk, `X`=keep, `#`=random digit)\n- `fake`: Generate realistic fake data (25+ generators)\n- `shuffle`: Redistribute values within column (preserves distribution)\n- `skip`: No redaction (passthrough)\n\n**Fake data generators:**\n\n`email`, `name`, `first_name`, `last_name`, `phone`, `address`, `city`, `state`, `zip`, `country`, `company`, `job_title`, `username`, `url`, `ip`, `ipv6`, `uuid`, `date`, `datetime`, `credit_card`, `iban`, `ssn`, `lorem`, `paragraph`, `sentence`\n\n### Query Options\n\n| Flag                | Description                                           | Default     |\n| ------------------- | ----------------------------------------------------- | ----------- |\n| `-f, --format`      | Output format: `table`, `json`, `jsonl`, `csv`, `tsv` | `table`     |\n| `-o, --output`      | Write output to file instead of stdout                | stdout      |\n| `-d, --dialect`     | SQL dialect: `mysql`, `postgres`, `sqlite`, `mssql`   | auto-detect |\n| `-i, --interactive` | Start interactive REPL session                        | —           |\n| `--disk`            | Use disk-based storage (for large dumps \u003e2GB)         | auto        |\n| `--cache`           | Cache imported database for repeated queries          | —           |\n| `-t, --tables`      | Only import specific tables (comma-separated)         | all         |\n| `--memory-limit`    | Memory limit for DuckDB (e.g., \"4GB\")                 | —           |\n| `--timing`          | Show query execution time                             | —           |\n| `-p, --progress`    | Show import progress                                  | —           |\n| `--list-cache`      | List cached databases                                 | —           |\n| `--clear-cache`     | Clear all cached databases                            | —           |\n\n**REPL commands:**\n\n- `.tables` — List all tables\n- `.schema [table]` — Show schema (all tables or specific table)\n- `.describe \u003ctable\u003e` — Describe a specific table\n- `.format \u003cfmt\u003e` — Set output format (table, json, csv, tsv)\n- `.count \u003ctable\u003e` — Count rows in a table\n- `.sample \u003ctable\u003e [n]` — Show sample rows (default: 10)\n- `.export \u003cfile\u003e \u003cquery\u003e` — Export query results to file\n- `.exit` — Exit the REPL\n\n## Performance\n\nSee [BENCHMARKS.md](BENCHMARKS.md) for detailed comparisons.\n\n## Testing\n\n```bash\n# Unit tests\ncargo test\n\n# Verify against real-world SQL dumps (MySQL, PostgreSQL, WordPress, etc.)\nmake verify-realworld\n```\n\n## AI Agent Integration\n\nsql-splitter includes documentation optimized for AI agents:\n\n- **[llms.txt](website/llms.txt)** - LLM-friendly documentation following the [llmstxt.org](https://llmstxt.org) specification\n- **[Agent Skill](skills/sql-splitter/SKILL.md)** - Claude Code / Amp skill for automatic tool discovery\n\nInstall the skill in Claude Code / Amp:\n\n```bash\namp skill add helgesverre/sql-splitter\n```\n\n## License\n\nMIT — see [LICENSE.md](LICENSE.md)\n","funding_links":[],"categories":["Toolkits"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHelgeSverre%2Fsql-splitter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHelgeSverre%2Fsql-splitter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHelgeSverre%2Fsql-splitter/lists"}