{"id":51013284,"url":"https://github.com/heptau/pg_atropos","last_synced_at":"2026-06-21T06:30:59.336Z","repository":{"id":362310879,"uuid":"1257655343","full_name":"heptau/pg_atropos","owner":"heptau","description":"✂️ Go command-line tool that splits PostgreSQL pg_dump -Fc (custom-format) dumps into individual files organized for Git versioning. Instead of wrestling with a single giant SQL file, each database object gets its own file — tables in TABLE/, functions in FUNCTION/, roles in ROLE/, and so on.","archived":false,"fork":false,"pushed_at":"2026-06-03T14:12:21.000Z","size":57,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-03T16:06:29.912Z","etag":null,"topics":["git","plpgsql","postresql","sql"],"latest_commit_sha":null,"homepage":"http://pg_atropos.80.cz","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/heptau.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null},"funding":{"custom":["https://revolut.me/zbynekvanzura","https://wise.com/pay/me/zbynekv17","https://app.binance.com/uni-qr/DbrFdVHA"]}},"created_at":"2026-06-02T22:14:37.000Z","updated_at":"2026-06-03T14:14:38.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/heptau/pg_atropos","commit_stats":null,"previous_names":["heptau/pg_atropos"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/heptau/pg_atropos","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heptau%2Fpg_atropos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heptau%2Fpg_atropos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heptau%2Fpg_atropos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heptau%2Fpg_atropos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/heptau","download_url":"https://codeload.github.com/heptau/pg_atropos/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heptau%2Fpg_atropos/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34597337,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-21T02:00:05.568Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["git","plpgsql","postresql","sql"],"created_at":"2026-06-21T06:30:53.920Z","updated_at":"2026-06-21T06:30:59.335Z","avatar_url":"https://github.com/heptau.png","language":"Go","funding_links":["https://revolut.me/zbynekvanzura","https://wise.com/pay/me/zbynekv17","https://app.binance.com/uni-qr/DbrFdVHA"],"categories":[],"sub_categories":[],"readme":"# pg_atropos\n\n[![Go](https://img.shields.io/badge/Go-1.23-blue)](https://go.dev/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![CLI Tool](https://img.shields.io/badge/Interface-CLI%20Wrapper-green.svg)](pg_atropos)\n\nSplit PostgreSQL `pg_dump -Fc` (custom-format) dumps into individual files\norganized for GIT versioning.\n\n## Why?\n\nWhen a team shares a single giant SQL dump in Git, every change — adding a\ncolumn, a new index, tweaking a function — creates a merge conflict across\nthe entire file. Resolving those conflicts is tedious and error-prone.\n\n`pg_atropos` splits the dump into individual objects (tables, functions,\nindexes, roles, …), each in its own file. In Git this means:\n\n- **Minimal conflicts** — two people can change different tables without\n  stepping on each other\n- **Clear code review** — a diff shows exactly the changed object, not\n  500 lines of dump header\n- **Readable history** — `git log -- TABLE/users.sql` shows every change\n  to that specific table\n- **CI/CD friendly** — deploy only the changed object, not the entire dump\n\n## How?\n\nInstead of parsing raw SQL text (brittle regex), `pg_atropos` pipes through\n`pg_restore -f -` to reliably extract each object via its header metadata.\n\n**Requirement:** `pg_restore` (from [PostgreSQL client tools](https://www.postgresql.org/download/))\nmust be installed. It is used to decompress and interpret the custom-format dump\n— `pg_atropos` never reads the binary format directly.\n\n## The Name\n\nIn Greek mythology, the three Moirai (Fates) spun the thread of life,\nmeasured it, and — finally — **Atropos** (the Inevitable) cut it with\nher shears.  `pg_atropos` does the same: it takes a giant `pg_dump`\nfile and snips it into small, manageable threads (files) ready for Git.\nWho wouldn't want shears that cut a dump into pieces?\n\n## Inspiration\n\nThis project was inspired by\n[michal-bartak/pgdump_splitter](https://github.com/michal-bartak/pgdump_splitter),\nwhich parses plain `pg_dump` text output.  That approach is fragile — object\nboundaries are hard to detect reliably when function bodies or comments contain\ntext that looks like dump markers.  `pg_atropos` solves this by using the\n**custom-format** (`-Fc`) dump + `pg_restore -f -` pipe, which emits clean\n`-- Name: ...; Type: ...` headers that the parser can trust.\n\nThe result: simpler, faster, and far more robust parsing.\n\n## Quick Start\n\n```bash\n# From a custom-format dump file\npg_atropos -f dump.pgdump -output ./structure\n\n# From a live database (auto-dump via pg_dump)\npg_atropos -d mydb -output ./structure\n\n# Pipe from pg_dump (no temp file needed)\npg_dump -Fc mydb | pg_atropos -f - -o ./structure\n\n# Pipe from a remote database via connection string\npg_dump -Fc postgresql://user@server:port/database | pg_atropos -f - -o ./structure\n\n# Pipe from a remote database via ssh\nssh dbserver 'pg_dump -Fc mydb' | pg_atropos -f - -o ./structure\n\n# Custom mode (lowercase directories for CI)\npg_atropos -f dump.pgdump -output ./structure -mode custom\n```\n\n## Installation\n\n### From source\n\n```bash\ngit clone https://github.com/your-project/pg_atropos.git\ncd pg_atropos \u0026\u0026 make build\n```\n\n### Docker\n\n```bash\ndocker build -t pg_atropos .\ndocker run --rm -v $(pwd)/dump.pgdump:/dump.pgdump pg_atropos -f /dump.pgdump\n```\n\n## Flags\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--db` | `\"\"` | Database name to dump |\n| `--conn` | `\"\"` | PostgreSQL connection string |\n| `--file`, `-f` | `\"\"` | Custom-format dump file (`\"-\"` for stdin) |\n| `--output`, `-o` | `./output` | Output directory |\n| `--mode`, `-m` | `origin` | Output mode: `origin` \\| `custom` |\n| `--clean` | `false` | Clean output directory before processing |\n| `--no-db-path` | `false` | Don't include database name in output path |\n| `--blacklist-db` | `^(template\\|postgres)` | Skip databases matching pattern |\n| `--whitelist-db` | `\"\"` | Only include databases matching pattern |\n| `--exclude-obj` | `\"\"` | Exclude object types matching pattern |\n| `--acl-files` | `false` | Save ACLs to separate `.acl.sql` files |\n| `--move-roles` | `false` | Move role files under database directory |\n| `--dry-run` | `false` | Print what would be extracted without writing |\n| `--quiet` | `false` | Suppress informational output |\n| `--version` | — | Print version and exit |\n\n## Modes\n\n### origin (default)\n\nMirrors the dump structure exactly — each object type gets its own\ndirectory (`TABLE/`, `FUNCTION/`, `INDEX/`, …).  Good for inspection.\n\n### custom\n\nLowercase directories (`table/`, `function/`, …).  Indexes, constraints,\ntriggers are **not** merged into their parent table (pg_restore headers\ndon't carry the parent table name, so merge would require SQL content\nparsing).  Better suited for CI / automation workflows.\n\n## Tests\n\n```bash\nmake test       # unit tests (no database required)\nmake coverage   # with coverage report\n```\n\nTests use the `--test-sql` flag to inject pre-extracted SQL directly,\nbypassing `pg_restore`.  No PostgreSQL installation needed.\n\n## Limitations\n\n- INDEX / CONSTRAINT / TRIGGER merging into table files is not supported\n  in custom mode (pg_restore headers lack parent table name).\n- Object names with spaces or special characters may not round-trip.\n\n## Performance\n\n~150 objects / 2226 SQL lines:\n\n| Version | Avg Time | vs pg_atropos |\n|---------|----------|---------------|\n| **pg_atropos (Go)** | **0.044s** | **1×** |\n| pgdump_splitter (Go) | 0.109s | 2.5× slower |\n\n## Future Improvements\n\n- **Merge INDEX/CONSTRAINT/TRIGGER under table/** — would require\n  parsing the SQL content to identify the parent table (doable, adds\n  complexity).\n- **Memory limits** — configurable scanner buffer for constrained\n  Docker environments.\n- **Git integration** — commit each object separately with structured\n  commit messages.\n\n## License\n\nMIT — see [LICENSE](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheptau%2Fpg_atropos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fheptau%2Fpg_atropos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheptau%2Fpg_atropos/lists"}