{"id":49693471,"url":"https://github.com/ralscha/mssql-copier","last_synced_at":"2026-05-07T20:01:11.393Z","repository":{"id":356351751,"uuid":"1230868145","full_name":"ralscha/mssql-copier","owner":"ralscha","description":"CLI that replicates SQL Server database","archived":false,"fork":false,"pushed_at":"2026-05-07T18:02:10.000Z","size":93,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-07T18:37:33.236Z","etag":null,"topics":["mssql"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ralscha.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-06T11:57:18.000Z","updated_at":"2026-05-07T18:00:04.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ralscha/mssql-copier","commit_stats":null,"previous_names":["ralscha/mssql-copier"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/ralscha/mssql-copier","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ralscha%2Fmssql-copier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ralscha%2Fmssql-copier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ralscha%2Fmssql-copier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ralscha%2Fmssql-copier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ralscha","download_url":"https://codeload.github.com/ralscha/mssql-copier/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ralscha%2Fmssql-copier/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32753938,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-07T02:14:30.463Z","status":"ssl_error","status_checked_at":"2026-05-07T02:14:29.405Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mssql"],"created_at":"2026-05-07T20:01:04.761Z","updated_at":"2026-05-07T20:01:11.366Z","avatar_url":"https://github.com/ralscha.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mssql-copier\n\nA fast, concurrent SQL Server copier that replicates SQL Server tables, alias user-defined types, user-defined table types, sequences, views, functions, stored procedures, DML triggers, and synonyms from a source database to a target database. It uses bulk copy where possible, with automatic fallback to row-by-row inserts for unsupported column types.\n\n## Features\n\n- **Metadata-driven** — discovers tables, alias user-defined types, user-defined table types, sequences, views, functions, stored procedures, DML triggers, synonyms, columns, primary keys, foreign keys, checks, and indexes from the source\n- **Concurrent copying** — copies multiple tables in parallel with configurable worker count\n- **Bulk copy** — uses SQL Server's `COPY IN` (TABLOCK) for compatible column types, falling back to row-by-row `INSERT` when needed\n- **Identity insert** — automatically handles `SET IDENTITY_INSERT ON/OFF`\n- **Object filters** — include/exclude schemas and object names using wildcard patterns (`*`, `%`, `?`, `_`)\n- **Sequence copy** — copies sequences so target-side defaults based on `NEXT VALUE FOR ...` keep working\n- **Alias type copy** — copies alias user-defined types and preserves them in recreated table definitions\n- **Table type copy** — copies user-defined table types so TVP-based procedures can be recreated on the target\n- **Trigger copy** — copies table-scoped DML triggers with rerun-safe `CREATE OR ALTER TRIGGER`\n- **View copy** — copies views with dependency-aware creation order and rerun-safe `CREATE OR ALTER VIEW`\n- **Function copy** — copies SQL functions with dependency-aware creation order and rerun-safe `CREATE OR ALTER FUNCTION`\n- **Stored procedure copy** — copies stored procedures with rerun-safe `CREATE OR ALTER PROCEDURE`\n- **Synonym copy** — copies synonyms with rerun-safe drop-and-create behavior\n- **Plan mode** — preview the execution plan without modifying the target\n- **Liquibase export mode** — writes an initial Liquibase formatted SQL file for the discovered schema objects\n- **Drop-existing mode** — optionally drop matching target tables before recreating them\n- **Fake data replacement** — replace configured column values during copy and data export using `gofakeit`\n- **Post-data objects** — creates primary keys, checks, foreign keys, and indexes after data is loaded\n- **Integration tested** — includes testcontainers-based integration tests\n\n## Installation\n\n```sh\ngo install ./cmd/mssql-copier\n```\n\nOr build from source:\n\n```sh\ngo build -o mssql-copier ./cmd/mssql-copier\n```\n\n## Project layout\n\n```text\ncmd/mssql-copier/   CLI entrypoint\ninternal/copier/    copier engine, SQL metadata logic, and tests\n```\n\n## Usage\n\n### Basic copy\n\n```sh\nmssql-copier \\\n  --source \"sqlserver://user:pass@source-host:1433?database=SourceDB\" \\\n  --target \"sqlserver://user:pass@target-host:1433?database=TargetDB\"\n```\n\nWhen the target host is not local (`localhost`, `127.0.0.1`, or loopback IPv6 such as `::1`), the CLI asks for an explicit `yes` before it opens the target connection.\n\n### Plan mode (dry run)\n\nPreview which objects would be copied without touching the target. In plan mode, only `--source` is required; `--target` is optional and is shown in the output only when provided:\n\n```sh\nmssql-copier --plan --source \"sqlserver://...\"\n```\n\n### DDL export\n\nWrite a source-only DDL baseline file for the selected schema objects. The generated file is Liquibase-formatted SQL, so it can be used directly as a Liquibase changelog. This mode exports DDL only; it does not export table data.\n\n```sh\nmssql-copier \\\n  --source \"sqlserver://...\" \\\n  --export-ddl ./export/initial.sql\n```\n\nThe generated file contains ordered Liquibase changesets for schemas, types, sequences, tables, constraints, indexes, views, functions, synonyms, procedures, and triggers. Because this is an initial baseline export, `--drop-existing` is not supported with this mode.\n\n### Data export\n\nWrite a source-only data seed file for the selected tables. The generated file is plain SQL with semicolon-terminated `SET IDENTITY_INSERT` and `INSERT` statements, with no `GO` batches.\n\n```sh\nmssql-copier \\\n  --source \"sqlserver://...\" \\\n  --export-data ./export/initial-data.sql\n```\n\nThe generated file contains deterministic table sections and row inserts ordered by primary key when available. It temporarily disables constraints on the exported tables before loading rows and re-checks them at the end so the script can run cleanly after a schema import even when foreign keys already exist. This mode exports table data only; it does not create schema objects, and `--drop-existing` is not supported with this mode.\n\n### Filtering objects\n\nFilters are applied by schema name and object name across copied tables and other discovered objects.\n\nWhen a filtered copy excludes a referenced parent table and that table is not already present on the target, foreign key recreation for the copied child table is skipped.\n\n```sh\n# Copy only tables in the \"sales\" schema\nmssql-copier --source \"...\" --target \"...\" --include-schemas \"sales\"\n\n# Exclude audit tables\nmssql-copier --source \"...\" --target \"...\" --exclude-tables \"*.audit_%\"\n\n# Copy only specific tables (schema-qualified)\nmssql-copier --source \"...\" --target \"...\" --include-tables \"dbo.orders,sales.customers\"\n```\n\n### Drop and recreate\n\n```sh\nmssql-copier --source \"...\" --target \"...\" --drop-existing\n```\n\nAlias user-defined types and user-defined table types are created before tables and procedures so recreated table definitions can keep alias types and TVP-based procedures can compile on the target. Sequences are created before tables so defaults like `NEXT VALUE FOR ...` work during table creation. Views are created after tables and indexes. Functions are created after views in dependency order. Synonyms are then recreated so late-bound references are available to programmable objects. Stored procedures are created after tables, views, functions, table types, and synonyms, with explicit dependency ordering across copied procedures. Table-scoped DML triggers are created after tables, procedures, and synonyms so they bind to recreated target tables without firing during the initial data copy and can reference copied synonyms. Existing target sequences, views, functions, procedures, triggers, and synonyms are refreshed on reruns. Alias types are recreated on rerun only when `--drop-existing` is set. Table types are created only when missing.\n\n### Tuning\n\n```sh\nmssql-copier \\\n  --source \"...\" --target \"...\" \\\n  --workers 8 \\\n  --batch-size 10000\n```\n\n### YAML configuration\n\nYou can keep most parameters in a YAML file. By default, the copier looks for `mssql-copier.yml` in the current working directory.\n\nA starter template is checked in at `mssql-copier.example.yml`.\n\n```sh\n# Uses ./mssql-copier.yml when present\nmssql-copier\n\n# Use a custom config file path\nmssql-copier --config ./config/prod.yml\n```\n\nCLI flags override values from YAML when both are provided.\n\n`--export-ddl` and `--export-data` must be passed as CLI flags. This lets export modes still reuse YAML values such as `source`, `workers`, and include/exclude filters.\n\n`fake-data` is YAML-only. Each entry maps a column selector to a [`gofakeit`](https://github.com/brianvoe/gofakeit) function. Selectors support:\n\n- exact column name: `name`\n- exact table and column: `users.name`\n- exact schema, table, and column: `dbo.users.name`\n- regex: any selector that is not a plain identifier path is treated as a case-insensitive regex and matched against `column`, `table.column`, and `schema.table.column`. Example: `name.*`\n\nUse `gofakeit` function names like `Email`, `FirstName`, or `LoremIpsumSentence`.\n\nParameters are optional and are appended after the function name using `;` in declared order. Examples: `LoremIpsumSentence;10` and `Price;1;100`.\n\nExample `mssql-copier.yml`:\n\n```yaml\nsource: sqlserver://user:pass@source-host:1433?database=SourceDB\ntarget: sqlserver://user:pass@target-host:1433?database=TargetDB\nworkers: 8\nbatch-size: 10000\nverbose: true\ndrop-existing: false\ninclude-schemas:\n  - sales\n  - reporting\nexclude-schemas:\n  - audit\ninclude-tables:\n  - sales.orders\n  - sales.customers\nexclude-tables:\n  - \"*.audit_%\"\nfake-data:\n  users.name: Name\n  email: Email\n  name.*: FirstName\n  summary: LoremIpsumSentence;10\n  amount: Price;1;100\n```\n\n### Flags\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--config` | `mssql-copier.yml` | Path to YAML config file; optional when using the default path |\n| `--source` | *(required)* | Source SQL Server DSN |\n| `--target` | *(required unless `--plan`)* | Target SQL Server DSN; non-local targets require an interactive `yes` confirmation |\n| `--plan` | `false` | Print execution plan without modifying target |\n| `--export-ddl` | | Write Liquibase-formatted DDL to a file; `--target` is not required |\n| `--export-data` | | Write plain SQL data inserts to a file; `--target` is not required |\n| `--workers` | `max(2, NumCPU())` | Number of concurrent table copy workers |\n| `--batch-size` | `5000` | Rows per bulk batch hint |\n| `--drop-existing` | `false` | Drop matching target tables before recreating |\n| `--verbose` | `true` | Log per-table activity |\n| `--include-schemas` | | Comma-separated schema names or wildcard patterns (YAML: list) |\n| `--exclude-schemas` | | Comma-separated schema names or wildcard patterns (YAML: list) |\n| `--include-tables` | | Comma-separated table names (`name` or `schema.name`) or wildcard patterns (YAML: list) |\n| `--exclude-tables` | | Comma-separated table names or wildcard patterns (YAML: list) |\n\n### Fake data replacement\n\nConfigured fake-data rules are applied in both copy mode and `--export-data` mode before values are written to the target or serialized into SQL inserts.\n\nRule precedence is:\n\n1. exact `schema.table.column`\n2. exact `table.column`\n3. exact `column`\n4. regex selectors, matched in deterministic order\n\nExamples:\n\n```yaml\nfake-data:\n  customer.email: Email\n  ssn: SSN\n  dbo.people.name: Name\n  name.*: FirstName\n  description: LoremIpsumSentence;10\n  price: Price;1;100\n```\n\nThe CLI validates every configured function and parameter list at startup and fails fast when a function name is unknown, parameters do not fit the selected function, or the function returns a complex value type that the copier cannot safely write to SQL Server.\n\n### DSN format\n\nUses the `go-mssqldb` driver. Examples:\n\n```\nsqlserver://user:password@host:1433?database=MyDB\nsqlserver://user:password@host:1433?database=MyDB\u0026encrypt=true\u0026trustservercertificate=true\n```\n\n### Wildcard patterns\n\nFilter arguments support SQL-style and glob-style wildcards:\n- `*` or `%` — matches any sequence of characters\n- `?` or `_` — matches exactly one character\n\nExamples: `sales*`, `dbo.%`, `audit_202?`, `*_archive`\n\n## How it works\n\n1. **Discover** — queries `sys.tables`, `sys.types`, `sys.table_types`, `sys.sequences`, `sys.views`, `sys.objects`, `sys.procedures`, `sys.triggers`, `sys.synonyms`, `sys.columns`, `sys.indexes`, and other system catalog views on the source to build metadata for copied objects\n2. **Filter** — applies include/exclude rules against schema names and object names\n3. **Plan** (optional) — prints the planned actions and exits\n4. **Create schemas** — creates any missing non-`dbo` schemas needed by copied objects on the target\n5. **Prepare target** — optionally drops existing target tables\n6. **Create alias types** — creates copied alias user-defined types before tables are recreated\n7. **Create table types** — creates copied user-defined table types before dependent procedures are recreated\n8. **Create sequences** — creates or updates copied sequences before tables are created\n9. **Create tables** — generates and executes `CREATE TABLE` statements from source column definitions (including defaults, computed columns, collations, and preserved alias types)\n10. **Copy data** — distributes tables across worker goroutines; each table is copied in a single transaction using bulk copy or row-insert depending on column type compatibility\n11. **Post-data objects** — creates primary keys, check constraints, foreign keys, and indexes\n12. **Create views** — creates or updates copied views in dependency order\n13. **Create functions** — creates or updates copied SQL functions in dependency order\n14. **Create synonyms** — recreates copied synonyms after referenced objects are in place\n15. **Create procedures** — creates or updates copied stored procedures after their copied dependencies are in place\n16. **Create triggers** — creates or updates copied table-scoped DML triggers after tables, procedures, and synonyms are in place\n\n### Bulk vs. row-insert\n\nThe copier prefers bulk copy (`COPY IN` with `TABLOCK`) for performance. It falls back to row-by-row `INSERT` statements when a table contains column types not supported by the bulk protocol (e.g., `xml`, `sql_variant`, user-defined types, etc.).\n\n### Identity columns\n\nTables with identity columns have `SET IDENTITY_INSERT ON` enabled during the copy so source identity values are preserved.\n\n### Views\n\nViews are copied automatically with table copy. The copier reads the source view definitions, keeps inter-view dependencies in order, and applies them to the target with `CREATE OR ALTER VIEW` so repeated runs stay idempotent.\n\n### Functions\n\nSQL scalar and table-valued functions are copied automatically with table copy. The copier reads their definitions with `OBJECT_DEFINITION`, orders copied functions by inter-function dependencies, and applies them with `CREATE OR ALTER FUNCTION` so reruns stay idempotent.\n\n### Sequences\n\nSequences are copied automatically with table copy. Their definitions are created on the target when missing and altered on rerun so defaults based on `NEXT VALUE FOR ...` continue to work after the table data is copied.\n\n### Alias User-Defined Types\n\nAlias user-defined types are copied automatically with table copy. For alias types based on built-in SQL Server scalar types, the copier preserves the alias type in recreated table column definitions. Existing target alias types are recreated automatically only when `--drop-existing` is set.\n\n### User-Defined Table Types\n\nUser-defined table types are copied automatically with procedure copy. The copier creates the table type object when it is missing so stored procedures that use table-valued parameters can be recreated on the target.\n\nSQL Server does not support `CREATE OR ALTER TYPE`, so existing table type definitions are not rewritten on rerun. If a copied table type changes shape on the source, drop the target type and rerun the copier.\n\n### Stored Procedures\n\nStored procedures are copied automatically with table copy. Their definitions are applied with `CREATE OR ALTER PROCEDURE`, and copied procedures are ordered using dependencies from `sys.sql_expression_dependencies` so procedures that reference other copied procedures or synonyms are created after those dependencies.\n\n### Triggers\n\nTable-scoped DML triggers are copied automatically with table copy. The copier reads trigger definitions from `OBJECT_DEFINITION`, resolves dependencies on copied programmable objects from `sys.sql_expression_dependencies`, recreates them with `CREATE OR ALTER TRIGGER`, and reapplies the enabled or disabled state from the source.\n\n### Synonyms\n\nSynonyms are copied automatically with table copy. Because SQL Server does not support `CREATE OR ALTER SYNONYM`, the copier drops an existing target synonym and recreates it from `sys.synonyms.base_object_name` on each run.\n\n## Development\n\nThis project uses [Task](https://taskfile.dev) for build automation.\n\n```sh\n# Show available tasks\ntask\n\n# Format code\ntask format\n\n# Build\ntask build\n\n# Run unit tests\ntask test\n\n# Run integration tests (requires Docker)\ntask test:integration\n```\n\nIntegration tests use [testcontainers-go](https://github.com/testcontainers/testcontainers-go) to spin up real SQL Server instances in Docker.\n\n## License\n\nMIT License. See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fralscha%2Fmssql-copier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fralscha%2Fmssql-copier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fralscha%2Fmssql-copier/lists"}