{"id":26796061,"url":"https://github.com/typisttech/vipdatasync","last_synced_at":"2025-03-29T18:17:11.341Z","repository":{"id":257783448,"uuid":"859599337","full_name":"typisttech/vipdatasync","owner":"typisttech","description":"WIP","archived":false,"fork":false,"pushed_at":"2024-12-09T11:52:34.000Z","size":103,"stargazers_count":0,"open_issues_count":5,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-22T05:32:32.174Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/typisttech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-19T00:24:02.000Z","updated_at":"2024-09-27T23:34:34.000Z","dependencies_parsed_at":"2024-09-20T06:43:10.995Z","dependency_job_id":"185c6933-6cce-43fb-b8b5-63c355deac3c","html_url":"https://github.com/typisttech/vipdatasync","commit_stats":null,"previous_names":["typisttech/vipdatasync"],"tags_count":42,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/typisttech%2Fvipdatasync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/typisttech%2Fvipdatasync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/typisttech%2Fvipdatasync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/typisttech%2Fvipdatasync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/typisttech","download_url":"https://codeload.github.com/typisttech/vipdatasync/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246223319,"owners_count":20743168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-29T18:17:10.682Z","updated_at":"2025-03-29T18:17:11.335Z","avatar_url":"https://github.com/typisttech.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# vipdatasync\n\nCLI utility for WordPress VIP [data sync](https://docs.wpvip.com/databases/data-sync/) management.\n\n## Usage\n\n### Validate\n\nTo validate the environment-specific YAML [config file](https://docs.wpvip.com/databases/data-sync/config-file/) against production URLs:\n\n1. Download the environment-specific YAML [config file](https://docs.wpvip.com/databases/data-sync/config-file/) from the **`master`** branches of [wpcomvip](https://github.com/wpcomvip) repos\n2. Export the **production** URLs via:\n    ```bash\n    vip @my-app.produciton -- wp site list --fields=url --format=json\n   ```\n3. Run the `validate` command:\n    ```bash\n    vipdatasync validate --config=config.yml --urls=sites.json\n    ```\n\n\u003cdetails\u003e\n\n\u003csummary\u003eExample: Bash Script\u003c/summary\u003e\n\n```bash\ngh api \\\n  -H \"Accept: application/vnd.github.raw+yaml\" \\\n  -H \"X-GitHub-Api-Version: 2022-11-28\" \\\n  /repos/wpcomvip/my-app/contents/config/.vip.my-app.staging.yml\\?ref=preprod \\\n  \u003e ~/Desktop/.vip.my-app.staging.yml\n\nvip @my-app.production --yes -- wp site list --fields=url --format=json \u003e ~/Desktop/my-app-production-sites.json\n\nvipdatasync validate --config=~/Desktop/.vip.my-app.staging.yml --urls=~/Desktop/my-app-production-sites.json\n```\n\n\u003c/details\u003e\n\n#### Duplicated Destinations\n\nMultiple domain map items to the same destination.\n\n\u003e [!IMPORTANT]\n\u003e **Special case**:\n\u003e It is considered **valid** if both `www` and non-`www` sources are mapped to the same destination where the `www` item comes first.\n\n\u003cdetails open\u003e\n\n\u003csummary\u003eDuplication\u003c/summary\u003e\n\n```yaml\ndata_sync:\n  domain_map:\n    vip-example.com: example-com-staging.go-vip.net\n    example.go-vip.net: example-com-staging.go-vip.net\n```\n\n```console\n$ vipdatasync validate\n\nDUPLICATED DESTINATIONS\n\n     1. example-com-staging.go-vip.net\n         - vip-example.com\n         - example.go-vip.net\n```\n\n*See [`validate_duplicated_destinations.txtar`](cmd/vipdatasync/testdata/script/validate_duplicated_destinations.txtar).*\n\n\u003c/details\u003e\n\n\u003cdetails open\u003e\n\n\u003csummary\u003e\u003cstrong\u003eSpecial case\u003c/strong\u003e: \u003ccode\u003ewww\u003c/code\u003e and non-\u003ccode\u003ewww\u003c/code\u003e with good ordering\u003c/summary\u003e\n\n```yaml\ndata_sync:\n  domain_map:\n    www.vip-example.com: example-com-staging.go-vip.net\n    vip-example.com: example-com-staging.go-vip.net\n```\n\n```console\n$ vipdatasync validate\n\nDUPLICATED DESTINATIONS\n\n        No problems found\n```\n\n*See [`validate_duplicated_destinations_www_ordering_good.txtar`](cmd/vipdatasync/testdata/script/validate_duplicated_destinations_www_ordering_good.txtar).*\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\n\u003csummary\u003eExact duplication\u003c/summary\u003e\n\n```yaml\ndata_sync:\n  domain_map:\n    vip-example.com: example-com-staging.go-vip.net\n    vip-example.com: example-com-staging.go-vip.net\n```\n\n```console\n$ vipdatasync validate\n\nDUPLICATED DESTINATIONS\n\n     1. example-com-staging.go-vip.net\n         - vip-example.com\n         - vip-example.com\n```\n\n*See [`validate_duplicated_destinations_exactly.txtar`](cmd/vipdatasync/testdata/script/validate_duplicated_destinations_exactly.txtar).*\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\n\u003csummary\u003e\u003ccode\u003ewww\u003c/code\u003e and non-\u003ccode\u003ewww\u003c/code\u003e with extra items\u003c/summary\u003e\n\n```yaml\ndata_sync:\n  domain_map:\n    www.vip-example.com: example-com-staging.go-vip.net\n    vip-example.com: example-com-staging.go-vip.net\n    foo.com: example-com-staging.go-vip.net\n```\n\n```console\n$ vipdatasync validate\n\nDUPLICATED DESTINATIONS\n\n     1. example-com-staging.go-vip.net\n         - www.vip-example.com\n         - vip-example.com\n         - foo.com\n```\n\n*See [`validate_duplicated_destinations_www_extra.txtar`](cmd/vipdatasync/testdata/script/validate_duplicated_destinations_www_extra.txtar).*\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\n\u003csummary\u003e\u003ccode\u003ewww\u003c/code\u003e and non-\u003ccode\u003ewww\u003c/code\u003e with bad ordering\u003c/summary\u003e\n\n```yaml\ndata_sync:\n  domain_map:\n    vip-example.com: example-com-staging.go-vip.net\n    www.vip-example.com: example-com-staging.go-vip.net\n```\n\n```console\n$ vipdatasync validate\n\nDUPLICATED DESTINATIONS\n\n     1. example-com-staging.go-vip.net\n         - vip-example.com\n         - www.vip-example.com\n```\n\n*See [`validate_duplicated_destinations_www_ordering_bad.txtar`](cmd/vipdatasync/testdata/script/validate_duplicated_destinations_www_ordering_bad.txtar).*\n\n\u003c/details\u003e\n\n##### Duplicated **Catch-All** Destinations\n\n\u003e [!NOTE]\n\u003e It is **safe to ignore catch-all** items which map both [convenience](https://docs.wpvip.com/domains/convenience-domains/) and production domains to the same destination.\n\nThe command reports duplicated destination problems for catch-all items.\nThere is no plan to fix this false-alarm because of the complexity to reliably identify the main production domain.\n\n\u003cdetails\u003e\n\n\u003csummary\u003eExample: Catch-all\u003c/summary\u003e\n\n**This is safe to ignore.**\n\n```yaml\ndata_sync:\n  domain_map:\n    vip-example.com: example-com-staging.go-vip.net\n    vip-example.go-vip.net: example-com-staging.go-vip.net\n```\n\n```console\n# This is safe to ignore\n\n$ vipdatasync validate\n\nDUPLICATED DESTINATIONS\n\n     1. example-com-staging.go-vip.net\n         - vip-example.com\n         - vip-example.go-vip.net\n```\n\n\u003c/details\u003e\n\n#### Duplicated Tos\n\nMultiple production URLs map to the same environment-specific URL.\n\n\u003cdetails open\u003e\n\n\u003csummary\u003eDuplication\u003c/summary\u003e\n\n```yaml\ndata_sync:\n   domain_map:\n      vip-example.com: example-com-staging.go-vip.net\n      example.go-vip.net/zh: example-com-staging.go-vip.net/en\n```\n\n```json\n[\n  {\n     \"url\": \"https://vip-example.com/en\"\n  },\n  {\n     \"url\": \"https://example.go-vip.net/zh\"\n  }\n]\n```\n\n```console\n$ vipdatasync validate\n\nDUPLICATED TOS\n\n     1. https://example-com-staging.go-vip.net/en\n         - https://vip-example.com/en\n           vip-example.com\n             -\u003e example-com-staging.go-vip.net\n         - https://example.go-vip.net/zh\n           example.go-vip.net/zh\n             -\u003e example-com-staging.go-vip.net/en\n```\n\n*See [`validate_duplicated_tos.txtar`](cmd/vipdatasync/testdata/script/validate_duplicated_tos.txtar).*\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\n\u003csummary\u003eUnreplaced\u003c/summary\u003e\n\n```yaml\ndata_sync:\n   domain_map:\n      vip-example.com: example-com-staging.go-vip.net\n```\n\n```json\n[\n   {\n      \"url\": \"https://vip-example.com/en\"\n   },\n   {\n      \"url\": \"https://example-com-staging.go-vip.net/en\"\n   }\n]\n```\n\n```console\n$ vipdatasync validate\n\nDUPLICATED TOS\n\n     1. https://example-com-staging.go-vip.net/en\n         - https://vip-example.com/en\n           vip-example.com\n             -\u003e example-com-staging.go-vip.net\n         - https://example-com-staging.go-vip.net/en\n```\n\n*See [`validate_duplicated_tos_unreplaced.txtar`](cmd/vipdatasync/testdata/script/validate_duplicated_tos_unreplaced.txtar).*\n\n\u003c/details\u003e\n\n#### Unreplaced URLs\n\nProduction URLs that are not replaced by any domain map item.\n\n\u003cdetails open\u003e\n\n\u003csummary\u003eUnreplaced\u003c/summary\u003e\n\n```yaml\ndata_sync:\n   domain_map:\n      vip-example.com: example-com-staging.go-vip.net\n      example.go-vip.net/zh: example-com-staging.go-vip.net/zh\n```\n\n```json\n[\n   {\n      \"url\": \"https://example.go-vip.net\"\n   },\n   {\n      \"url\": \"https://vip-example.com/en\"\n   },\n   {\n      \"url\": \"https://example.go-vip.net/en\"\n   },\n   {\n      \"url\": \"https://example.go-vip.net/zh\"\n   }\n]\n```\n\n```console\n$ vipdatasync validate\n\nUNREPLACED URLS\n\n     1. https://example.go-vip.net\n     2. https://example.go-vip.net/en\n```\n\n*See [`validate_duplicated_tos_unreplaced.txtar`](cmd/vipdatasync/testdata/script/validate_duplicated_tos_unreplaced.txtar).*\n\n\u003c/details\u003e\n\n#### Unused Domain Map Items\n\nDomain map items that are not used by any production URLs.\n\n\u003cdetails open\u003e\n\n\u003csummary\u003eUnused\u003c/summary\u003e\n\n```yaml\ndata_sync:\n   domain_map:\n      foo.com/zh: foo-com-staging.go-vip.net/zh\n      foo.com: foo-com-staging.go-vip.net\n\n      bar.com/zh: bar-com-staging.go-vip.net/zh\n      bar.com/fr: bar-com-staging.go-vip.net/fr\n      bar.com: bar-com-staging.go-vip.net\n\n      vip-example.com: example-com-staging.go-vip.net\n```\n\n```json\n[\n   {\n      \"url\": \"https://vip-example.com\"\n   },\n   {\n      \"url\": \"https://vip-example.com/en\"\n   },\n   {\n      \"url\": \"https://foo.com/zh\"\n   },\n   {\n      \"url\": \"https://bar.com/en\"\n   },\n   {\n      \"url\": \"https://bar.com/zh\"\n   },\n   {\n      \"url\": \"https://bar.com\"\n   }\n]\n```\n\n```console\n$ vipdatasync validate\n\nUNUSED DOMAIN MAP ITEMS\n\n     1. foo.com\n          -\u003e foo-com-staging.go-vip.net\n     2. bar.com/fr\n          -\u003e bar-com-staging.go-vip.net/fr\n```\n\n*See [`validate_unused_domain_map_items.txtar`](cmd/vipdatasync/testdata/script/validate_unused_domain_map_items.txtar).*\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\n\u003csummary\u003eBad ordering\u003c/summary\u003e\n\n```yaml\ndata_sync:\n   domain_map:\n      foo.com: foo-com-staging.go-vip.net\n      foo.com/zh: foo-com-staging.go-vip.net/zh\n\n      bar.com: bar-com-staging.go-vip.net\n      bar.com/fr: bar-com-staging.go-vip.net/fr\n      bar.com/zh: bar-com-staging.go-vip.net/zh\n\n      vip-example.com: example-com-staging.go-vip.net\n```\n\n```json\n[\n   {\n      \"url\": \"https://vip-example.com\"\n   },\n   {\n      \"url\": \"https://vip-example.com/en\"\n   },\n   {\n      \"url\": \"https://foo.com/zh\"\n   },\n   {\n      \"url\": \"https://bar.com/en\"\n   },\n   {\n      \"url\": \"https://bar.com/zh\"\n   },\n   {\n      \"url\": \"https://bar.com\"\n   }\n]\n```\n\n```console\n$ vipdatasync validate\n\nUNUSED DOMAIN MAP ITEMS\n\n     1. foo.com/zh\n          -\u003e foo-com-staging.go-vip.net/zh\n     2. bar.com/fr\n          -\u003e bar-com-staging.go-vip.net/fr\n     3. bar.com/zh\n          -\u003e bar-com-staging.go-vip.net/zh\n```\n\n*See [`validate_unused_domain_map_items_bad_ordering.txtar`](cmd/vipdatasync/testdata/script/validate_unused_domain_map_items_bad_ordering.txtar).*\n\n\u003c/details\u003e\n\n## Install\n\n### Prebuilt Binaries (Preferred)\n\n\u003e [!TIP]\n\u003e Find your operating system and hardware architecture via:\n\u003e \n\u003e ```console\n\u003e $ uname -om\n\u003e Darwin arm64\n\u003e ```\n\nPrebuilt binaries are available for a variety of operating systems and architectures.\nVisit the [latest release](https://github.com/typisttech/vipdatasync/releases/latest) page, and scroll down to the Assets section.\n\n1. Download the archive for the desired edition, operating system, and architecture\n2. Extract the archive\n3. *(Optional)* Verify the integrity and provenance of the executable\n    ```console\n    $ gh attestation verify /path/to/vipdatasync_999.888.777_xxx_yyy/vipdatasync --repo typisttech/vipdatasync --signer-repo typisttech/vipdatasync\n\n    Loaded digest sha256:xxxxxxxxxxxxxxxxxx for file:///path/to/vipdatasync_999.888.777_xxx_yyy/vipdatasync\n    Loaded 1 attestation from GitHub API\n    ✓ Verification succeeded!\n\n    sha256:xxxxxxxxxxxxxxxxxx was attested by:\n    REPO                    PREDICATE_TYPE                  WORKFLOW\n    typisttech/vipdatasync  https://slsa.dev/provenance/v1  .github/workflows/go-release.yml@refs/tags/v999.888.777\n    ```\n4. Move the executable to the desired directory\n5. Add this directory to the `PATH` environment variable\n6. Verify that you have execute permission on the file\n\nPlease consult your operating system documentation if you need help setting file permissions or modifying your `PATH` environment variable.\n\nIf you do not see a prebuilt binary for the desired edition, operating system, and architecture,\ninstall using one of the methods described below.\n\n### Install from Source\n\n\u003e [!TIP]\n\u003e See `$ go help install` or [https://golang.org/ref/mod#go-install](https://golang.org/ref/mod#go-install) for details.\n\n```bash\ngo install github.com/typisttech/vipdatasync/cmd/vipdatasync@latest\n```\n\n## Develop\n\n```bash\n# Run the tests\ngo test -v -count=1 -race -shuffle=on ./...\n\n# Update testscript golden files\n# Note: This doesn't update the README.md examples\nUPDATE_SCRIPTS=true go test ./cmd/...\n\n# Lint\ngolangci-lint run\ngoreleaser check\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftypisttech%2Fvipdatasync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftypisttech%2Fvipdatasync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftypisttech%2Fvipdatasync/lists"}