{"id":30615660,"url":"https://github.com/alexfalkowski/migrieren","last_synced_at":"2026-06-05T06:01:32.168Z","repository":{"id":37881225,"uuid":"486162925","full_name":"alexfalkowski/migrieren","owner":"alexfalkowski","description":"Migrieren provides a way to migrate your databases.","archived":false,"fork":false,"pushed_at":"2026-05-31T14:36:31.000Z","size":3934,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-05-31T15:16:39.828Z","etag":null,"topics":["cucumber","golang","make","ruby"],"latest_commit_sha":null,"homepage":"https://alexfalkowski.github.io/migrieren","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alexfalkowski.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2022-04-27T11:24:05.000Z","updated_at":"2026-05-31T14:32:46.000Z","dependencies_parsed_at":"2026-04-08T10:00:48.904Z","dependency_job_id":null,"html_url":"https://github.com/alexfalkowski/migrieren","commit_stats":{"total_commits":359,"total_committers":2,"mean_commits":179.5,"dds":0.350974930362117,"last_synced_commit":"d5f2c55794febc55a7c11a4b1bb0e64634676cb6"},"previous_names":[],"tags_count":1269,"template":false,"template_full_name":null,"purl":"pkg:github/alexfalkowski/migrieren","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexfalkowski%2Fmigrieren","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexfalkowski%2Fmigrieren/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexfalkowski%2Fmigrieren/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexfalkowski%2Fmigrieren/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alexfalkowski","download_url":"https://codeload.github.com/alexfalkowski/migrieren/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexfalkowski%2Fmigrieren/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33932040,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cucumber","golang","make","ruby"],"created_at":"2025-08-30T08:06:22.340Z","updated_at":"2026-06-05T06:01:32.142Z","avatar_url":"https://github.com/alexfalkowski.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Gopher](assets/gopher.png)\n[![CircleCI](https://circleci.com/gh/alexfalkowski/migrieren.svg?style=shield)](https://circleci.com/gh/alexfalkowski/migrieren)\n[![codecov](https://codecov.io/gh/alexfalkowski/migrieren/graph/badge.svg?token=R2OD8WIKD0)](https://codecov.io/gh/alexfalkowski/migrieren)\n[![Go Report Card](https://goreportcard.com/badge/github.com/alexfalkowski/migrieren)](https://goreportcard.com/report/github.com/alexfalkowski/migrieren)\n[![Go Reference](https://pkg.go.dev/badge/github.com/alexfalkowski/migrieren.svg)](https://pkg.go.dev/github.com/alexfalkowski/migrieren)\n[![Stability: Active](https://masterminds.github.io/stability/active.svg)](https://masterminds.github.io/stability/active.html)\n\n# Migrieren\n\nMigrieren is a Go service that runs database schema migrations through a gRPC API with an HTTP RPC façade.\n\nThe service wraps [`golang-migrate/migrate`](https://github.com/golang-migrate/migrate) so callers can ask it to migrate a named database to a target version without embedding migration logic into every application.\n\n## What the service does\n\n- Exposes a single RPC: `migrieren.v1.Service/Migrate`.\n- Looks up a logical database name in configuration.\n- Reads the migration source URL and database URL through the service filesystem abstraction.\n- Executes the migration with `golang-migrate`.\n- Returns migration logs and request metadata to the caller.\n- Publishes HTTP and gRPC health checks plus Prometheus-style metrics when configured through the shared service runtime.\n\n## Supported drivers\n\nThe currently wired drivers are defined in code:\n\n- Migration sources:\n  - `file://...`\n  - `github://...`\n- Databases:\n  - Postgres via `pgx5://...`\n\nDatabase URLs use the `pgx5://` scheme in config and secrets. Internally the service rewrites that to the driver URL format expected by the Postgres migrate driver.\n\n## Prerequisites\n\n- Go `1.26.0` or newer.\n- Ruby and Bundler for the feature-test harness under `test/`.\n- The `bin/` git submodule. Most `make` targets delegate into scripts under that submodule.\n\nInitialize the submodule before relying on `make`:\n\n```sh\ngit submodule sync\ngit submodule update --init\n```\n\nIf you hit vendoring errors such as \"inconsistent vendoring\", refresh dependencies with:\n\n```sh\nmake dep\n```\n\n## Quick start\n\nInstall dependencies, build the binary, and start the server with the checked-in development/test config:\n\n```sh\nmake dep\nmake build\n./migrieren server -i file:test/.config/server.yml\n```\n\nThis builds `./migrieren` in the repository root.\n\nFor live reload during local development:\n\n```sh\nmake dev\n```\n\n`make dev` uses `air` and starts the same `server` command with `test/.config/server.yml`.\n\n## How Migrieren resolves a migration\n\nThe API accepts a logical database name such as `postgres`. The service then:\n\n1. Looks that name up in `migrate.databases`.\n2. Reads the configured `source` value through its filesystem abstraction.\n3. Reads the configured `url` value through the same abstraction.\n4. Passes the resolved source URL and database URL to the core migrator.\n\nThat means `source` and `url` in the YAML config are usually references to files or other resolvable inputs, not the final literal migration/source strings themselves.\n\n## Configuration\n\nMigrieren is configured through the `server` command input file. The repository includes a representative config at `test/.config/server.yml`.\n\n### Minimal configuration shape\n\nAt minimum, you need:\n\n- `migrate.databases`\n- `transport.http.address`\n- `transport.grpc.address`\n\nExample:\n\n```yaml\nenvironment: development\nhealth:\n  duration: 1s\n  timeout: 1s\nmigrate:\n  databases:\n    - name: postgres\n      source: file:secrets/source\n      url: file:secrets/pg\ntransport:\n  http:\n    address: tcp://:11000\n    timeout: 5s\n  grpc:\n    address: tcp://:12000\n    timeout: 5s\n```\n\n### Migration database entries\n\nEach configured database entry has:\n\n- `name`: the logical database name used by the API request.\n- `source`: how to resolve the migration source URL.\n- `url`: how to resolve the database connection URL.\n\nIn the checked-in test setup, the referenced secret files contain the actual values consumed by the migrator:\n\n```text\n# test/secrets/source\nfile://migrations\n\n# test/secrets/github\ngithub://alexfalkowski/app-config/test/migrations\n\n# test/secrets/pg\npgx5://test:test@localhost:5433/test?sslmode=disable\n```\n\nSo a config entry like this:\n\n```yaml\nmigrate:\n  databases:\n    - name: postgres\n      source: file:secrets/source\n      url: file:secrets/pg\n```\n\nultimately migrates Postgres using the `file://migrations` source and the `pgx5://...` database URL resolved from those files.\n\n### About the checked-in test config\n\n`test/.config/server.yml` intentionally contains both valid and invalid database definitions:\n\n- `postgres` and `github` are used for successful migration scenarios.\n- `missing_source`, `invalid_source`, `missing_url`, `invalid_url`, `invalid_db`, and `invalid_port` exist to exercise failure paths in feature tests.\n\nThat is why the checked-in config is useful for development and testing, but it is not a \"healthy production example\" as-is.\n\n## API\n\nThe protobuf contract lives at `api/migrieren/v1/service.proto`.\n\n### gRPC\n\nThe service exposes:\n\n- `migrieren.v1.Service/Migrate`\n\nRequest:\n\n- `database`: logical database name from config.\n- `version`: target migration version as `uint64`.\n\nResponse:\n\n- `meta`: request metadata emitted by the service runtime.\n- `migration.database`: echoed database name.\n- `migration.version`: echoed target version.\n- `migration.logs`: in-memory migration log lines collected during execution.\n\nConceptual request:\n\n```protobuf\ndatabase: \"postgres\"\nversion: 1\n```\n\n### HTTP façade\n\nThe HTTP RPC façade exposes the same operation at:\n\n- `POST /migrieren.v1.Service/Migrate`\n\nExample:\n\n```http\nPOST http://localhost:11000/migrieren.v1.Service/Migrate\nContent-Type: application/json\n\n{\n  \"database\": \"postgres\",\n  \"version\": 1\n}\n```\n\n### Error mapping\n\nTransport behavior is intentionally simple:\n\n- Unknown database name:\n  - gRPC: `NotFound`\n  - HTTP: `404`\n- Configuration, source, database, or migration failures:\n  - gRPC: `Internal`\n  - HTTP: `500`\n\nThe core migrator also treats `migrate.ErrNoChange` as a successful no-op and still returns any accumulated logs.\n\n## Health and observability\n\nWhen running with the shared service runtime, Migrieren exposes:\n\n- HTTP health endpoints:\n  - `/healthz`\n  - `/livez`\n  - `/readyz`\n- HTTP metrics:\n  - `/metrics`\n- gRPC health checks for `migrieren.v1.Service`\n\nThere is an important detail in the checked-in test config:\n\n- gRPC health for `migrieren.v1.Service` reports `SERVING`.\n- HTTP `/livez` and `/readyz` report healthy.\n- HTTP `/healthz` is expected to be unhealthy because the test config deliberately registers invalid database entries for failure-path coverage.\n\nThe sample test config also enables:\n\n- text logging,\n- Prometheus metrics, and\n- OTLP tracing with `http://localhost:4318/v1/traces`.\n\n## Development\n\n### Common commands\n\nUse `make help` to list available targets. Common ones are:\n\n```sh\nmake dep\nmake build\nmake build-test\nmake specs\nmake features\nmake benchmarks\nmake lint\nmake format\nmake sec\nmake coverage\nmake proto-generate\nmake proto-lint\nmake proto-breaking\n```\n\nWhat those do in this repository:\n\n- `make dep`: installs Go dependencies, runs `go mod tidy`, vendors modules, and installs Ruby gems for `test/`.\n- `make build`: builds the release binary `./migrieren`.\n- `make build-test`: builds a test binary with the `features` build tag.\n- `make specs`: runs Go tests with `gotestsum`, `-race`, and vendored dependencies.\n- `make features`: builds the test binary and runs the Ruby/Cucumber feature suite in `test/`.\n- `make benchmarks`: builds the release binary and runs the benchmark-tagged Ruby harness.\n- `make lint`: lints Go, the Ruby test harness, and protobuf definitions.\n- `make sec`: runs `govulncheck`.\n- `make coverage`: creates HTML and function coverage reports under `test/reports/`.\n\n### Local test harness expectations\n\nThe Ruby harness under `test/` assumes:\n\n- HTTP server on `http://localhost:11000`\n- gRPC server on `localhost:12000`\n- Postgres reachable on `localhost:5433`\n\nThe feature harness process wiring lives in `test/nonnative.yml`.\n\n### Repository layout\n\nKey locations:\n\n- `main.go`: CLI entrypoint.\n- `internal/cmd/server.go`: registers the `server` command.\n- `internal/config/config.go`: top-level config composition.\n- `internal/migrate/`: core migration engine and driver wiring.\n- `internal/api/migrate/`: transport-facing adapter that resolves database names through config.\n- `internal/api/v1/transport/grpc/`: gRPC server implementation.\n- `internal/api/v1/transport/http/`: HTTP RPC route registration.\n- `internal/health/`: health registration and database-specific health checks.\n- `api/migrieren/v1/service.proto`: protobuf contract.\n- `test/`: Ruby feature-test harness, migrations, and local test fixtures.\n\n## Protobuf workflow\n\nThe API contract is managed with `buf`.\n\nGenerate code:\n\n```sh\nmake proto-generate\n```\n\nLint and breaking-change checks:\n\n```sh\nmake proto-lint\nmake proto-breaking\n```\n\nGeneration is configured in `api/buf.gen.yaml` and currently writes:\n\n- Go protobuf and gRPC files into `api/`\n- Ruby protobuf and gRPC files into `test/lib/`\n\nDo not hand-edit generated protobuf stubs. Update `api/migrieren/v1/service.proto` and regenerate instead.\n\n## Notes for contributors\n\n- Package documentation for Go packages belongs in `doc.go`.\n- The feature harness public API is primarily the Ruby helpers under `test/lib/`.\n- If `go test` or `go list` starts failing because of vendoring drift, run `make dep`.\n\n## Changelog\n\nSee `CHANGELOG.md` for release notes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexfalkowski%2Fmigrieren","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falexfalkowski%2Fmigrieren","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexfalkowski%2Fmigrieren/lists"}