{"id":31654653,"url":"https://github.com/andrei-polukhin/pgdbtemplate","last_synced_at":"2025-12-30T00:07:49.119Z","repository":{"id":313448450,"uuid":"1051457402","full_name":"andrei-polukhin/pgdbtemplate","owner":"andrei-polukhin","description":"Go library for creating PostgreSQL test databases using template databases for lightning-fast test execution.","archived":false,"fork":false,"pushed_at":"2025-09-29T08:23:25.000Z","size":126,"stargazers_count":23,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-29T10:18:14.305Z","etag":null,"topics":["database","go","golang","performance","pgx","postgresql","pq","testcontainers","testing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/andrei-polukhin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"docs/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-06T03:22:24.000Z","updated_at":"2025-09-29T08:23:28.000Z","dependencies_parsed_at":"2025-09-21T06:11:00.528Z","dependency_job_id":"c9d0b079-45a7-444f-845f-b98281939f61","html_url":"https://github.com/andrei-polukhin/pgdbtemplate","commit_stats":null,"previous_names":["andrei-polukhin/pgdbtemplate"],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/andrei-polukhin/pgdbtemplate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrei-polukhin%2Fpgdbtemplate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrei-polukhin%2Fpgdbtemplate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrei-polukhin%2Fpgdbtemplate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrei-polukhin%2Fpgdbtemplate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/andrei-polukhin","download_url":"https://codeload.github.com/andrei-polukhin/pgdbtemplate/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrei-polukhin%2Fpgdbtemplate/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278770726,"owners_count":26042828,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","go","golang","performance","pgx","postgresql","pq","testcontainers","testing"],"created_at":"2025-10-07T12:01:45.718Z","updated_at":"2025-12-30T00:07:49.113Z","avatar_url":"https://github.com/andrei-polukhin.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# pgdbtemplate\n\n[![Go Reference](https://pkg.go.dev/badge/github.com/andrei-polukhin/pgdbtemplate.svg)](https://pkg.go.dev/github.com/andrei-polukhin/pgdbtemplate)\n[![CI](https://github.com/andrei-polukhin/pgdbtemplate/actions/workflows/test.yml/badge.svg)](https://github.com/andrei-polukhin/pgdbtemplate/actions/workflows/test.yml)\n[![Coverage](https://codecov.io/gh/andrei-polukhin/pgdbtemplate/branch/main/graph/badge.svg)](https://codecov.io/gh/andrei-polukhin/pgdbtemplate)\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/andrei-polukhin/pgdbtemplate/blob/main/LICENSE)\n\nA high-performance Go library for creating PostgreSQL test databases using\ntemplate databases for lightning-fast test execution.\n\n## Features\n\n- **🚀 Lightning-fast test databases** - 1.2-1.6x faster than traditional approach,\n  scales to 500 databases, ~17% less memory usage\n- **🔒 Thread-safe** - concurrent test database management\n- **📊 Scales with complexity** - performance advantage increases with schema complexity\n- **⚡ Multiple drivers** - supports both `pq` and `pgx` drivers\n- **🧪 Flexible testing** support for various test scenarios\n- **📦 Testcontainers integration** for containerized testing\n- **🔧 Configurable** migration runners and connection providers\n\n## Why Choose `pgdbtemplate`?\n\nEvaluating PostgreSQL testing libraries? See our detailed\n[comparison with other solutions](docs/COMPARISON.md).\n\n## Installation\n\n```bash\ngo get github.com/andrei-polukhin/pgdbtemplate\n```\n\n## Choose a PostgreSQL Driver\n\nChoose either one of these PostgreSQL drivers:\n\n- `github.com/andrei-polukhin/pgdbtemplate-pgx` (for `pgx/v5` with connection pooling)\n- `github.com/andrei-polukhin/pgdbtemplate-pq` (for `database/sql` with `lib/pq`)\n\n## Migration Library Support\n\n`pgdbtemplate` supports multiple migration libraries through dedicated adapters:\n\n- **[goose](https://github.com/andrei-polukhin/pgdbtemplate-goose)** - Popular migration tool with broad feature set\n- **[golang-migrate](https://github.com/andrei-polukhin/pgdbtemplate-golang-migrate)** - Widely-used CLI and library\n- **[Atlas](https://github.com/andrei-polukhin/pgdbtemplate-atlas)** - Modern declarative migration tool\n\nOr use the built-in file-based migration runner for simple SQL migrations.\n\n```bash\n# Choose your migration adapter\ngo get github.com/andrei-polukhin/pgdbtemplate-goose\n# or\ngo get github.com/andrei-polukhin/pgdbtemplate-golang-migrate\n# or\ngo get github.com/andrei-polukhin/pgdbtemplate-atlas\n```\n\n## Quick Start\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/andrei-polukhin/pgdbtemplate\"\n\t\"github.com/andrei-polukhin/pgdbtemplate-pgx\"\n)\n\nfunc main() {\n\t// Create a connection provider with pooling options.\n\tconnStringFunc := func(dbName string) string {\n\t\treturn fmt.Sprintf(\"postgres://user:pass@localhost/%s\", dbName)\n\t}\n\tprovider := pgdbtemplatepgx.NewConnectionProvider(connStringFunc)\n\tdefer provider.Close() // Close all connection pools.\n\n\t// Create migration runner.\n\tmigrationRunner := pgdbtemplate.NewFileMigrationRunner(\n\t\t[]string{\"./migrations\"}, \n\t\tpgdbtemplate.AlphabeticalMigrationFilesSorting,\n\t)\n\n\t// Create template manager.\n\tconfig := pgdbtemplate.Config{\n\t\tConnectionProvider: provider,\n\t\tMigrationRunner:    migrationRunner,\n\t}\n\n\ttm, err := pgdbtemplate.NewTemplateManager(config)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// Initialize template with migrations.\n\tctx := context.Background()\n\tif err := tm.Initialize(ctx); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// Create test database (fast!).\n\ttestDB, testDBName, err := tm.CreateTestDatabase(ctx)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer testDB.Close()\n\tdefer tm.DropTestDatabase(ctx, testDBName)\n\n\t// Use testDB for testing...\n\tlog.Printf(\"Test database %s ready!\", testDBName)\n}\n```\n\n## Usage Examples\n\n### 1. Pgx Testing with Existing PostgreSQL\n\n```go\npackage myapp_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/andrei-polukhin/pgdbtemplate\"\n\t\"github.com/andrei-polukhin/pgdbtemplate-pgx\"\n)\n\nvar templateManager *pgdbtemplate.TemplateManager\nvar provider *pgdbtemplatepgx.ConnectionProvider\n\nfunc TestMain(m *testing.M) {\n\t// Setup template manager once.\n\tctx := context.Background()\n\tif err := setupPgxTemplateManager(ctx); err != nil {\n\t\tlog.Fatalf(\"failed to setup template manager: %v\", err)\n\t}\n\n\t// Run tests.\n\tcode := m.Run()\n\n\t// Cleanup.\n\ttemplateManager.Cleanup(ctx)\n\tprovider.Close()\n\n\tos.Exit(code)\n}\n\nfunc setupPgxTemplateManager(ctx context.Context) error {\n\tbaseConnString := \"postgres://postgres:password@localhost:5432/postgres?sslmode=disable\"\n\n\t// Create pgx connection provider with connection pooling.\n\tconnStringFunc := func(dbName string) string {\n\t\treturn pgdbtemplate.ReplaceDatabaseInConnectionString(baseConnString, dbName)\n\t}\n\t\n\t// Configure connection pool settings using options.\n\tprovider = pgdbtemplatepgx.NewConnectionProvider(\n\t\tconnStringFunc,\n\t\tpgdbtemplatepgx.WithMaxConns(10),\n\t\tpgdbtemplatepgx.WithMinConns(2),\n\t)\n\n\t// Create migration runner.\n\tmigrationRunner := pgdbtemplate.NewFileMigrationRunner(\n\t\t[]string{\"./testdata/migrations\"},\n\t\tpgdbtemplate.AlphabeticalMigrationFilesSorting,\n\t)\n\n\t// Configure template manager.\n\tconfig := pgdbtemplate.Config{\n\t\tConnectionProvider: provider,\n\t\tMigrationRunner:    migrationRunner,\n\t}\n\n\tvar err error\n\ttemplateManager, err = pgdbtemplate.NewTemplateManager(config)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to create template manager: %w\", err)\n\t}\n\n\t// Initialize template database with migrations.\n\tif err = templateManager.Initialize(ctx); err != nil {\n\t\treturn fmt.Errorf(\"failed to initialize template: %w\", err)\n\t}\n\treturn nil\n}\n\n// Individual test function using pgx.\nfunc TestUserRepositoryPgx(t *testing.T) {\n\tctx := context.Background()\n\n\t// Create isolated test database with pgx connection.\n\ttestConn, testDBName, err := templateManager.CreateTestDatabase(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer testConn.Close()\n\tdefer templateManager.DropTestDatabase(ctx, testDBName)\n\n\t// Use pgx-specific features like native PostgreSQL types.\n\t_, err = testConn.ExecContext(ctx, \n\t\t\"INSERT INTO users (name, email, created_at) VALUES ($1, $2, NOW())\", \n\t\t\"Jane Doe\", \"jane@example.com\")\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tvar count int\n\terr = testConn.QueryRowContext(ctx, \"SELECT COUNT(*) FROM users\").Scan(\u0026count)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif count != 1 {\n\t\tt.Errorf(\"Expected 1 user, got %d\", count)\n\t}\n}\n```\n\n### 2. Integration with `testcontainers-go`\n\n```go\npackage myapp_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/andrei-polukhin/pgdbtemplate\"\n\t\"github.com/andrei-polukhin/pgdbtemplate-pq\"\n\t\"github.com/testcontainers/testcontainers-go\"\n\t\"github.com/testcontainers/testcontainers-go/modules/postgres\"\n\t\"github.com/testcontainers/testcontainers-go/wait\"\n)\n\nvar (\n\tpgContainer     *postgres.PostgresContainer\n\ttemplateManager *pgdbtemplate.TemplateManager\n)\n\nfunc TestMain(m *testing.M) {\n\tctx := context.Background()\n\n\t// Start PostgreSQL container.\n\tif err := setupPostgresContainer(ctx); err != nil {\n\t\tlog.Fatalf(\"failed to setup postgres container: %v\", err)\n\t}\n\n\t// Setup template manager.\n\tif err := setupTemplateManagerWithContainer(ctx); err != nil {\n\t\tlog.Fatalf(\"failed to setup template manager: %v\", err)\n\t}\n\n\t// Run tests.\n\tcode := m.Run()\n\n\t// Cleanup.\n\ttemplateManager.Cleanup(ctx)\n\tpgContainer.Terminate(ctx)\n\n\tos.Exit(code)\n}\n\nfunc setupPostgresContainer(ctx context.Context) error {\n\tvar err error\n\tpgContainer, err = postgres.RunContainer(ctx,\n\t\ttestcontainers.WithImage(\"postgres:15\"),\n\t\tpostgres.WithDatabase(\"testdb\"),\n\t\tpostgres.WithUsername(\"testuser\"),\n\t\tpostgres.WithPassword(\"testpass\"),\n\t\ttestcontainers.WithWaitStrategy(\n\t\t\twait.ForLog(\"database system is ready to accept connections\").\n\t\t\tWithOccurrence(2).\n\t\t\tWithStartupTimeout(5*time.Second),\n\t\t),\n\t)\n\treturn err\n}\n\nfunc setupTemplateManagerWithContainer(ctx context.Context) error {\n\t// Get connection details from container.\n\tconnStr, err := pgContainer.ConnectionString(ctx, \"sslmode=disable\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Create connection provider using the built-in standard provider.\n\tconnStringFunc := func(dbName string) string {\n\t\t// Replace the database name in the connection string.\n\t\treturn pgdbtemplate.ReplaceDatabaseInConnectionString(connStr, dbName)\n\t}\n\tprovider := pgdbtemplatepq.NewConnectionProvider(connStringFunc)\n\n\t// Create migration runner.\n\tmigrationRunner := pgdbtemplate.NewFileMigrationRunner(\n\t\t[]string{\"./testdata/migrations\"},\n\t\tpgdbtemplate.AlphabeticalMigrationFilesSorting,\n\t)\n\n\t// Configure template manager.\n\tconfig := pgdbtemplate.Config{\n\t\tConnectionProvider: provider,\n\t\tMigrationRunner:    migrationRunner,\n\t\tAdminDBName:        \"testdb\", // Use the container's default database.\n\t}\n\n\ttemplateManager, err = pgdbtemplate.NewTemplateManager(config)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Initialize template database.\n\treturn templateManager.Initialize(ctx)\n}\n\n// Example test using testcontainers.\nfunc TestUserServiceWithContainer(t *testing.T) {\n\tctx := context.Background()\n\n\t// Create test database from template.\n\ttestDB, testDBName, err := templateManager.CreateTestDatabase(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tdefer testDB.Close()\n\tdefer templateManager.DropTestDatabase(ctx, testDBName)\n\n\t// Test your service with the isolated database.\n\tuserService := NewUserService(testDB)\n\n\tuser := \u0026User{Name: \"Alice\", Email: \"alice@example.com\"}\n\tif err := userService.Create(ctx, user); err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tusers, err := userService.List(ctx)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\n\tif len(users) != 1 {\n\t\tt.Errorf(\"Expected 1 user, got %d\", len(users))\n\t}\n}\n```\n\n## Advanced Cases\n\nFor advanced usage scenarios including custom connection providers and\ncustom migration runners, see **[ADVANCED.md](docs/ADVANCED.md)**.\n\n## Performance Benefits\n\nUsing template databases provides significant performance improvements over\ntraditional database creation and migration:\n\n### Real Benchmark Results (Apple M4 Pro)\n- **Traditional approach**: ~28.9–43.1ms per database (scales with schema complexity)\n- **Template approach**: **~28.2–28.8ms per database** (consistent regardless of complexity)\n- **Performance gain increases with schema complexity**: 1.03x → 1.43x → 1.50x faster\n- **Superior concurrency**: Thread-safe operations with ~86.5 ops/sec vs ~78.5 ops/sec traditional\n- **Memory efficient**: 17% less memory usage per operation\n\n### Schema Complexity Impact\n| Schema Size | Traditional | Template | Performance Gain |\n|-------------|-------------|----------|------------------|\n| 1 Table     | ~28.9ms    | ~28.2ms  | **1.03x faster** |\n| 3 Tables    | ~39.5ms    | ~27.6ms  | **1.43x faster** |\n| 5 Tables    | ~43.1ms    | ~28.8ms  | **1.50x faster** |\n\n### Scaling Benefits  \n| Test Databases | Traditional | Template | Time Saved |\n|---|---|---|---|\n| 20 DBs | 906.8ms (45.3ms/db) | 613.8ms (29.4ms/db) | **32% faster** |\n| 50 DBs | 2.29s (45.8ms/db) | 1.53s (29.8ms/db) | **33% faster** |\n| 200 DBs | 9.21s (46.0ms/db) | 5.84s (29.2ms/db) | **37% faster** |\n| 500 DBs | 22.31s (44.6ms/db) | 14.82s (29.6ms/db) | **34% faster** |\n\nFor comprehensive benchmark analysis, methodology, and detailed results,\nsee **[BENCHMARKS.md](docs/BENCHMARKS.md)**.\n\n## Migration Files Structure\n\nOrganize your migration files for automatic alphabetical ordering:\n\n```\nmigrations/\n├── 001_create_users_table.sql\n├── 002_create_posts_table.sql\n├── 003_add_user_posts_relation.sql\n└── 004_add_indexes.sql\n```\n\n## Thread Safety\n\nThe library is **fully thread-safe** and designed for concurrent use\nin production test suites:\n\n### Concurrency Guarantees\n- **Template initialization**: Protected by mutex - safe to call from multiple goroutines\n- **Database creation**: Each `CreateTestDatabase()` call is fully isolated\n- **Unique naming**: Automatic collision-free database naming\n  with timestamps and atomic counters\n- **Parallel testing**: Safe for `go test -parallel N` with any parallelism level\n\nThe template manager internally handles all synchronization,\nmaking it safe to use in any concurrent testing scenario.\n\n## Best Practices\n\n1. **Initialize once**: Set up the template manager in `TestMain()`\n\n2. **Cleanup**: Always call `DropTestDatabase()` for each created test database,\n   and `Cleanup()` once at the end:\n   - `DropTestDatabase(dbName)`: Drops a specific test database and removes it from tracking.\n   - `Cleanup()`: Drops all remaining tracked test databases AND the template database\n   (call once in `TestMain()`).\n\n3. **Isolation**: Each test should use its own database to prevent interference\nbetween tests.\n\n4. **Naming**: Let `pgdbtemplate` auto-generate unique database names\n(recommended for most cases):\n   ```go\n   // Good: Auto-generated name (recommended).\n   testDB, testDBName, err := templateManager.CreateTestDatabase(ctx)\n   \n   // Advanced: Custom name only when needed for debugging.\n   testDB, testDBName, err := templateManager.CreateTestDatabase(ctx, \"my_test_debug\")\n   ```\n\n5. **Migration order**: Use numbered prefixes for deterministic ordering.\n\n## Requirements\n\n- PostgreSQL 9.5+ (for template database support)\n- Go 1.20+\n\n## Related Projects\n\n### Connection Providers\n- [pgdbtemplate-pq](https://github.com/andrei-polukhin/pgdbtemplate-pq) - `lib/pq` (`database/sql`) connection provider\n- [pgdbtemplate-pgx](https://github.com/andrei-polukhin/pgdbtemplate-pgx) - `pgx/v5` connection provider with pooling\n\n### Migration Adapters\n- [pgdbtemplate-goose](https://github.com/andrei-polukhin/pgdbtemplate-goose) - [goose](https://github.com/pressly/goose) migration adapter\n- [pgdbtemplate-golang-migrate](https://github.com/andrei-polukhin/pgdbtemplate-golang-migrate) - [golang-migrate](https://github.com/golang-migrate/migrate) adapter\n- [pgdbtemplate-atlas](https://github.com/andrei-polukhin/pgdbtemplate-atlas) - [Atlas](https://atlasgo.io/) migration adapter\n\n## Contributors\n\nWe appreciate all the contributors who have helped make this project better!\nPlease see [CONTRIBUTORS.md](docs/CONTRIBUTORS.md) for the full list.\n\n## Contributing\n\nContributions are welcome! Please see [CONTRIBUTING.md](docs/CONTRIBUTING.md) for\nguidelines.\n\n## Security\n\nIf you discover a security vulnerability, please report it responsibly.\nSee [SECURITY.md](docs/SECURITY.md) for our security policy and reporting process.\n\n## Disclaimer\n\nThis is a personal project developed in my own time.\nIt is not affiliated with or endorsed by any company.\n\n## License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandrei-polukhin%2Fpgdbtemplate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandrei-polukhin%2Fpgdbtemplate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandrei-polukhin%2Fpgdbtemplate/lists"}