{"id":17826382,"url":"https://github.com/arsham/dbtools","last_synced_at":"2025-03-18T23:30:51.458Z","repository":{"id":42023493,"uuid":"194940207","full_name":"arsham/dbtools","owner":"arsham","description":"Go db helpers library for using in production code and tests.","archived":false,"fork":false,"pushed_at":"2024-06-19T08:20:59.000Z","size":665,"stargazers_count":4,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-16T15:48:24.946Z","etag":null,"topics":["database","go","golang","library","pgx","sqlmock-helpers","testing","transaction"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/arsham/dbtools","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arsham.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-02T21:57:18.000Z","updated_at":"2024-06-19T08:18:20.000Z","dependencies_parsed_at":"2024-06-17T21:52:36.099Z","dependency_job_id":"eaa5311c-ce0a-489f-9c05-2b5f4b64e72a","html_url":"https://github.com/arsham/dbtools","commit_stats":{"total_commits":38,"total_committers":1,"mean_commits":38.0,"dds":0.0,"last_synced_commit":"20b4de9210143447971c9bac498ee191467baaff"},"previous_names":["arsham/dbtesting"],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arsham%2Fdbtools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arsham%2Fdbtools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arsham%2Fdbtools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arsham%2Fdbtools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arsham","download_url":"https://codeload.github.com/arsham/dbtools/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244325180,"owners_count":20435054,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","go","golang","library","pgx","sqlmock-helpers","testing","transaction"],"created_at":"2024-10-27T18:45:12.198Z","updated_at":"2025-03-18T23:30:51.221Z","avatar_url":"https://github.com/arsham.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dbtools\n\n[![PkgGoDev](https://pkg.go.dev/badge/github.com/arsham/dbtools)](https://pkg.go.dev/github.com/arsham/dbtools)\n![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/arsham/dbtools)\n[![Build Status](https://github.com/arsham/dbtools/actions/workflows/go.yml/badge.svg)](https://github.com/arsham/dbtools/actions/workflows/go.yml)\n[![Coverage Status](https://codecov.io/gh/arsham/dbtools/branch/master/graph/badge.svg)](https://codecov.io/gh/arsham/dbtools)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Go Report Card](https://goreportcard.com/badge/github.com/arsham/dbtools)](https://goreportcard.com/report/github.com/arsham/dbtools)\n\nThis library contains concurrent safe helpers for retrying transactions until\nthey succeed and handles errors in a developer friendly way. There are helpers\nfor using with [go-sqlmock][go-sqlmock] in tests. There is also a `Mocha`\ninspired reporter for [spec BDD library][spec].\n\nThis library supports `Go \u003e= 1.22`. To use this library use this import path:\n\n```\ngithub.com/arsham/dbtools/v4\n```\n\nFor Go \u003e= 1.20 support use the v3:\n\n```\ngithub.com/arsham/dbtools/v3\n```\n\nFor older Go's support use the v2:\n\n```\ngithub.com/arsham/dbtools/v2\n```\n\n1. [PGX Transaction](#pgx-transaction)\n   - [Common Patterns](#common-patterns)\n2. [SQLMock Helpers](#sqlmock-helpers)\n   - [ValueRecorder](#valuerecorder)\n   - [OkValue](#okvalue)\n3. [Spec Reports](#spec-reports)\n   - [Usage](#usage)\n4. [Development](#development)\n5. [License](#license)\n\n## PGX Transaction\n\nThe `PGX` struct helps reducing the amount of code you put in the logic by\ntaking care of errors. For example instead of writing:\n\n```go\ntx, err := db.Begin(ctx)\nif err != nil {\n\treturn errors.Wrap(err, \"starting transaction\")\n}\nerr := firstQueryCall(tx)\nif err != nil {\n\te := errors.Wrap(tx.Rollback(ctx), \"rolling back transaction\")\n\treturn multierror.Append(err, e).ErrorOrNil()\n}\nerr := secondQueryCall(tx)\nif err != nil {\n\te := errors.Wrap(tx.Rollback(ctx), \"rolling back transaction\")\n\treturn multierror.Append(err, e).ErrorOrNil()\n}\nerr := thirdQueryCall(tx)\nif err != nil {\n\te := errors.Wrap(tx.Rollback(ctx), \"rolling back transaction\")\n\treturn multierror.Append(err, e).ErrorOrNil()\n}\n\nreturn errors.Wrap(tx.Commit(ctx), \"committing transaction\")\n\n```\n\nYou will write:\n\n```go\n// for using with pgx connections:\np, err := dbtools.NewPGX(conn)\n// handle the error!\nreturn p.Transaction(ctx, firstQueryCall, secondQueryCall, thirdQueryCall)\n```\n\nAt any point any of the callback functions return an error, the transaction is\nrolled-back, after the given delay the operation is retried in a new\ntransaction.\n\nYou may set the retry count, delays, and the delay method by passing\n`dbtools.ConfigFunc` helpers to the constructor. If you don't pass any config,\nthe `Transaction` method will run only once.\n\nYou can prematurely stop retrying by returning a `*retry.StopError` error:\n\n```go\nerr = p.Transaction(ctx, func(tx pgx.Tx) error {\n\t_, err := tx.Exec(ctx, query)\n\treturn \u0026retry.StopError{Err: err}\n})\n```\n\nSee [retry][retry] library for more information.\n\nThe callback functions should be of `func(pgx.Tx) error` type. To try up to 20\ntime until your queries succeed:\n\n```go\n// conn is a *pgxpool.Pool instance\np, err := dbtools.NewPGX(conn, dbtools.Retry(20))\n// handle the error\nerr = p.Transaction(ctx, func(tx pgx.Tx) error {\n\t// use tx to run your queries\n\treturn someErr\n}, func(tx pgx.Tx) error {\n\treturn someErr\n}, func(tx pgx.Tx) error {\n\treturn someErr\n\t// add more callbacks if required.\n})\n// handle the error!\n```\n\n### Common Patterns\n\nStop retrying when the row is not found:\n\n```go\nerr := retrier.Do(func() error {\n\tconst query = `SELECT foo FROM bar WHERE id = $1::int`\n\terr := conn.QueryRow(ctx, query, msgID).Scan(\u0026foo)\n\tif errors.Is(err, pgx.ErrNoRows) {\n\t\treturn \u0026retry.StopError{Err: ErrFooNotFound}\n\t}\n\treturn errors.Wrap(err, \"quering database\")\n})\n```\n\nStop retrying when there are integrity errors:\n\n```go\n// integrityCheckErr returns a *retry.StopError wrapping the err with the msg\n// if the query causes integrity constraint violation error. You should use\n// this check to stop the retry mechanism, otherwise the transaction repeats.\nfunc integrityCheckErr(err error, msg string) error {\n    var v *pgconn.PgError\n    if errors.As(err, \u0026v) \u0026\u0026 isIntegrityConstraintViolation(v.Code) {\n        return \u0026retry.StopError{Err: errors.Wrap(err, msg)}\n    }\n    return errors.Wrap(err, msg)\n}\n\nfunc isIntegrityConstraintViolation(code string) bool {\n    switch code {\n    case pgerrcode.IntegrityConstraintViolation,\n        pgerrcode.RestrictViolation,\n        pgerrcode.NotNullViolation,\n        pgerrcode.ForeignKeyViolation,\n        pgerrcode.CheckViolation,\n        pgerrcode.ExclusionViolation:\n        return true\n    }\n    return false\n}\n\nerr := p.Transaction(ctx, func(tx pgx.Tx) error {\n    const query = `INSERT INTO foo (bar) VALUES ($1::text)`\n    err := tx.Exec(ctx, query, name)\n    return integrityCheckErr(err, \"creating new record\")\n}, func(tx pgx.Tx) error {\n    const query = `UPDATE baz SET updated_at=NOW()::timestamptz WHERE id = $1::int`\n    _, err := tx.Exec(ctx, query, msgID)\n    return err\n})\n```\n\nThis is not a part of the `dbtools` library, but it deserves a mention. Here is\na common pattern for querying for multiple rows:\n\n```go\nresult := make([]Result, 0, expectedTotal)\nerr := retrier.Do(func() error {\n\trows, err := r.pool.Query(ctx, query, args...)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"making query\")\n\t}\n\n\tdefer rows.Close()\n\n\t// make sure you reset the slice, otherwise in the next retry it adds the\n\t// same data to the slice again.\n\tresult = result[:0]\n\tfor rows.Next() {\n\t\tvar doc Result\n\t\terr := rows.Scan(\u0026doc.A, \u0026doc.B)\n\t\tif err != nil {\n\t\t\treturn errors.Wrap(err, \"scanning rows\")\n\t\t}\n\t\tresult = append(result, doc)\n\t}\n\n\treturn errors.Wrap(rows.Err(), \"row error\")\n})\n// handle the error!\n```\n\n## SQLMock Helpers\n\nThere a couple of helpers for using with [go-sqlmock][go-sqlmock] test cases for\ncases that values are random but it is important to check the values passed in\nqueries.\n\n### ValueRecorder\n\nIf you have an value and use it in multiple queries, and you want to\nmake sure the queries are passed with correct values, you can use the\n`ValueRecorder`. For example UUIDs, time and random values.\n\nFor instance if the first query generates a random number but it is essential to\nuse the same value on next queries:\n\n```go\nimport \"database/sql\"\n\nfunc TestFoo(t *testing.T) {\n\t// ...\n\t// assume num has been generated randomly\n\tnum := 666\n\t_, err := tx.ExecContext(ctx, \"INSERT INTO life (value) VALUE ($1)\", num)\n\t// error check\n\t_, err = tx.ExecContext(ctx, \"INSERT INTO reality (value) VALUE ($1)\", num)\n\t// error check\n\t_, err = tx.ExecContext(ctx, \"INSERT INTO everywhere (value) VALUE ($1)\", num)\n\t// error check\n}\n```\n\nYour tests can be checked easily like this:\n\n```go\nimport (\n\t\"github.com/DATA-DOG/go-sqlmock\"\n\t\"github.com/arsham/dbtools/v4/dbtesting\"\n)\n\nfunc TestFoo(t *testing.T) {\n\t// ...\n\trec := dbtesting.NewValueRecorder()\n\tmock.ExpectExec(\"INSERT INTO life .+\").\n\t\tWithArgs(rec.Record(\"truth\")).\n\t\tWillReturnResult(sqlmock.NewResult(1, 1))\n\tmock.ExpectExec(\"INSERT INTO reality .+\").\n\t\tWithArgs(rec.For(\"truth\")).\n\t\tWillReturnResult(sqlmock.NewResult(1, 1))\n\tmock.ExpectExec(\"INSERT INTO everywhere .+\").\n\t\tWithArgs(rec.For(\"truth\")).\n\t\tWillReturnResult(sqlmock.NewResult(1, 1))\n}\n```\n\nRecorded values can be retrieved by casting to their types:\n\n```go\nrec.Value(\"true\").(string)\n```\n\nThere are two rules for using the `ValueRecorder`:\n\n1. You can only record for a value once.\n2. You should record a value before you call `For` or `Value`.\n\nIt will panic if these requirements are not met.\n\n### OkValue\n\nIf you are only interested in checking some arguments passed to the Exec/Query\nfunctions and you don't want to check everything (maybe because thy are not\nrelevant to the current test), you can use `OkValue`.\n\n```go\nimport (\n    \"github.com/arsham/dbtools/v4/dbtesting\"\n    \"github.com/DATA-DOG/go-sqlmock\"\n)\n\nok := dbtesting.OkValue\nmock.ExpectExec(\"INSERT INTO life .+\").\n    WithArgs(\n        ok,\n        ok,\n        ok,\n        \"important value\"\n        ok,\n        ok,\n        ok,\n    )\n```\n\n## Spec Reports\n\n`Mocha` is a reporter for printing Mocha inspired reports when using\n[spec BDD library][spec].\n\n### Usage\n\n```go\nimport \"github.com/arsham/dbtools/v4/dbtesting\"\n\nfunc TestFoo(t *testing.T) {\n\tspec.Run(t, \"Foo\", func(t *testing.T, when spec.G, it spec.S) {\n\t\t// ...\n\t}, spec.Report(\u0026dbtesting.Mocha{}))\n}\n\n```\n\nYou can set an `io.Writer` to `Mocha.Out` to redirect the output, otherwise it\nprints to the `os.Stdout`.\n\n## Development\n\nRun the `tests` target for watching file changes and running tests:\n\n```bash\nmake tests\n```\n\nYou can pass flags as such:\n\n```bash\nmake tests flags=\"-race -v -count=5\"\n```\n\nYou need to run the `dependencies` target for installing [reflex][reflex] task\nrunner:\n\n```bash\nmake dependencies\n```\n\n## License\n\nUse of this source code is governed by the Apache 2.0 license. License can be\nfound in the [LICENSE](./LICENSE) file.\n\n[retry]: https://github.com/arsham/retry\n[pgx]: https://github.com/jackc/pgx\n[go-sqlmock]: https://github.com/DATA-DOG/go-sqlmock\n[spec]: https://github.com/sclevine/spec\n[reflex]: https://github.com/cespare/reflex\n\n\u003c!--\nvim: foldlevel=1\n--\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farsham%2Fdbtools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farsham%2Fdbtools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farsham%2Fdbtools/lists"}