{"id":15060782,"url":"https://github.com/tufin/espresso","last_synced_at":"2025-10-04T19:31:35.642Z","repository":{"id":37028727,"uuid":"403085433","full_name":"Tufin/espresso","owner":"Tufin","description":"A framework for writing testable BigQuery queries","archived":true,"fork":false,"pushed_at":"2023-04-20T12:59:43.000Z","size":654,"stargazers_count":8,"open_issues_count":4,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-09-29T06:21:37.311Z","etag":null,"topics":["bigquery","sql","testing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Tufin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-04T15:01:57.000Z","updated_at":"2023-04-24T08:21:11.000Z","dependencies_parsed_at":"2024-06-19T19:06:59.389Z","dependency_job_id":"1cfdd506-45cf-439d-8dc7-db78ce4156af","html_url":"https://github.com/Tufin/espresso","commit_stats":{"total_commits":159,"total_committers":2,"mean_commits":79.5,"dds":0.4968553459119497,"last_synced_commit":"4303d9254f1544a63eb534236e57978a5733c482"},"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"purl":"pkg:github/Tufin/espresso","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tufin%2Fespresso","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tufin%2Fespresso/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tufin%2Fespresso/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tufin%2Fespresso/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Tufin","download_url":"https://codeload.github.com/Tufin/espresso/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tufin%2Fespresso/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278365883,"owners_count":25975087,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","sql","testing"],"created_at":"2024-09-24T23:04:24.556Z","updated_at":"2025-10-04T19:31:35.348Z","avatar_url":"https://github.com/Tufin.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![CI](https://github.com/Tufin/espresso/workflows/go/badge.svg)](https://github.com/Tufin/espresso/actions)\n[![codecov](https://codecov.io/gh/tufin/espresso/branch/main/graph/badge.svg?token=4neEgts50n)](https://codecov.io/gh/tufin/espresso)\n[![Go Report Card](https://goreportcard.com/badge/github.com/tufin/espresso)](https://goreportcard.com/report/github.com/tufin/espresso)\n[![GoDoc](https://godoc.org/github.com/tufin/espresso?status.svg)](https://godoc.org/github.com/tufin/espresso)\n[![Docker Image Version](https://img.shields.io/docker/v/tufin/espresso?sort=semver)](https://hub.docker.com/r/tufin/espresso/tags)\n\n# espresso - a framework for testing BigQuery queries\n\n## Goals\n- Componentization: compose complex queries from smaller, reusable components\n- Test driven development: write tests for each query and run them like unit tests (except for the fact that they make calls to BigQuery)\n- Data as code: input and required output for tests can be defined as part of the code (as well as in real database tables)\n- No new languages to learn\n- Write tests in your own development stack: programming language, IDE and CI/CD pipeline\n\n## Writing Your Own SQL Tests\n1. Write an SQL query using Go Text Template notation, for example:\n   ```sql\n   {{ define \"fruit\" }}\n \n   WITH base AS (\n       {{ .Base }}\n   )\n   SELECT\n       fruit\n   FROM base\n\n   {{ end }}\n   ```\n\n   The query may contain parameters, like {{ .Base }}\n2. Add additional SQL queries to pass as arguments to the main query, for example:  \n   ```sql\n   {{ define \"base\" }}\n\n   SELECT\n       \"orange\" AS fruit\n   UNION ALL\n   SELECT\n       \"apple\"\n\n   {{ end }}\n   ```\n   \n3. Write your expected result query, for example:\n   ```sql\n   {{ define \"fruit_result\" }}\n\n   SELECT\n       \"orange\" AS fruit\n   UNION ALL\n   SELECT\n       \"apple\"\n\n   {{ end }}\n   ```\n4. Create a query definition file describing your query and one or more tests, for example:\n   ```yaml\n   Name: fruit\n   Requires:\n   - Base\n   Tests:\n     Test1:\n       Args:\n       - Name: Base\n         Source: base\n       Result:\n         Source: fruit_result\n   ```\n5. Put all files together in a directory\n\n## Query Definitions\nThe query definition file specifies how to construct an SQL query from the SQL templates.  \nA definition file contains Tests, each specifying how to construct an SQL query and an optional expected result.  \nTo create a query from its components, espresso looks for an SQL template with the same name as the definition file itself, and then executes it with the given Args.  \nEach Arg must have a Name field that corresponds to a required argument in the SQL template and one of the following fields:\n1. Source - Another SQL template which will be executed and injected into the containing query.\n2. Table - a table name which will be combined with the Google project and BigQuery dataset (defined in Shot) and injected into the containing query.\n3. Const - a string that will be injected into the containing query.\n\nSource may contain its own Args. If it doesn't, espresso will look for a corresponding template file with a test of the same name and parse the args from there.\n\nResult is an optional field specifiying the test's expected result.\nIt should have a Source field specifying an SQL template that will be executed to generate the expected result query.\nSource may contain Args as described above.\n\n## Access To BigQuery\nThe tests require access to BigQuery API. \nPlease set the following environment variables to grant espresso access to BigQuery:\n- `export GCLOUD_PROJECT_ID=\u003cyour GCP project ID\u003e`\n- `export BIGQUERY_KEY=\u003ca service account with permissions to use BigQuery\u003e`\n\n## Running Tests From The Command-line\n```bash\ngo build\n./espresso -dir=\"./shot/queries/fruit/\" -query=\"fruit\" -test=\"Test1\"\n```\n\n## Running Tests From Docker\n```bash\ndocker run --rm -t -e GCLOUD_PROJECT_ID=$GCLOUD_PROJECT_ID -e BIGQUERY_KEY=$BIGQUERY_KEY -v $(pwd)/shot:/shot:ro tufin/espresso -dir=\"/shot\" -query=\"fruit\" -test=\"Test1\"\n```\n\n## Running Tests From Golang\n1. Embed your tests directory\n2. Create an \"Espresso Shot\" and run it\n3. Use standard Go assertions to check the expected result against the actual output\n```go\nfunc TestEspressoShot_Filesystem(t *testing.T) {\n\tqueryValues, resultValues, err := shot.NewShotWithClient(env.GetGCPProjectID(), \"\", os.DirFS(\"./queries/fruit\")).RunTest(\"fruit\", \"Test1\", []bigquery.QueryParameter{}, \u0026map[string]bigquery.Value{})\n\trequire.NoError(t, err)\n\trequire.ElementsMatch(t, queryValues, resultValues)\n}\n```\n\nYou can also embed the SQL templates directory into the code:\n```go\n//go:embed queries/fruit\nvar templates embed.FS\n\nfunc TestEspressoShot_Embed(t *testing.T) {\n\tqueryValues, resultValues, err := shot.NewShotWithClient(env.GetGCPProjectID(), \"\", templates).RunTest(\"fruit\", \"Test1\", []bigquery.QueryParameter{}, \u0026map[string]bigquery.Value{})\n\trequire.NoError(t, err)\n\trequire.ElementsMatch(t, queryValues, resultValues)\n}\n```\n\n## Running Tests In Other Programming Languages\nCurrently only Go is supported.\nIf you'd like to contribute additional language support, please start a dicssussion.\n\n## Current Status\n- This is an initial proof-of-concept and request-for-comments\n- Please submit your feedback as pull requests, issues or discussions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftufin%2Fespresso","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftufin%2Fespresso","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftufin%2Fespresso/lists"}