{"id":19224332,"url":"https://github.com/snowplow/sql-runner","last_synced_at":"2025-04-20T23:32:02.988Z","repository":{"id":25288736,"uuid":"28714735","full_name":"snowplow/sql-runner","owner":"snowplow","description":"Run templatable playbooks of SQL scripts in series and parallel on Redshift, PostgreSQL, BigQuery and Snowflake","archived":false,"fork":false,"pushed_at":"2023-05-09T23:26:31.000Z","size":1584,"stargazers_count":81,"open_issues_count":50,"forks_count":15,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-04-01T07:04:55.843Z","etag":null,"topics":["bigquery","postgresql","redshift","snowflake","snowplow","sql-runner"],"latest_commit_sha":null,"homepage":"http://snowplowanalytics.com","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snowplow.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-01-02T13:57:37.000Z","updated_at":"2024-10-01T10:11:58.000Z","dependencies_parsed_at":"2024-06-20T07:43:03.552Z","dependency_job_id":null,"html_url":"https://github.com/snowplow/sql-runner","commit_stats":null,"previous_names":[],"tags_count":42,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowplow%2Fsql-runner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowplow%2Fsql-runner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowplow%2Fsql-runner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowplow%2Fsql-runner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snowplow","download_url":"https://codeload.github.com/snowplow/sql-runner/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249977433,"owners_count":21354859,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","postgresql","redshift","snowflake","snowplow","sql-runner"],"created_at":"2024-11-09T15:11:22.165Z","updated_at":"2025-04-20T23:32:02.465Z","avatar_url":"https://github.com/snowplow.png","language":"Go","readme":"# SQL Runner\n\n[![Build Status][gh-actions-image]][gh-actions] [![Coveralls][coveralls-image]][coveralls] [![Go Report Card][goreport-image]][goreport] [![Release][release-image]][releases] [![License][license-image]][license]\n\n## Overview\n\nRun playbooks of SQL scripts in series and parallel on Snowflake DB, Amazon Redshift and PostgreSQL.\n\nUsed with **[Snowplow][snowplow]** for scheduled SQL-based transformations of event stream data.\n\n|  **[Setup Guide][setup-guide]**     | **[User Guide][user-guide]**     |\n|:--------------------------------------:|:-----------------------------------------:|\n|  [![i1][setup-image]][setup-guide] | [![i2][user-image]][user-guide] |\n\n## Quick start\n\nAssuming [go][go-url], [docker][docker-url] and [docker-compose][docker-compose-url] are installed:\n\n```bash\n host\u003e git clone https://github.com/snowplow/sql-runner\n host\u003e cd sql-runner\n host\u003e make setup-up    # Launches Consul + Postgres for testing\n host\u003e make             # Builds sql-runner binaries\n host\u003e make test        # Runs unit tests\n\n # DISTRO specifies which binary you want to run integration tests with\n host\u003e DISTRO=darwin make integration\n```\n\n_Note_: You will need to ensure that `~/go/bin` is on your PATH for `gox` to work - the underlying tool that we use for building the binaries.\n\nWhen you are done with testing simply execute `make setup-down` to terminate the docker-compose stack.\n\nTo reset the testing resources execute `make setup-reset` which will rebuild the docker containers.  This can be useful if the state of these systems gets out of sync with what the tests expect.\n\nTo remove all build files:\n\n```bash\nguest\u003e make clean\n```\n\nTo format the golang code in the source directory:\n\n```bash\nguest\u003e make format\n```\n\n**Note:** Always run `make format` before submitting any code.\n\n**Note:** The `make test` command also generates a code coverage file which can be found at `build/coverage/coverage.html`.\n\n## How to use?\n\nFirst either compile the binary from source using the above `make` command or download the published Binary directly from the GitHub release:\n\n* [Darwin (macOS)](https://github.com/snowplow/sql-runner/releases/download/0.10.1/sql_runner_0.10.1_darwin_amd64.zip)\n* [Linux](https://github.com/snowplow/sql-runner/releases/download/0.10.1/sql_runner_0.10.1_linux_amd64.zip)\n* [Windows](https://github.com/snowplow/sql-runner/releases/download/0.10.1/sql_runner_0.10.1_windows_amd64.zip)\n\n### CLI Output\n\n```bash\nsql-runner version: 0.10.1\nRun playbooks of SQL scripts in series and parallel on Redshift and Postgres\nUsage:\n  -checkLock string\n    \tChecks whether the lockfile already exists\n  -consul string\n    \tThe address of a consul server with playbooks and SQL files stored in KV pairs\n  -consulOnlyForLock\n    \tWill read playbooks locally, but use Consul for locking.\n  -deleteLock string\n    \tWill attempt to delete a lockfile if it exists\n  -dryRun\n    \tRuns through a playbook without executing any of the SQL\n  -fillTemplates\n    \tWill print all queries after templates are filled\n  -fromStep string\n    \tStarts from a given step defined in your playbook\n  -help\n    \tShows this message\n  -lock string\n    \tOptional argument which checks and sets a lockfile to ensure this run is a singleton. Deletes lock on run completing successfully\n  -playbook string\n    \tPlaybook of SQL scripts to execute\n  -runQuery string\n    \tWill run a single query in the playbook\n  -showQueryOutput\n    \tWill print all output from queries\n  -softLock string\n    \tOptional argument, like '-lock' but the lockfile will be deleted even if the run fails\n  -sqlroot string\n    \tAbsolute path to SQL scripts. Use PLAYBOOK, BINARY and PLAYBOOK_CHILD for those respective paths (default \"PLAYBOOK\")\n  -var value\n    \tVariables to be passed to the playbook, in the key=value format\n  -version\n    \tShows the program version\n```\n\n## Copyright and license\n\nSQL Runner is copyright 2015-2022 Snowplow Analytics Ltd.\n\nLicensed under the **[Apache License, Version 2.0][license]** (the \"License\");\nyou may not use this software except in compliance with the License.\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n[go-url]: https://golang.org/doc/install\n[docker-url]: https://docs.docker.com/get-docker/\n[docker-compose-url]: https://docs.docker.com/compose/install/\n\n[gh-actions]: https://github.com/snowplow/sql-runner/actions\n[gh-actions-image]: https://github.com/snowplow/sql-runner/workflows/Build/badge.svg?branch=master\n\n[release-image]: https://img.shields.io/github/v/release/snowplow/sql-runner\n[releases]: https://github.com/snowplow/sql-runner/releases\n\n[license-image]: http://img.shields.io/badge/license-Apache--2-blue.svg?style=flat\n[license]: http://www.apache.org/licenses/LICENSE-2.0\n\n[coveralls-image]: https://coveralls.io/repos/github/snowplow/sql-runner/badge.svg?branch=master\n[coveralls]: https://coveralls.io/github/snowplow/sql-runner?branch=master\n\n[goreport]: https://goreportcard.com/report/github.com/snowplow/sql-runner\n[goreport-image]: https://goreportcard.com/badge/github.com/snowplow/sql-runner\n\n[snowplow]: https://github.com/snowplow/snowplow\n\n[setup-guide]: https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-sql-runner/\n[user-guide]: https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-sql-runner/#user-guide\n\n[setup-image]:  https://d3i6fms1cm1j0i.cloudfront.net/github/images/setup.png\n[user-image]: https://d3i6fms1cm1j0i.cloudfront.net/github/images/techdocs.png\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnowplow%2Fsql-runner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnowplow%2Fsql-runner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnowplow%2Fsql-runner/lists"}