{"id":37173424,"url":"https://github.com/jwbargsten/go-mssql-load","last_synced_at":"2026-01-14T20:16:22.484Z","repository":{"id":200272435,"uuid":"598112779","full_name":"jwbargsten/go-mssql-load","owner":"jwbargsten","description":"An alternative to the sqlcmd utility of Microsoft. Can load CSV in bulk. Great for CI/CD pipelines that run integration tests.","archived":false,"fork":false,"pushed_at":"2023-10-24T22:47:50.000Z","size":23,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-06-20T14:26:03.027Z","etag":null,"topics":["mssql","sqlcmd"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jwbargsten.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-06T12:32:17.000Z","updated_at":"2024-02-25T22:14:30.000Z","dependencies_parsed_at":"2023-10-16T10:50:46.444Z","dependency_job_id":"26bea3ba-5efe-4423-aee1-5f0bb3c0fe18","html_url":"https://github.com/jwbargsten/go-mssql-load","commit_stats":null,"previous_names":["jwbargsten/go-mssql-load"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/jwbargsten/go-mssql-load","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwbargsten%2Fgo-mssql-load","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwbargsten%2Fgo-mssql-load/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwbargsten%2Fgo-mssql-load/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwbargsten%2Fgo-mssql-load/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jwbargsten","download_url":"https://codeload.github.com/jwbargsten/go-mssql-load/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwbargsten%2Fgo-mssql-load/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28434422,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T18:57:19.464Z","status":"ssl_error","status_checked_at":"2026-01-14T18:52:48.501Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mssql","sqlcmd"],"created_at":"2026-01-14T20:16:21.730Z","updated_at":"2026-01-14T20:16:22.478Z","avatar_url":"https://github.com/jwbargsten.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# go-mssql-load - load sql scripts and test data into MSSQL databases\n\nThis is a small tool that serves two main purposes\n\n- help with populating a MSSQL db with schema and test data during integration testing.\n  This can be on your local machine or part of a CI/CD pipeline.\n- provide a simple way for (data) pipelines to set up the schema of a db where the data\n  is exported to.\n\n## Use Cases\n\n### Integration Testing\n\nThe [sqlcmd utility](https://learn.microsoft.com/en-us/sql/tools/sqlcmd/sqlcmd-utility)\nis usually the go-to solution for interacting with MSSQL databases. However, it can be\ndifficult to install and use, especially in the context of Apple M1 machines. That can\npose a problem to developers, because now two different ways of dealing with the\ndatabase exist. One in the CI/CD pipeline, that uses sqlcmd and one on the developer's\nlocal machine.\n\nTo make the DB setup process consistent, `go-mssql-load` can be used.\n\n### Set up a DB schema\n\nEvery Java developer will probably scream [FLYWAY](https://flywaydb.org/) or similar,\nbut let's assume that your group is more interdisciplinary, consisting of devs and data\nengineers. Having a simple CLI tool to setup up the DB can help with shared ownership of\nthe schema.\n\n### Bulk load test data\n\nYou can bulk load CSV data. This allows you to populate a test db with some or many\nrows. In particular for data engineering purposes it is a good way to load a bigger\nchunk of data for a more _realistic_ feeling.\n\n## Installation\n\nYou need to have go installed.\n\nTo install it directly:\n\n```console\ngo get github.com/jwbargsten/go-mssql-load\n```\n\nIf you want to install it from the repo, you can run\n\n```console\n$ go build\n```\n\nwhich will create the executable `./go-mssql-load` or install it into `$GOPATH/bin`\nusing\n\n```console\n$ go install\n```\n\n## Usage\n\nBefore you can use `go-mssql-load`, you have to have a mssql db running. The easiest way\nwould be to use one of the official Microsoft docker containers (Azure SQL edge is\nmostly compatible with mssql and it runs on M1 machines):\n\n```console\n$ docker run --rm -p 1433:1433 \\\n  -e \"ACCEPT_EULA=1\" \\\n  -e \"MSSQL_SA_PASSWORD=Passw0rd\" \\\n  mcr.microsoft.com/azure-sql-edge:latest\n```\n\n### Basic usage\n\nWith docker running, you can start playing:\n\n```console\n# print the help/usage\n$ go-mssql-load help\n\n# print the DSN\n$ go-mssql-load printdsn\n\n# check the connection for localhost:1433\n$ go-mssql-load --user sa --pass Passw0rd check\n\n# load some example data\n$ go-mssql-load --user sa --pass Passw0rd loadsql sql/init.sql\n\n# try a query\n$ echo \"select * from pokemon.pokemon\" | go-mssql-load --user sa --pass Passw0rd querysql -\n# try a query, ignore logging\n$ echo \"select * from pokemon.pokemon\" | go-mssql-load --user sa --pass Passw0rd querysql - 2\u003e/dev/null\n```\n\n### CSV loading\n\nYou can use this tool to do CSV bulk loading. By default all columns are treated as\nstring, but you can specify a data type as\n\n* part of the column name or\n* external definition by using the `--types` parameter\n\n`loadcsv` doesn't have any parsing magic and uses the\n[csv parser provided by the go std lib](https://pkg.go.dev/encoding/csv). So, if you\ndon't specify it, it won't happen. The spec is as follows:\n\n```\n\u003cname\u003e::\u003cdatatype\u003e[!]\n```\n\nwith an optional `!` indicating a nullable column.\n\n| column spec      | column name | data type | nullable |\n| ---------------- | ----------- | --------- | -------- |\n| `name::string!`  | `name`      | `string`  | yes      |\n| `name::!`        | `name`      | `string`  | yes      |\n| `name`           | `name`      | `string`  | no       |\n| `age::int!`      | `age`       | `int64`   | yes      |\n| `height::float!` | `age`       | `float64` | yes      |\n| `height::float`  | `age`       | `float64` | no       |\n\nInternally the golang parse functions are used.\n\nSupported types:\n\n- `int` » `strconv.ParseInt(v, 10, 64)`\n- `float` » `strconv.ParseFloat(v, 64)`\n- `bool` » custom parsing, anything that looks like: `TRUE`, `true`, `T`, `t`, `YES`,\n  `yes`, `Y`, `y`, `1` is considered true.\n- `string` as default\n\nExample: `./sql/pokemon_typed.csv`\n\nYou can set the null string and the separator via cli flags. The null string is by\ndefault the empty string `\"\"`.\n\nThe TAB character is a bit tricky to specify, but you can just supply a quoted TAB to\nparse [TSV files](https://en.wikipedia.org/wiki/Tab-separated_values):\n\n```console\n$ go-mssql-load \\\n    --user sa --pass Passw0rd loadcsv --sep \"\t\" \\\n    pokemon.pokemon sql/pokemon_typed.csv\n```\n\nOnly columns that have the nullable flag `!` will use the `nullstr` flag.\n\nAs mentioned in the beginning, you can also supply an external types file in JSON\nformat. There are two options supported, as dict or as list.\n\nThe dict looks as follows (order is not important):\n\n```\n{\n  \"colname2\": \"int\",\n  \"colname1\": \"string!\",\n  \"colname3\": \"float!\"\n}\n```\n\nOr the same as list (order is important):\n\n```\n[\n  \"string!\",\n  \"int\",\n  \"float!\"\n]\n```\n\nYou can add the types via the `--types` parameter:\n\n```\n$ go-mssql-load --user sa --pass Passw0rd loadcsv \\\n    --sep \";\" \\\n    --types sql/pokemon_types.json \\\n    pokemon.pokemon sql/pokemon.csv\n```\n\n### SQL execution\n\n`go-mssql-load` uses the\n[batch mechanism](https://github.com/microsoft/go-mssqldb/blob/main/batch/batch.go) of\nthe [go-mssqldb](https://github.com/microsoft/go-mssqldb) lib. This means that each file\nread and split into statements separated by a `GO` keyword.\n\nExample: `./sql/init.sql`.\n\n```console\n$ go-mssql-load --user sa --pass Passw0rd loadsql sql/init.sql\n```\n\n### SQL querying\n\nSimilar to SQL execution, query scripts are split by the keyword `GO`. This means you\ncan have multiple query statements per file. Each query results in a set of\n[Newline Delimited JSON](http://ndjson.org/) records, separated by `---`.\n\nExample: `./sql/select.sql`\n\n```console\n$ go-mssql-load --user sa --pass Passw0rd querysql sql/select.sql  2\u003e/dev/null\n{\"name\":\"Wartortle\"}\n---\n{\"hp\":4}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjwbargsten%2Fgo-mssql-load","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjwbargsten%2Fgo-mssql-load","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjwbargsten%2Fgo-mssql-load/lists"}