{"id":15976794,"url":"https://github.com/joshuataylor/req_snowflake","last_synced_at":"2026-03-11T13:39:53.729Z","repository":{"id":137158605,"uuid":"497576023","full_name":"joshuataylor/req_snowflake","owner":"joshuataylor","description":"❄️ Snowflake integration for Elixir using req, supporting both JSON and Arrow","archived":false,"fork":false,"pushed_at":"2022-08-06T14:25:11.000Z","size":522,"stargazers_count":10,"open_issues_count":7,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-04T08:20:42.288Z","etag":null,"topics":["database","elixir","snowflake"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joshuataylor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-29T11:52:34.000Z","updated_at":"2025-09-26T18:10:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"17a3d6d6-6b38-4838-824a-3380d42e090e","html_url":"https://github.com/joshuataylor/req_snowflake","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/joshuataylor/req_snowflake","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuataylor%2Freq_snowflake","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuataylor%2Freq_snowflake/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuataylor%2Freq_snowflake/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuataylor%2Freq_snowflake/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joshuataylor","download_url":"https://codeload.github.com/joshuataylor/req_snowflake/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joshuataylor%2Freq_snowflake/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30382674,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-11T12:49:11.341Z","status":"ssl_error","status_checked_at":"2026-03-11T12:46:41.342Z","response_time":84,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","elixir","snowflake"],"created_at":"2024-10-07T22:41:00.381Z","updated_at":"2026-03-11T13:39:53.713Z","avatar_url":"https://github.com/joshuataylor.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/joshuataylor/req_snowflake\"\u003e\n    \u003cimg alt=\"req_snowflake\" src=\"https://user-images.githubusercontent.com/225131/175007256-0f3e5afd-8e90-47ad-a836-a38c57bf28ec.png\" width=\"50\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n# req_snowflake\n\n**NOTE: THIS DRIVER/CONNECTOR IS NOT OFFICIALLY AFFILIATED WITH SNOWFLAKE, NOR HAS OFFICIAL SUPPORT FROM THEM.**\n\nAn Elixir driver for [Snowflake](https://www.snowflake.com/), the cloud data platform.\n\nAlso has support for both pure-Elixir using JSON, or decoding Arrow files via [snowflake_arrow_elixir](), a Rust library which decodes the Arrow streaming file into Elixir.\n\n## Table of Contents\n\n- [Usage](#usage)\n- [What this is](#what-this-is)\n- [Features](#features)\n- [Options](#options)\n- [Table Integration](#table-integration)\n- [Short term Roadmap](#short-term-roadmap)\n- [Medium term roadmap](#medium-term-roadmap)\n- [Thanks](#thanks)\n\n## Usage\n\n```elixir\nMix.install([\n  {:req_snowflake, github: \"joshuataylor/req_snowflake\"}\n])\n\n# With plain string query\nReq.new()\n|\u003e ReqSnowflake.attach(\n  username: \"rosebud\",\n  password: \"hunter2\",\n  account_name: \"foobar\",\n  region: \"us-east-1\",\n  warehouse: \"compute_wh\", # optional\n  role: \"myrole\", # optional\n  database: \"mydb\", # optional\n  schema: \"myschema\" # optional,\n  session_parameters: %{} # Passing in session parameters from\n  parallel_downloads: 10 # optional, but recommended. Defaults to 5 (what the other connectors default to).\n)\n|\u003e Req.post!(query: \"select L_ORDERKEY, L_PARTKEY from snowflake_sample_data.tpch_sf1.lineitem limit 2\").body\n#=\u003e\n# %ReqSnowflake.Result{\n#   columns: [\"L_ORDERKEY\", \"L_PARTKEY\"],\n#   total_rows: 2,\n#   rows: [[3_000_001, 14406], [3_000_002, 34422]],\n#   success: true\n# }\n\n# With query parameters for inserting\nReq.new()\n|\u003e ReqSnowflake.attach(\n  username: \"rosebud\",\n  password: \"hunter2\",\n  account_name: \"foobar\",\n  region: \"us-east-1\",\n  warehouse: \"compute_wh\", # optional\n  role: \"myrole\", # optional\n  database: \"mydb\", # optional\n  schema: \"myschema\" # optional\n)\n|\u003e Req.post!(\n  query: \"INSERT INTO \\\"foo\\\".\\\"bar\\\".\\\"baz\\\" (\\\"hello\\\") VALUES (?)\",\n  bindings: %{\"1\" =\u003e %{type: \"TEXT\", value: \"xxx\"}}\n)\n#=\u003e\n# %ReqSnowflake.Result {\n#   columns: [\"number of rows inserted\"],\n#   total_rows: 1,\n#   rows: [[1]],\n#   success: true\n# }\n```\n\n## What this is\n\nIt uses the Snowflake REST API to communicate with Snowflake, with an earlier version set for JSON (with support for Arrow if using [snowflake_arrow](https://github.com/joshuataylor/snowflake_arrow).\nThe REST API is used by the Python, Golang, NodeJS and other languages to send requests to Snowflake, so it is stable and changes are communicated.\n\nThis library does not use the [Snowflake SQL API](https://docs.snowflake.com/en/developer-guide/sql-api/index.html), which is limited in its implementation and features.\n\nRight now the library doesn't support [MFA](https://docs.snowflake.com/en/user-guide/security-mfa.html), so you'll need to either use [private key auth](https://docs.snowflake.com/en/user-guide/odbc-parameters.html#using-key-pair-authentication) or connect using a username \u0026 password. A private key auth is highly recommended as you can rotate passwords easier.\n\nOne of the major notes when using Ecto is you will need to enable Snowflakes `QUOTED_IDENTIFIERS_IGNORE_CASE` setting, which you can find here: https://docs.snowflake.com/en/sql-reference/identifiers-syntax.html#third-party-tools-and-case-insensitive-identifier-resolution\n\nNote that this can be done on an account or if needed on a session level which you can set below.\n\n## Features\n* Running queries and inserts.\n* JSON row decoding, using Flow this is much faster to decode. It's recommended to also use Jiffy, benchmarking shows a 2x speedup due to the size of the JSON files Snowflake returns.\n* Arrow row decoding, using [snowflake_arrow](https://github.com/joshuataylor/snowflake_arrow)\n* Passing async queries and getting back a query ID, or polling for results\n\n## Options\nThere are a lot of options that you can pass in, and you can also pass in [Snowflake Session Parameters](https://docs.snowflake.com/en/sql-reference/parameters.html) as documented below.\n\n- snowflake_query *string* **required**\n\n  Your snowflake query. `select L_ORDERKEY, L_PARTKEY from snowflake_sample_data.tpch_sf1.lineitem limit 2`\n\n- username *string* **required**\n\n  Your snowflake username.\n\n- password *string* **required**\n\n  Your snowflake password.\n\n- account_name  *string* **required**\n\n  Your account name, this is found before the region name in the URL. `https://abc1234.us-east-1.snowflakecomputing.com` would be `abc1234` .\n\n- region *string* **required**\n\n  Your snowflake region, this is found after your account name.\n\n- arrow (boolean) *optional*\n\n  Whether or not to use Arrow. You must have snowflake_arrow included in your project for this to work.\n- cache_token *optional*\n\n  Cache the login token between queries, for up to 10 minutes. 10 minutes is the standard login token time for Snowflake.\n  If you change a parameter (apart from the query) this will relog you in.\n- warehouse *optional*\n  The warehouse to use. If none is provided, will use the users default warehouse\n\n- role *optional*\n\n  The role to use. If none is provided, will use the users default role\n- database *optional*\n\n  The database to use. If none is provided, will use the users default database\n- schema **string** *optional*\n\n  The schema to use. If none is provided, will use the users default schema\n- application_name **string** *optional*\n\n  Application name to pass. By default will not use an application name.\n- bindings **map** *optional*\n\n  Any bindings to pass.\n- session_parameters **map** *optional*\n\n  You can pass any session parameters from https://docs.snowflake.com/en/sql-reference/parameters.html.\n\n  Example: `session_parameters: %{ROWS_PER_RESULTSET: 50}` will return 50 results only.\n\n- parallel_downloads **integer** *optional*\n\n  How many parallel downloads to perform for s3. This defaults to 5, which is the default for other connectors.\n- async **boolean** *optional*\n\n  Will run the query in async mode, returning you the query ID.\n\n- async_poll **boolean** *optional*\n\n  Will run the query in async mode, then poll every 5000ms (unless defined by `async_poll_interval`) for the result.\n\n- async_poll_interval **integer**  *optional*\n\n  Will run the query in async mode, then poll every interval milliseconds.\n\n- async_poll_timeout **integer** *optional*\n\n  How many times it will try to poll for the result before giving up.\n\n- download_chunks **boolean** *optional*\n\n  Whether to download the chunks or just return the base64.\n\n- table **boolean** *optional*\n\n  If true, will return the results with the expectation you will use [table]() to process the results.\n  By default this is `false`. For kino_db, this is set as `true` as this uses Table by default.\n\n  If you are dealing with a lot of data, it's **highly recommended** to use Table as you can stream the chunks\n  out as you need them, saving you memory, CPU and bandwidth.\n\n- cache_results **boolean *optional*\n\n  Whether to cache the downloaded chunks, if you are re-iterating over the same dataset this will be cached locally\n  so that you don't have to redownload the data. This is false by default, but enabled for kino_db as when paging\n  we don't want to redownload the chunk.\n\n- json_library **module** *optional*\n\n  When decoding JSON, jiffy has shown to be 2x faster and use less memory than Jason for larger JSON blobs, as Snowflake can send large JSON files.\n  Examples: `json_library: Jason` or `json_library: :jiffy`. :jiffy is an atom because it's an Erlang library.\n  Defaults to JSON.\n\n## Table Integration\nUsing [table](https://github.com/dashbitco/table) makes it incredibly easy to manipulate and stream over the data. If you\ndownload all the chunks at once, you might end up using a lot of memory if you do this over millions of rows.\n\nWith table, you can enumerate over the data and only the relevant chunks will be downloaded and decoded.\n\nTo get results 0-200, you could do the following:\n```elixir\nReq.new()\n|\u003e ReqSnowflake.attach(\n# your details here!\n)\n|\u003e Req.post!(\n  snowflake_query: \"select * from foo.bar.baz\"\n)\n|\u003e Map.get(:body)\n|\u003e Table.to_rows()\n|\u003e Enum.slice(0, 200)\n```\nWhich results in the following:\n```elixir\n[\n%{\n  \"ROW_NUMBER\" =\u003e 688156,\n  \"SF_ARRAY\" =\u003e nil,\n  \"SF_BOOLEAN\" =\u003e false,\n  \"SF_DATE\" =\u003e ~D[2024-04-19],\n  \"SF_DECIMAL_38_2\" =\u003e 2146.89,\n  \"SF_FLOAT\" =\u003e nil,\n  \"SF_FLOAT_TWO_PRECISION\" =\u003e nil,\n  \"SF_INTEGER\" =\u003e 8562093803,\n  \"SF_OBJECT\" =\u003e nil,\n  \"SF_TIMESTAMP\" =\u003e ~N[2025-04-20 07:31:58.064000],\n  \"SF_TIMESTAMP_LTZ\" =\u003e nil,\n  \"SF_TIMESTAMP_NTZ\" =\u003e nil,\n  \"SF_VARCHAR\" =\u003e \"ylPPdH0leSF3f5lK9kEC\",\n  \"SF_VARIANT_JSON\" =\u003e \"{\\n  \\\"key_eV6kL4tVYXLFrEyyDNol\\\": true\\n}\"\n},\n...\n]\n```\n\n## kino_db\nkino_db integration is supported out of the box, which allows you to connect to an account, create queries and slice the data.\nIt will only download the chunks that it needs, thanks to slicing. This allows you to see results in the table extremely quickly.\n\n## Short term Roadmap\n- Add this to `db_connection_snowflake` as a generic Snowflake library for db_connection\n- Add this to `ecto_snowflake` as a Snowflake library for Ecto.\n\n## Medium term roadmap\n- Add support for MFA\n- Add support for private key auth\n- Add support for [telemetry](https://github.com/beam-telemetry/telemetry)\n\n## Thanks\nI just want to thank the opensource community, especially dbconnection/ecto/ecto_sql/postgrex for being amazing, and \nbeing able to copy most of the decoding code from that. I also want to thank the [@wojtekmach](https://github.com/wojtekmach) for the awesome work on req, and the help converting this to req.\n\nThanks to [@lewisvrobinson](https://github.com/lewisvrobinson) for the logo.\n\n## License\n\nCopyright (C) 2022 Josh Taylor\n\n    Licensed under the Apache License, Version 2.0 (the \"License\");\n    you may not use this file except in compliance with the License.\n    You may obtain a copy of the License at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)\n\n    Unless required by applicable law or agreed to in writing, software\n    distributed under the License is distributed on an \"AS IS\" BASIS,\n    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n    See the License for the specific language governing permissions and\n    limitations under the License.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoshuataylor%2Freq_snowflake","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoshuataylor%2Freq_snowflake","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoshuataylor%2Freq_snowflake/lists"}