{"id":29482223,"url":"https://github.com/runnerty/executor-snowflake","last_synced_at":"2025-07-15T01:33:11.954Z","repository":{"id":300750372,"uuid":"1007027138","full_name":"runnerty/executor-snowflake","owner":"runnerty","description":null,"archived":false,"fork":false,"pushed_at":"2025-06-23T11:51:36.000Z","size":63,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-23T12:25:29.618Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/runnerty.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-23T10:57:35.000Z","updated_at":"2025-06-23T11:51:40.000Z","dependencies_parsed_at":"2025-06-23T12:36:18.786Z","dependency_job_id":null,"html_url":"https://github.com/runnerty/executor-snowflake","commit_stats":null,"previous_names":["runnerty/executor-snowflake"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/runnerty/executor-snowflake","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runnerty%2Fexecutor-snowflake","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runnerty%2Fexecutor-snowflake/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runnerty%2Fexecutor-snowflake/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runnerty%2Fexecutor-snowflake/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/runnerty","download_url":"https://codeload.github.com/runnerty/executor-snowflake/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runnerty%2Fexecutor-snowflake/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265385756,"owners_count":23756728,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-15T01:31:18.487Z","updated_at":"2025-07-15T01:33:11.946Z","avatar_url":"https://github.com/runnerty.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ca href=\"http://runnerty.io\"\u003e\n    \u003cimg height=\"257\" src=\"https://runnerty.io/assets/header/logo-stroked.png\"\u003e\n  \u003c/a\u003e\n  \u003cp align=\"center\"\u003eSmart Processes Management\u003c/p\u003e\n\u003c/p\u003e\n\n[![NPM version][npm-image]][npm-url] [![Downloads][downloads-image]][npm-url]\n\u003ca href=\"#badge\"\u003e\n\u003cimg alt=\"code style: prettier\" src=\"https://img.shields.io/badge/code_style-prettier-ff69b4.svg\"\u003e\n\u003c/a\u003e\n\n# Snowflake executor for [Runnerty]:\n\n### Installation:\n\nThrough NPM\n\n```bash\nnpm i @runnerty/executor-snowflake\n```\n\nYou can also add modules to your project with [runnerty-cli]\n\n```bash\nnpx runnerty-cli add @runnerty/executor-snowflake\n```\n\nThis command installs the module in your project, adds example configuration in your `config.json` and creates an example plan of use.\n\nIf you have installed [runnerty-cli] globally you can include the module with this command:\n\n```bash\nrty add @runnerty/executor-snowflake\n```\n\n### Configuration:\n\nAdd in [config.json]:\n\n#### OAuth Authentication (Recommended):\n\nThis executor uses OAuth token authentication by default. You need to configure your OAuth token provider and set the following environment variables:\n\n```bash\nexport SNOWFLAKE_ACCOUNT=\"myaccount.us-east-1\"\nexport SNOWFLAKE_USERNAME=\"myuser@company.com\"\nexport SNOWFLAKE_PASSWORD=\"mypassword\"\n```\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"type\": \"@runnerty-executor-snowflake\",\n  \"account\": \"@ENV(SNOWFLAKE_ACCOUNT)\",\n  \"username\": \"@ENV(SNOWFLAKE_USERNAME)\",\n  \"database\": \"@ENV(SNOWFLAKE_DATABASE)\",\n  \"schema\": \"@ENV(SNOWFLAKE_SCHEMA)\",\n  \"warehouse\": \"@ENV(SNOWFLAKE_WAREHOUSE)\",\n  \"role\": \"@ENV(SNOWFLAKE_ROLE)\"\n}\n```\n\n#### Direct Configuration:\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"type\": \"@runnerty-executor-snowflake\",\n  \"account\": \"myaccount.us-east-1\",\n  \"username\": \"myuser@company.com\",\n  \"database\": \"MYDATABASE\",\n  \"schema\": \"PUBLIC\",\n  \"warehouse\": \"COMPUTE_WH\",\n  \"role\": \"MYROLE\"\n}\n```\n\n#### Configuration params:\n\n| Parameter   | Description                                                     |\n| :---------- | :-------------------------------------------------------------- |\n| account     | Snowflake account identifier (e.g., \"myaccount.us-east-1\")      |\n| username    | The Snowflake user to authenticate as.                          |\n| database    | Name of the database to use for this connection. (Optional)     |\n| schema      | Name of the schema to use for this connection. (Optional)       |\n| warehouse   | Name of the warehouse to use for this connection. (Optional)    |\n| role        | Name of the role to use for this connection. (Optional)         |\n| timeout     | Connection timeout in milliseconds. (Default: 60000)            |\n| application | Application name for connection tracking. (Default: \"runnerty\") |\n\n### Plan samples:\n\nAdd in [plan.json]:\n\n#### Basic query:\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"command_file\": \"./sql/my_query.sql\"\n}\n```\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"command\": \"SELECT CURRENT_TIMESTAMP() as now, CURRENT_USER() as user\"\n}\n```\n\n#### Query with parameters:\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"command\": \"SELECT * FROM users WHERE created_date \u003e= :start_date AND status = :status\",\n  \"args\": {\n    \"start_date\": \"2023-01-01\",\n    \"status\": \"active\"\n  }\n}\n```\n\n#### Export to files:\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"command\": \"SELECT * FROM sales_data WHERE year = 2023\",\n  \"xlsxFileExport\": \"./reports/sales_2023.xlsx\",\n  \"xlsxSheetName\": \"Sales Report\",\n  \"xlsxAuthorName\": \"Data Team\"\n}\n```\n\n### Generation of files:\n\nThe results can be exported to csv, xlsx and json format files. These files are generated using streams for optimal performance with large datasets.\nYou only have to indicate the corresponding property in the parameters:\n\n#### XLSX\n\nXLSX Format with streaming support for large datasets.\n\n| Parameter      | Description                   |\n| :------------- | :---------------------------- |\n| xlsxFileExport | Path of xlsx file export.     |\n| xlsxAuthorName | Author file name. (Optional)  |\n| xlsxSheetName  | Name of the sheet. (Optional) |\n\nSample:\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"command\": \"SELECT * FROM users\",\n  \"xlsxFileExport\": \"./exports/users.xlsx\",\n  \"xlsxAuthorName\": \"Runnerty\",\n  \"xlsxSheetName\": \"Users Data\"\n}\n```\n\n#### CSV\n\nCSV Format with streaming support and customizable options.\n\n| Parameter               | Description                                                           |\n| :---------------------- | :-------------------------------------------------------------------- |\n| csvFileExport           | Path of csv file export.                                              |\n| csvOptions/headers      | Type: boolean. Whether to include headers in the CSV. (Default: true) |\n| csvOptions/delimiter    | Alternate delimiter. (Default: ',')                                   |\n| csvOptions/quote        | Alternate quote. (Default: '\"')                                       |\n| csvOptions/rowDelimiter | Specify an alternate row delimiter (i.e \\r\\n). (Default: '\\n')        |\n| csvOptions/escape       | Alternate escaping value. (Default: '\"')                              |\n\nSample:\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"command\": \"SELECT * FROM users\",\n  \"csvFileExport\": \"@GV(WORK_DIR)/users.csv\",\n  \"csvOptions\": {\n    \"delimiter\": \";\",\n    \"quote\": \"\\\"\",\n    \"headers\": true\n  }\n}\n```\n\n#### JSON\n\nJSON Format with streaming support.\n\n| Parameter      | Description               |\n| :------------- | :------------------------ |\n| jsonFileExport | Path of json file export. |\n\nSample:\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"command\": \"SELECT * FROM users\",\n  \"jsonFileExport\": \"@GV(WORK_DIR)/users.json\"\n}\n```\n\n### Output (Process values):\n\n#### Standard\n\n- `PROCESS_EXEC_MSG_OUTPUT`: Snowflake output message.\n- `PROCESS_EXEC_ERR_OUTPUT`: Error output message.\n- `PROCESS_EXEC_COMMAND_EXECUTED`: Executed SQL command.\n\n#### Query output\n\n- `PROCESS_EXEC_DATA_OUTPUT`: Snowflake query output data (array of objects).\n- `PROCESS_EXEC_DB_COUNTROWS`: Snowflake query count rows.\n- `PROCESS_EXEC_DB_FIRSTROW`: Snowflake query first row data (object).\n- `PROCESS_EXEC_DB_FIRSTROW_[FIELD_NAME]`: Snowflake first row field data.\n\nExample of first row field access:\n\n```json\n{\n  \"id\": \"snowflake_default\",\n  \"command\": \"SELECT user_id, email, created_date FROM users LIMIT 1\"\n}\n```\n\nAvailable variables after execution:\n\n- `PROCESS_EXEC_DB_FIRSTROW_USER_ID`: First row user_id value\n- `PROCESS_EXEC_DB_FIRSTROW_EMAIL`: First row email value\n- `PROCESS_EXEC_DB_FIRSTROW_CREATED_DATE`: First row created_date value\n\n### Authentication Setup:\n\nThis executor requires OAuth authentication setup. Make sure you have:\n\n1. **Environment variables configured:**\n\n   ```bash\n   export SNOWFLAKE_ACCOUNT=\"your-account\"\n   export SNOWFLAKE_USERNAME=\"your-username\"\n   export SNOWFLAKE_PASSWORD=\"your-password\"\n   ```\n\n2. **OAuth token provider configured** (internal API endpoint)\n\n3. **Required dependencies installed:**\n   ```bash\n   npm install snowflake-sdk axios exceljs fast-csv jsonstream\n   ```\n\n### Features:\n\n- ✅ **OAuth Authentication** - Secure token-based authentication\n- ✅ **Streaming Support** - Efficient processing of large datasets\n- ✅ **Multiple Export Formats** - XLSX, CSV, JSON\n- ✅ **Parameterized Queries** - Support for `:parameter` placeholders\n- ✅ **Environment Variables** - Secure configuration management\n- ✅ **Error Handling** - Comprehensive error reporting\n- ✅ **Connection Pooling** - Optimized connection management\n\n[runnerty]: http://www.runnerty.io\n[downloads-image]: https://img.shields.io/npm/dm/@runnerty/executor-snowflake.svg\n[npm-url]: https://www.npmjs.com/package/@runnerty/executor-snowflake\n[npm-image]: https://img.shields.io/npm/v/@runnerty/executor-snowflake.svg\n[david-badge]: https://david-dm.org/runnerty/executor-snowflake.svg\n[david-badge-url]: https://david-dm.org/runnerty/executor-snowflake\n[config.json]: http://docs.runnerty.io/config/\n[runnerty-cli]: https://www.npmjs.com/package/runnerty-cli\n[plan.json]: http://docs.runnerty.io/plan/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunnerty%2Fexecutor-snowflake","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frunnerty%2Fexecutor-snowflake","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunnerty%2Fexecutor-snowflake/lists"}