{"id":16213642,"url":"https://github.com/dmanjunath/node-redshift","last_synced_at":"2025-04-07T16:18:20.241Z","repository":{"id":40685839,"uuid":"41779894","full_name":"dmanjunath/node-redshift","owner":"dmanjunath","description":"A simple collection of tools to help you get started with Amazon Redshift from node.js","archived":false,"fork":false,"pushed_at":"2022-11-18T05:12:00.000Z","size":538,"stargazers_count":69,"open_issues_count":22,"forks_count":48,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-31T14:12:16.822Z","etag":null,"topics":["connection-pool","node-redshift","orm","redshift"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dmanjunath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-09-02T04:30:03.000Z","updated_at":"2024-02-25T05:19:08.000Z","dependencies_parsed_at":"2023-01-22T11:16:25.448Z","dependency_job_id":null,"html_url":"https://github.com/dmanjunath/node-redshift","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmanjunath%2Fnode-redshift","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmanjunath%2Fnode-redshift/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmanjunath%2Fnode-redshift/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmanjunath%2Fnode-redshift/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dmanjunath","download_url":"https://codeload.github.com/dmanjunath/node-redshift/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247685628,"owners_count":20979085,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["connection-pool","node-redshift","orm","redshift"],"created_at":"2024-10-10T11:07:17.983Z","updated_at":"2025-04-07T16:18:20.216Z","avatar_url":"https://github.com/dmanjunath.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Navigation\n\n#### [Overview](https://github.com/dmanjunath/node-redshift#overview-1)\n\n#### [Installation](https://github.com/dmanjunath/node-redshift#installation-1)\n\n#### [Setup](https://github.com/dmanjunath/node-redshift#setup-1)\n\n#### [Usage](https://github.com/dmanjunath/node-redshift#usage-1)\n\n- #### [Query API](https://github.com/dmanjunath/node-redshift#query-api-2)\n- #### [CLI](https://github.com/dmanjunath/node-redshift#cli-2)\n- #### [Models](https://github.com/dmanjunath/node-redshift#models-2)\n- #### [ORM](https://github.com/dmanjunath/node-redshift#orm-api)\n\n#### [Upcoming Features](https://github.com/dmanjunath/node-redshift#upcoming-features-1)\n\n#### [License](https://github.com/dmanjunath/node-redshift#license-1)\n\n## Overview\nThis package is a simple wrapper for common functionality you want when using Redshift. It can do\n- Redshift connections \u0026 querying\n- Creating and running migrations\n- Create and manage models\n- CRUD API with ORM wrapper with type validation\n\nWarning!!!!!! This is new and still under development. The API is bound to change. Use at your own risk.\n\n## Installation\nInstall the package by running\n```javascript\nnpm install node-redshift\n```\nLink to npm repository https://www.npmjs.com/package/node-redshift\n\n## Setup\n\nThe code to connect to redshift should be something like this:\n```javascript\n//redshift.js\nvar Redshift = require('node-redshift');\n\nvar client = {\n  user: user,\n  database: database,\n  password: password,\n  port: port,\n  host: host,\n};\n\n// The values passed in to the options object will be the difference between a connection pool and raw connection\nvar redshiftClient = new Redshift(client, [options]);\n\nmodule.exports = redshiftClient;\n```\n\nThere are two ways to setup a connection to redshift. \n\n- [Connection Pooling](https://github.com/dmanjunath/node-redshift#connection-pooling) -  you can open a connection pool and open connections to Redshift which will be managed by pg-pool (https://github.com/brianc/node-pg-pool)\n- [Raw Connection](https://github.com/dmanjunath/node-redshift#raw-connection) - a one time connection you must manually initialize and close to run queries\n\n\n###### ***By default node-redshift uses connection pooling\n\n#### \n##### Raw Connection\nPass in the rawConnection parameter in the redshift instantiation options to specify a raw connection. Raw connections need extra code to specify when to connect and disconnect from Redshift. [Here's an example of the raw connection query](https://github.com/dmanjunath/node-redshift/blob/master/examples/raw_connection.js)\n\n```javascript\nvar redshiftClient = new Redshift(client, {rawConnection: true});\n```\n\n##### Connection Pooling \nConnection pooling works by default with no extra configuration. [Here's an example of connection pooling](https://github.com/dmanjunath/node-redshift/blob/master/examples/connection_pooling.js)\n\n##### Setup Options\nThere are two options that can be passed into the options object in the Redshift constructor.\n\n| Option                | Type          | Description                                                                       |\n| --------------------- |:-------------:| ---------------------------------------------------------------------------------:|\n| rawConnection         | Boolean       | If you want a raw connection, pass true with this option                          |\n| longStackTraces       | Boolean       | Default: true. If you want to disable [bluebird's longStackTraces](http://bluebirdjs.com/docs/api/promise.longstacktraces.html), pass in false   |\n\n\n## Usage\n\n#### [Query API](https://github.com/dmanjunath/node-redshift#query-api-2)\n#### [CLI](https://github.com/dmanjunath/node-redshift#cli-2)\n#### [Models](https://github.com/dmanjunath/node-redshift#models-2)\n#### [ORM](https://github.com/dmanjunath/node-redshift#orm-api)\n#\n### Query API\nPlease see examples/ folder for full code examples using both raw connections and connection pools.\n\nFor those looking for a library to build robust, injection safe SQL, I like [sql-bricks](http://csnw.github.io/sql-bricks/) to build query strings.\n\nBoth Raw Connections and Connection Pool connections have two query functions that are bound to the initialized Redshift object: `query()` and a `parameterizedQuery()`.\n\nAll `query()` and `parameterizedQuery()` functions support **both callback and promise style**. If there's a function as a third argument, the callback will fire. If there's no third function argument, but instead (query, [options]).then({})... the promise will fire.\n\n```javascript\n//raw connection\nvar redshiftClient = require('./redshift.js');\n\nredshiftClient.connect(function(err){\n  if(err) throw err;\n  else{\n    redshiftClient.query('SELECT * FROM \"TableName\"', [options], function(err, data){\n      if(err) throw err;\n      else{\n        console.log(data);\n        redshiftClient.close();\n      }\n    });\n  }\n});\n```\n#\n```javascript\n//connection pool\nvar redshiftClient = require('./redshift.js');\n\n// options is an optional object with one property so far {raw: true} returns \n// just the data from redshift. {raw: false} returns the data with the pg object\nredshiftClient.query(queryString, [options])\n.then(function(data){\n    console.log(data);\n})\n.catch(function(err){\n    console.error(err);\n});\n//instead of promises you can also use callbacks to get the data\n```\n\n##### Parameterized Queries \nIf you parameterize the SQL string yourself, you can call the `parameterizeQuery()` function \n```javascript\n//connection pool\nvar redshiftClient = require('./redshift.js');\n\n// options is an optional object with one property so far {raw: true} returns \n// just the data from redshift. {raw: false} returns the data with the pg object\nredshiftClient.parameterizedQuery('SELECT * FROM \"TableName\" WHERE \"parameter\" = $1', [42], [options], function(err, data){\n  if(err) throw err;\n  else{\n    console.log(data);\n  }\n});\n//you can also use promises to get the data\n```\n\n##### Template Literal Queries \nIf you use template literals to write your SQL, you can use a tagged template parser like https://github.com/felixfbecker/node-sql-template-strings to parameterize the template literal\n```javascript\n//connection pool\nvar redshiftClient = require('./redshift.js');\nvar SQL = require('sql-template-strings');\n\n// options is an optional object with one property so far {raw: true} returns \n// just the data from redshift. {raw: false} returns the data with the pg object\nlet value = 42;\n\nredshiftClient.query(SQL`SELECT * FROM \"TableName\" WHERE \"parameter\" = ${value}`, [options], function(err, data){\n  if(err) throw err;\n  else{\n    console.log(data);\n  }\n});\n//you can also use promises to get the data\n```\n\n##### `rawQuery()` \nIf you want to make a one time raw query, but you don't want to call connect \u0026 disconnect manually and you dont want to use conection pooling, you can use `rawQuery()`\n```javascript\n//connection pool\nvar redshiftClient = require('./redshift.js');\n\n// options is an optional object with one property so far {raw: true} returns \n// just the data from redshift. {raw: false} returns the data with the pg object\nredshiftClient.rawQuery('SELECT * FROM \"TableName\"', [options], function(err, data){\n  if(err) throw err;\n  else{\n    console.log(data);\n  }\n});\n//you can also use promises to get the data\n```\n\n##### Query Options \nThere's only a single query option so far. For the options object, the only valid option is {raw: true}, which returns just the data from redshift. {raw: false} or not specifying the value will return the data along with the entire pg object with data such as row count, table statistics etc.\n\n\n### CLI\nThere's a CLI with options for easy migration management. Creating a migration will create a `redshift_migrations/` folder with a state file called `.migrate` in it which contains the state of your completed migrations. The .migrate file keeps track of which migrations have been run, and when you run db:migrate, it computes the migrations that have not yet been run on your Redshift instance and runs them and saves the state of `.migrate`\n\nWARNING!!! IF YOU HAVE SEPARATE DEV AND PROD REDSHIFT INSTANCES, DO NOT COMMIT THE `.migrate` FILE TO YOUR VCS OR DEPLOY TO YOUR SERVERS. YOU'LL NEED A NEW VERSION OF THIS FILE FOR EVERY INSTANCE OF REDSHIFT.\n\n##### Create a new migration file in redshift_migrations/ folder\n#\n```\nnode_modules/.bin/node-redshift migration:create \u003cfilename\u003e\n```\n\n##### Run all remaining migrations on database\n#\n```\nnode_modules/.bin/node-redshift db:migrate \u003cfilename\u003e\n```\n\n##### Undo last migration\n#\n```\nnode_modules/.bin/node-redshift db:migrate:undo \u003cfilename\u003e\n```\n\n##### Creating a model using the command line\n#\n```\nnode_modules/.bin/node-redshift model:create \u003cfilename\u003e\n```\n\n### Models\n\nA model will look like this\n```javascript\n'use strict';\n  var person = {\n    'tableName': 'people',\n    'tableProperties': {\n      'id': {\n        'type': 'key'\n      },\n      'name': { \n        'type': 'string',\n        'required': true\n      },\n      'email': { \n        'type': 'string',\n        'required': true\n      }\n    }\n  };\n  module.exports = person;\n```\n##### Importing and using model with ORM\n#\nThere are two ways you could import and use redshift models. The first is using redshift.import in every file where you want to use the model ORM.\n```javascript\nvar redshift = require(\"../redshift.js\");\nvar person = redshift.import(\"./redshift_models/person.js\");\n\nperson.create({name: 'Dheeraj', email: 'dheeraj@email.com'}, function(err, data){\n    if(err) throw err;\n    else{\n      console.log(data);\n    }\n  });\n```\n\nThe alternative(my preferred way) is to abstract the import calls and export all the models with the redshift object right after initialization\n\n```javascript\n//redshift.js\n...redshift connection code...\n\nvar person = redshift.import(\"./redshift_models/person.js\");\nredshift.models = {};\nredshift.models.person = person;\n\nmodule.exports = redshift;\n\n//usage in person.js\nvar redshiftConnection = require('./redshift.js');\nvar person = redshift.models.person;\n\nperson.create({name: 'Dheeraj', email: 'dheeraj@email.com'}, function(err, data){\n    if(err) throw err;\n    else{\n      console.log(data);\n    }\n  });\n```\n\n### ORM API\nThere are 3 functions supported by the ORM\n```javascript\n/**\n * create a new instance of object\n * @param  {Object or Array}   data Object/Array with keys/values to create in database. keys are column names, values are data\n * @param  {Function} cb   \n * @return {Object}        Object that's inserted into redshift\n */\nPerson.create({emailAddress: 'dheeraj@email.com', name: 'Dheeraj'}, function(err, data){\n  if(err) throw err;\n  else console.log(data);\n});\n \n/**\n * update an existing item in redshift\n * @param  {Object}   whereClause The properties that identify the rows to update. Essentially the WHERE clause in the UPDATE statement\n * @param  {Object}   data        Properties to overwrite in the record\n * @param  {Function} callback    \n * @return {Object}               Object that's updated in redshift\n *\n */\nPerson.update({id: 72}, {emailAddress: 'dheeraj@email.com', name: 'Dheeraj'}, function(err, data){\n  if(err) throw err;\n  else console.log(data);\n});\n\n/**\n * delete rows from redshift\n * @param  {Object}   whereClause The properties that identify the rows to update. Essentially the WHERE clause in the UPDATE statement\n * @param  {Function} cb   \n * @return {Object}        Object that's deleted from redshift\n */\nPerson.delete({emailAddress: 'dheeraj@email.com', name: 'Dheeraj'}, function(err, data){\n  if(err) throw err;\n  else console.log(data);\n});\n```\n\n## Upcoming features\n- Ability to customize location of `.migrate` file or even from S3\n- Model checking prior to queries to verify property name and type\n- Add class \u0026 instance methods to model\n\n## License\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmanjunath%2Fnode-redshift","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmanjunath%2Fnode-redshift","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmanjunath%2Fnode-redshift/lists"}