{"id":20324918,"url":"https://github.com/xeeo/nc-db-handler","last_synced_at":"2025-09-22T23:31:59.403Z","repository":{"id":57309162,"uuid":"78422326","full_name":"xeeo/nc-db-handler","owner":"xeeo","description":null,"archived":false,"fork":false,"pushed_at":"2017-08-03T12:43:40.000Z","size":26,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-15T09:54:45.610Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xeeo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-09T11:21:33.000Z","updated_at":"2017-01-11T10:40:55.000Z","dependencies_parsed_at":"2022-09-09T04:12:05.494Z","dependency_job_id":null,"html_url":"https://github.com/xeeo/nc-db-handler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xeeo%2Fnc-db-handler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xeeo%2Fnc-db-handler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xeeo%2Fnc-db-handler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xeeo%2Fnc-db-handler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xeeo","download_url":"https://codeload.github.com/xeeo/nc-db-handler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233905339,"owners_count":18748623,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T19:38:09.617Z","updated_at":"2025-09-22T23:31:54.133Z","avatar_url":"https://github.com/xeeo.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nc-db-handler\n\nThe DS-Handler is the Module that handles the handling of the Data Store connections (PG and Elastic).\n\n## Installation\n```\nnpm install --save nc-db-handler\n```\n\n## Usage\n\n```javascript\nvar NcDbHandler = require('nc-db-handler');\nvar dsHandler   = new NcDbHandler();\nvar config      = {\n    'write-store': {\n        db1: 'postgres://postgres:postgres@localhost/db1',\n        db2: 'postgres://postgres:postgres@localhost/db1'\n    },\n    'read-store': {\n        index1: 'https://user:pass@localhost',\n        index2: 'https://user:pass@localhost'\n    }\n};\n\ndsHandler.config(pgConfig);\n\ndsHandler.create({...}).then({...});\n```\n\n## Method Summary\n\n##### Write-Store Methods (PG)\n- [Create](#create)\n- [Read](#read)\n- [Remove](#remove)\n- [SoftRemove](#softremove)\n- [Update](#update)\n- [Count](#count)\n- [PG](#pg)\n\n##### Read-Store Methods (Elastic)\n- [Search](#search)\n- [Index](#index)\n- [ReIndex](#reindex)\n- [UnIndex](#unindex)\n- [ES](#es)\n\n## Write-Store Methods\n\n#### Create\n\nThis methods hits PG\n\n```javascript\nncDbHandler.create({\n    dataSource: 'reviews.reviews', /* {String} | required | format: \"database.table\" */\n    logQuery: true, /* {Boolean} | optional | default: false */\n    mapping: require('adding-review-mapping'), /* {Function} | optional */\n    payload: { /* {Object} | required */\n        text: 'Ola',\n        rating: '1',\n        reviewerId: '123',\n        profileId: '20',\n        product: 'TK',\n        platform: 'web',\n        ip: '127.0.0.1',\n        weddingDate: '21.01.2015'\n    }\n    outputMapping: require('some-mapper'), /* {Function} | optional */\n    payloadSchema: {  /* {Object} | optional | Joi validation schema applied on payload */\n        text: Joi.....\n    },\n    outputSchema: { /* {Object} | optional | Joi validation schema applied on the result of the outputMapping */\n        text: Joi.....\n    }\n}).then(function success(result) {\n   /**\n    * @return {Object} - the Object will be the returned value of outputMapping and validated by the outputSchema\n    */\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### Read\n\nThis methods hits PG\n\n```javascript\nncDbHandler.read({\n    dataSource: 'reviews.reviews', /* {String} | required | format: \"database.table\" */\n    logQuery: true, /* {Boolean} | optional | default: false */\n    native: 'WHERE data-\u003e\u003e\\'id\\' IN (\\'13\\', \\'123\\', \\'1234\\') LIMIT 2 ORDER BY data-\u003e\u003e\\'id\\' DESC', /* {String} | optional */\n    query:\n        and: [\n            { /* {Array|Object} | optional */\n                {\n                    native: 'data-\u003e\\'id\\' IN (\\'13\\', \\'123\\', \\'1234\\')'\n                },\n                {\n                    or : [\n                        {\n                            'data-\u003eaddress-\u003e\u003estreet': 'Anderson'\n                        },\n                        {\n                            always: 1\n                        }\n                    ]\n                },\n                {\n                    email : [\n                        'joe.doe@gmail.com',\n                        'jane.doe@gmail.com',\n                    ]\n                }\n            } /* WHERE data-\u003e'id' IN (13,123,1234) AND (data-\u003e'address'-\u003e\u003e'street' = 'Anderson' OR data-\u003e\u003e'always' = 1) AND (data-\u003e\u003e'email' = 'joe.doe@gmail.com' OR data-\u003e\u003e'email' = 'jane.doe@gmail.com') */\n        ],\n    sort: { /* {Object} | optional */\n        'data-\u003eaddress-\u003e\u003estreet': 'asc'\n    },\n    limit: 1, /* {Number} | optional */\n    offset: 20, /* {Number} | optional */\n    outputMapping: require('some-mapper'), /* {Function} | optional */\n    outputSchema: { /* {Object} | optional | Joi validation schema applied on the result of the outputMapping */\n        address: Joi.....\n    }\n}).then(function success(results) {\n    /**\n     * @return {Array} - results is an Array of Objects mapped with outputMapping and validated using the outputSchema and matching the search criteria\n     */\n});\n\n/* Other query examples\nquery: {\n    or : {\n        { 'native': 'id IN (13,123,1234)' },\n        { and : {\n            { 'meta-\u003eaddress-\u003e\u003estreet': 'Anderson\"' },\n            { 'meta-\u003e\u003ealways': 1 }\n        } },\n        { 'meta-\u003e\u003eemail' : [\n            'joe.doe@gmail.com',\n            'jane.doe@gmail.com',\n        ] }\n    }\n} === WHERE id IN (13,123,1234) OR (meta-\u003eaddress-\u003e\u003estreet = \"Anderson\" AND meta-\u003e\u003ealways = 1) OR (meta-\u003e\u003eemail = \"joe.doe@gmail.com\" OR meta-\u003e\u003eemail = \"jane.doe@gmail.com\")\n*/\n\n/*\nquery: [\n    { 'meta-\u003e\u003ereviewer': 123 },\n    { 'meta-\u003e\u003eprofile': 45 }\n] === WHERE meta-\u003e\u003ereviewer = 123 AND meta-\u003e\u003eprofile = 45\n*/\n\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### Remove\n\nThis methods hits PG\n\n```javascript\nncDbHandler.remove({\n    dataSource: 'reviews.reviews', /* {String} | required | format: \"database.table\" */\n    logQuery: true, /* {Boolean} | optional | default: false */\n    native: 'WHERE data-\u003e\u003e\\'id\\' IN (\\'1\\', \\'2\\', \\'3\\')', /* {String} | optional */\n    query: [ /* {Array|Object} | optional */\n        {\n            'data-\u003e\u003eid': ['1', '2', '3']\n        }\n    ],\n    sort: { /* {Object} | optional */\n        'data-\u003eaddress-\u003e\u003estreet': 'asc'\n    },\n    limit: 1, /* {Number} | optional */\n    offset: 20, /* {Number} | optional */\n}).then(function success(results) {\n    /**\n     * @return {Array} - results is an Array of Objects containing the deleted Documents\n     */\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### SoftRemove\n\nThis methods hits PG\n\n```javascript\nncDbHandler.softRemove({\n    dataSource: 'reviews.reviews', /* {String} | required | format: \"database.table\" */\n    logQuery: true, /* {Boolean} | optional | default: false */\n    native: 'WHERE data-\u003e\u003e\\'id\\' = \\'UUID\\'' /* {String} | optional */\n    query: { /* {Array|Object} | optional */\n        'data-\u003e\u003eid' : 'UUID'\n    },\n    sort: { /* {Object} | optional */\n        'data-\u003eaddress-\u003e\u003estreet': 'asc'\n    },\n    limit: 1, /* {Number} | optional */\n    offset: 20, /* {Number} | optional */\n}).then(function success(result) {\n    /**\n     * @return {Array} - results is an Array of Objects containing the updated Documents with {status: 'deleted', deletedAt: Date.now()}\n     */\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### Update\n\nThis methods hits PG\n\n```javascript\nncDbHandler.update({\n    dataSource: 'reviews.reviews', /* {String} | required | format: \"database.table\" */\n    logQuery: true, /* {Boolean} | optional | default: false */\n    native: 'WHERE data-\u003e\u003e\\'id\\' = \\'UUID\\'' /* {String} | optional */\n    query: { /* {Array|Object} | optional */\n        'data-\u003e\u003eid' : 'UUID'\n    },\n    payload: { /* {Object} | required */\n       profiles: [555]\n    },\n    replaceData: false, /* {Boolean} | optional | defaults: false */\n    mapping: require('mapping-function'), /* {Function} | optional */\n    outputMapping: require('mapping-function') /* {Function} | optional */\n    payloadSchema: {  /* {Object} | optional | Joi validation schema applied on the payload object */\n       profiles: Joi.....\n    },\n    outputSchema: { /* {Object} | optional | Joi validation schema applied on the result of the outputMapping */\n       profiles: Joi.....\n    }\n}).then(function success(results) {\n    /**\n     * @return {Array} - results is an Array of Objects representing all updated rows mapped with outputMapping and validated with outputSchema\n     */\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### Count\n\nThis methods hits PG\n\n```javascript\nncDbHandler.count({\n    dataSource: 'reviews.reviews', /* {String} | required | format: \"database.table\" */\n    logQuery: true, /* {Boolean} | optional | default: false */\n    native: 'WHERE data-\u003e\u003e\\'id\\' = \\'UUID\\'' /* {String} | optional */\n    query: { /* {Array|Object} | optional */\n        'data-\u003e\u003eid' : 'UUID'\n    },\n}).then(function success(result) {\n    /**\n     * @return {Number} - result is the COUNT\n     */\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### PG\n\nThis methods hits PG\n\n```javascript\nncDbHandler.pg('db-reviews', 'SELECT * FROM reviews WHERE meta-\u003e\u003e\\'id\\' = \\'245342\\'')\n    .then(function success(result) {\n        /**\n         * @return {Object|Array} - result is the raw Postgres data\n         */\n    });\n```\n\n[back to Method Summary](#nc-db-handler)\n\n## Read-Store Methods\n\n\n#### Search\n\nThis methods hits Elastic\n\n```javascript\nncDbHandler.search({\n    dataSource: 'reviews.reviews', /* {String} | required | format: \"index.type\" */\n    logQuery: true, /* {Boolean} | optional | default: false */\n    query: {}, /* {Object} | optional | native Elastic QUERY object from body */\n    filter: {}, /* {Object} | optional | native Elastic FILTER object from body */\n    aggs: {}, /* {Object} | optional | native Elastic AGGREGATION object from body */\n    /* even though the previous 3 are optional, at least one needs to be set */\n    sort: { /* {Object} | optional */\n        'address.street': 'asc'\n    },\n    limit: 1, /* {Number} | optional */\n    offset: 20, /* {Number} | optional */\n    outputMapping: require('some-mapper'), /* {Function} | optional */\n    outputSchema: {  /* {Object} | optional | Joi validation schema applied on the result of the outputMapping */\n       Joi.....\n   }\n}).then(function success(result) {\n    /**\n     * @return {Array} results is an Array of objects from ElasticSearch\n     */\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### Index\n\nThis methods hits Elastic\n\n```javascript\nncDbHandler.index({\n    dataSource: 'reviews.reviews', /* {String} | required | format: \"index.type\" */\n    logQuery: true, /* {Boolean} | optional | default: false */\n    payload: [ /* {Object|Array} | required */\n       {\n            id: 'UUID', /* required */\n            user: 'some user name',\n            content: 'some content here'\n       },\n       {\n            id: 'UUID', /* required */\n            user: 'another user',\n            content: 'some content here'\n       }\n    ],\n    mapping: require('some-mapper'), /* {Function} | optional */\n    payloadSchema: {  /* {Object} | optional | Joi validation schema applied on the result of the mapping */\n       id: Joi.....\n    },\n    replaceData: false /* {Boolean} | optional | defaults: false */\n}).then(function success(result) {\n    /**\n     * @return {Boolean} - result is true if documents were indexed succesfully\n     */\n}).catch(function error(result) {\n    /* Handle error here */\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### ReIndex\n\nThis methods hits Elastic\n\n```javascript\nncDbHandler.reindex({\n    dataSource: 'reviews.reviews', /* {String} | required | format: \"index.type\" */\n    logQuery: true, /* {Boolean} | optional | default: false */\n    query: { /* {Object} | required */\n       'id' : 'UUID' /* this is the only accepted format for updating */\n    },\n    payload: { /* {Array} | required */\n       tags: ['red']\n    },\n    payloadSchema: { /* {Object} | optional | Joi validation schema applied on the result of the mapping */\n       tags: Joi.....\n    },\n    replaceData: false /* {Boolean} | optional | defaults: false */\n}).then(function success(result) {\n    /**\n     * {Boolean} - result is true if documents were updated succesfully\n     */\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### UnIndex\n\nThis methods hits Elastic\n\n```javascript\nncDbHandler.unindex({\n    dataSource: 'reviews.reviews' /* {String} | required */\n    logQuery : true, /* {Boolean} | optional | default: false */\n    query: { /* {Object} | optional | at least one between query and payload is mandatory */\n        {\n          term: {\n            status: 'normal'\n          }\n        }\n    },\n    payload: ['UUID1', 'UUID2', 'UUID3'] /* {Array} | optional | an array of ES document ids */\n}).then(function(response) {\n    /**\n     * {Boolean} - result is true if documents were removed succesfully\n     */\n}).catch(function(error) {\n    // Handle errors throwns by method\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n#### ES\n\nThis methods hits Elastic\n\n```javascript\nncDbHandler.es(\n    'reviews', /* {String} connectionName - the key of connection string used in config['read-store'] */\n    'index', /* {String} action - elastic search action ('bulk', 'index', 'count', 'create', 'delete', 'deleteByQuery', 'get', 'search', 'update') */\n    { /* {Object} query - raw elastic search query to be sent */\n        index: 'reviews',\n        type : 'review',\n        id   : 'UUID',\n        body : {\n            tags: [\"red\"],\n            user: 'Fred Jacobs'\n        }\n    }).then(function(response) {\n    /**\n     * @return {Object} - raw response from elastic search\n     */\n}).catch(function(error) {\n    // Handle errors throws by method\n});\n```\n\n[back to Method Summary](#nc-db-handler)\n\n# Development\nOn your machine, we can initialize some data store dependencies to facilitate integration test.\n\n## Mocha Integration Test\nCreates a docker compose environment with Elasticsearch and Postgres with a container testing code in watch mode.\n```Text\nmake run\n```\n\n## Install Packages\nRemoves node_modules folder and reinstalls from shrinkwrap or package.json\n```Text\nmake install\n```\n\n## Create New Shrinkwrap From Scratch\nRemoves node_modules folder and npm_shrinkwrap.json and recreates from package.json using the latest from npm.\n```Text\nmake clean\n```\n\n## Test with Mocha\nInitiate mocha in watch mode on your local machine.  This will not boot up any docker resources so your integration test will fail.  But if you have postgres and elasticsearch containers running and hosted correctly, then these test will work.\n```Text\nmake test\n```\n\n# Maintenance\n\n### Docker Build\nDocker image build process for containerizing code and test.\n```Text\nmake docker-build\n```\n\n[back to top](#nc-db-handler)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxeeo%2Fnc-db-handler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxeeo%2Fnc-db-handler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxeeo%2Fnc-db-handler/lists"}