{"id":19622145,"url":"https://github.com/commercetools/sphere-node-utils","last_synced_at":"2025-04-28T03:32:23.574Z","repository":{"id":14314876,"uuid":"17023888","full_name":"commercetools/sphere-node-utils","owner":"commercetools","description":"Utils shared among all sphere node components","archived":false,"fork":false,"pushed_at":"2023-04-15T17:47:55.000Z","size":454,"stargazers_count":2,"open_issues_count":15,"forks_count":3,"subscribers_count":26,"default_branch":"master","last_synced_at":"2025-04-20T15:17:59.570Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"CoffeeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/commercetools.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-02-20T14:55:43.000Z","updated_at":"2021-10-13T07:28:13.000Z","dependencies_parsed_at":"2024-06-18T22:50:28.334Z","dependency_job_id":"6012b74d-4086-4eba-bb1b-8892ae5db116","html_url":"https://github.com/commercetools/sphere-node-utils","commit_stats":{"total_commits":319,"total_committers":20,"mean_commits":15.95,"dds":0.6614420062695925,"last_synced_commit":"d7860e7a1b3fc4d541aa276c64ad47d53b1edeb8"},"previous_names":["sphereio/sphere-node-utils"],"tags_count":50,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commercetools%2Fsphere-node-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commercetools%2Fsphere-node-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commercetools%2Fsphere-node-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commercetools%2Fsphere-node-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/commercetools","download_url":"https://codeload.github.com/commercetools/sphere-node-utils/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251246339,"owners_count":21558762,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T11:26:11.698Z","updated_at":"2025-04-28T03:32:22.977Z","avatar_url":"https://github.com/commercetools.png","language":"CoffeeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"![commercetools logo](https://cdn.rawgit.com/commercetools/press-kit/master/PNG/72DPI/CT%20logo%20horizontal%20RGB%2072dpi.png)\n\n# Node.js Utils\n\n[![NPM](https://nodei.co/npm/sphere-node-utils.png?downloads=true)](https://www.npmjs.org/package/sphere-node-utils)\n\n[![Build Status](https://secure.travis-ci.org/sphereio/sphere-node-utils.png?branch=master)](http://travis-ci.org/sphereio/sphere-node-utils) [![NPM version](https://badge.fury.io/js/sphere-node-utils.png)](http://badge.fury.io/js/sphere-node-utils) [![Coverage Status](https://coveralls.io/repos/sphereio/sphere-node-utils/badge.png)](https://coveralls.io/r/sphereio/sphere-node-utils) [![Dependency Status](https://david-dm.org/sphereio/sphere-node-utils.png?theme=shields.io)](https://david-dm.org/sphereio/sphere-node-utils) [![devDependency Status](https://david-dm.org/sphereio/sphere-node-utils/dev-status.png?theme=shields.io)](https://david-dm.org/sphereio/sphere-node-utils#info=devDependencies)\n\nThis module shares helpers among all [commercetools](https://commercetools.com/) Node.js components.\n\n## Table of Contents\n* [Getting Started](#getting-started)\n* [Documentation](#documentation)\n  * [Helpers](#helpers)\n    * [Logger](#logger)\n    * [ExtendedLogger](#extendedlogger)\n    * [TaskQueue](#taskqueue)\n    * [Sftp](#sftp)\n    * [ProjectCredentialsConfig](#projectcredentialsconfig)\n    * [Repeater](#repeater)\n    * [ElasticIo](#elasticio)\n    * [UserAgent](#useragent)\n  * [Mixins](#mixins)\n    * [Qutils](#qutils)\n* [Examples](#examples)\n* [Releasing](#releasing)\n* [License](#license)\n\n\n## Getting Started\n\n```coffeescript\nSphereUtils = require 'sphere-node-utils'\nLogger = SphereUtils.Logger\nTaskQueue = SphereUtils.TaskQueue\n...\n\n# or\n{Logger, TaskQueue, ...} = require 'sphere-node-utils'\n```\n\n## Documentation\n\n### Helpers\nCurrently following helpers are provided by `SphereUtils`:\n\n- `Logger`\n- `ExtendedLogger`\n- `TaskQueue`\n- `Sftp`\n- `ProjectCredentialsConfig`\n- `ElasticIo`\n- `userAgent`\n- `Repeater`\n\n#### Logger\nLogging is supported by the lightweight JSON logging module called [Bunyan](https://github.com/trentm/node-bunyan).\n\nThe `Logger` can be configured with following options\n```coffeescript\nlogConfig:\n  levelStream: 'info' # log level for stdout stream\n  levelFile: 'debug' # log level for file stream\n  path: './sphere-node-utils-debug.log' # where to write the file stream\n  name: 'sphere-node-utils' # name of the application\n  serializers:\n    request: reqSerializer # function that maps the request object with fields (uri, method, headers)\n    response: resSerializer # function that maps the response object with fields (status, headers, body)\n  src: false # includes a log of the call source location (file, line, function).\n             # Determining the source call is slow, therefor it's recommended not to enable this on production.\n  silent: false # don't instantiate the {Bunyan} logger, instead use `console`\n  streams: [ # a list of streams that defines the type of output for log messages\n    {level: 'info', stream: process.stdout}\n    {level: 'debug', path: './sphere-node-utils-debug.log'}\n  ]\n```\n\n\u003e A `Logger` instance should be extended by the component that needs logging by providing the correct configuration\n\n```coffeescript\n{Logger} = require 'sphere-node-utils'\n\nmodule.exports = class extends Logger\n\n  # we can override here some of the configuration options\n  @appName: 'my-application-name'\n  @path: './my-application-name.log'\n```\n\nA `Bunyan` logger can also be created from another existing logger. This is useful to connect sub-components of the same application by sharing the same configuration.\nThis concept is called **[child logger](https://github.com/trentm/node-bunyan#logchild)**.\n\n```coffeescript\n{Logger} = require 'sphere-node-utils'\nclass MyCustomLogger extends Logger\n  @appName: 'my-application-name'\n\nmyLogger = new MyCustomLogger logConfig\n\n# assume we have a component which already implements logging\nappWithLogger = new AppWithLogger\n  logConfig:\n    logger: myLogger\n\n# now we can use `myLogger` to log and everything logged from the child logger of `AppWithLogger`\n# will be logged with a `widget_type` field, meaning the log comes from the child logger\n```\n\nOnce you configure your logger, you will get JSON stream of logs based on the level you defined. This is great for processing, but not for really human-friendly.\nThis is where the `bunyan` command-line tool comes in handy, by providing **pretty-printed** logs and **filtering**. More info [here](https://github.com/trentm/node-bunyan#cli-usage).\n\n```bash\n# examples\n\n# this will output the content of the log file in a `short` format\nbunyan sphere-node-connect-debug.log -o short\n00:31:47.760Z  INFO sphere-node-connect: Retrieving access_token...\n00:31:48.232Z  INFO sphere-node-connect: GET /products\n\n# directly pipe the stdout stream\njasmine-node --verbose --captureExceptions test | ./node_modules/bunyan/bin/bunyan -o short\n00:34:03.936Z DEBUG sphere-node-connect: OAuth constructor initialized. (host=auth.sphere.io, accessTokenUrl=/oauth/token, timeout=20000, rejectUnauthorized=true)\n    config: {\n      \"client_id\": \"S6AD07quPeeTfRoOHXdTx2NZ\",\n      \"client_secret\": \"7d3xSWTN5jQJNpnRnMLd4qICmfahGPka\",\n      \"project_key\": \"my-project\",\n      \"oauth_host\": \"auth.sphere.io\",\n      \"api_host\": \"api.sphere.io\"\n    }\n00:34:03.933Z DEBUG sphere-node-connect: Failed to retrieve access_token, retrying...1\n\n```\n\n##### Silent logs, use `console`\nYou can pass a `silent` flag to override the level functions of the `bunyan` logger (debug, info, ...) to print to stdout / stderr using console.\n\n#### ExtendedLogger\nAn `ExtendedLogger` allows you to wrap additional fields to the logged JSON object, by either defining them on class instantiation or by chaining them before calling the log level method.\n\n\u003e Under the hood it uses the [Logger](#logger) `Bunyan` object\n\n```coffeescript\nlogger = new ExtendedLogger\n  additionalFields:\n    project_key: 'foo'\n    another_field: 'bar'\n  logConfig: # see config above (Logger)\n    streams: [\n      { level: 'info', stream: process.stdout }\n    ]\n\n# then use the logger as usual\n\nlogger.info {id: 123}, 'Hello world'\n# =\u003e {\"name\":\"sphere-node-utils\",\"hostname\":\"Nicolas-MacBook-Pro.local\",\"pid\":25856,\"level\":30,\"id\":123,\"project_key\":\"foo\",\"another_field\":\"bar\",\"msg\":\"Hello world\",\"time\":\"2014-04-17T10:54:05.237Z\",\"v\":0}\n\n# or by chaining\n\nlogger.withField({token: 'qwerty'}).info {id: 123}, 'Hello world'\n# =\u003e {\"name\":\"sphere-node-utils\",\"hostname\":\"Nicolas-MacBook-Pro.local\",\"pid\":25856,\"level\":30,\"id\":123,\"project_key\":\"foo\",\"another_field\":\"bar\", \"token\": \"qwerty\",\"msg\":\"Hello world\",\"time\":\"2014-04-17T10:54:05.237Z\",\"v\":0}\n```\n\n#### TaskQueue\nA `TaskQueue` allows you to queue promises (or function that return promises) which will be executed in parallel sequentially, meaning that new tasks will not be triggered until the previous ones are resolved.\n\n```coffeescript\n{TaskQueue} = require 'sphere-node-utils'\n\ncallMe = -\u003e\n  new Promise (resolve, reject) -\u003e\n    setTimeout -\u003e\n      resolve true\n    , 5000\ntask = new TaskQueue maxParallel: 50 # default 20\ntask.addTask callMe\n.then (result) -\u003e # result == true\n.catch (error) -\u003e\n```\n```coffeescript\n# adding and executing multiple tasks\ncallMe(name) = -\u003e\n  new Promise (resolve, reject) -\u003e\n    setTimeout -\u003e\n      resolve name\n    , 5000\ntaskQueue = new TaskQueue\ntask1Promise = taskQueue.addTask(() -\u003e callMe('task1'))\ntask2Promise = taskQueue.addTask(() -\u003e callMe('task2'))\n\nPromise.all([task1Promise, task2Promise])\n.then (result) -\u003e # result == ['task1', 'task2']\n.catch (error) -\u003e\n```\n\nAvailable methods:\n- `setMaxParallel` sets the `maxParallel` parameter (default is `20`). **If \u003c 1 or \u003e 100 it throws an error**\n- `addTask` adds a task (promise) to the queue and returns a new promise\n\n#### Sftp\nProvides promised based wrapped functionalities for some `SFTP` methods\n\n- `listFiles`\n- `stats`\n- `readFile` _not implemented yet_\n- `saveFile` _not implemented yet_\n- `getFile`\n- `putFile`\n- `safePutFile`\n- `renameFile`\n- `safeRenameFile`\n- `removeFile`\n- `openSftp`\n- `close`\n- `downloadAllFiles`\n\n\u003e The client using the `Sftp` helper should take care of how to send requests to manage remote files, by controlling e.g. concurrency via `TaskQueue` and/or functions of `Bluebird` promise [API](https://github.com/petkaantonov/bluebird/blob/master/API.md)\n\n\n#### ProjectCredentialsConfig\nAllows to read SPHERE.IO credentials from a file or via environment variables, based on the `project_key`.\n\n##### From a file\n\nBy default the module will try to read the credentials from the following locations (descending priority):\n\n* ./.sphere-project-credentials\n* ./.sphere-project-credentials.json\n* ~/.sphere-project-credentials\n* ~/.sphere-project-credentials.json\n* /etc/sphere-project-credentials\n* /etc/sphere-project-credentials.json\n\nThe versions of these without the `.json` extension consist of a series of lines, each of which contains a project key, client ID and client secret, separated by colons:\n\n```\n\u003cproject-key1\u003e:\u003cclient-id1\u003e:\u003cclient-secret1\u003e\n\u003cproject-key2\u003e:\u003cclient-id2\u003e:\u003cclient-secret2\u003e\n```\n\nThe JSON versions are structured as follows:\n\n```\n{\n  \"\u003cproject-key1\u003e\": {\n    \"client_id\":\"\u003cclient-id1\u003e\",\n    \"client_secret\":\"\u003cclient-secret1\u003e\"\n  },\n  \"\u003cproject-key2\u003e\": {\n    \"client_id\":\"\u003cclient-id2\u003e\",\n    \"client_secret\":\"\u003cclient-secret2\u003e\"\n  }\n}\n```\n\nExample usage:\n```js\nimport { SphereClient } from 'sphere-node-sdk'\nimport { ProjectCredentialsConfig } from 'sphere-node-utils'\n\nconst PROJECT_KEY = 'your-project-key'\n\nProjectCredentialsConfig.create()\n.then((credentials) =\u003e{\n  const sphereCredentials = credentials.enrichCredentials({\n   project_key: PROJECT_KEY,\n   // you can pass some fallback options as well here\n   client_id: argv.clientId,\n   client_secret: argv.clientSecret,\n  })\n  // got the credentials\n  // do something with them e.g. initialize the SphereClient from the node-sdk\n  const sphereClient = new SphereClient({ config: sphereCredentials })\n})\n```\n\n##### From environment variables\n\nThis is a little bit more restricted, since you can only define one set of credentials with the environment variables. Nevertheless this is very useful for deployments, where you really only need one set of credentials per deployment.\nYou can define your credentials using these variables:\n\n```sh\nexport SPHERE_PROJECT_KEY=\"your-project-key\"\nexport SPHERE_CLIENT_ID=\"your-client-id\"\nexport SPHERE_CLIENT_SECRET=\"your-client-secret\"\n```\n\n#### ElasticIo\n_(Coming soon)_\n\n#### UserAgent\nA synchronous module that builds _user\\_agent_ according to the standard specified by commercetools\n\nsphere-node-sdk module must be installed in the node_modules because it's required to build the user_agent\n\n```js\nconst user_agent = userAgent('sphere-node-utils', '1.0.0')\n```\n\nExample of returned user_agent\n```\n1.16.2 Node.js/v6.5.0 (darwin; x64) sphere-node-utils/0.8.6\n```\n\n#### Repeater\nA Repeater allows to execute a promise function and recover from it in case of errors, for a certain number of times before giving up.\n\n\u003e By default the initial task will be retried unless a new task is returned from the recover function (see example below).\n\nThe only method exposed is `execute`, which accepts 2 arguments:\n- task: a `Function` that returns a Promise\n- recover: a `Function` that returns a Promise, called when the task fails\n\nFollowing options are supported when initializing a new `Repeater`:\n- `attempts` (Int) how many times the task should be repeated, if failed, before giving up (**default 10**)\n- `timeout` (Int) the delay between attempts before retrying, in `ms` (**default 100**)\n- `timeoutType` (String) the type of the timeout (**default c**)\n  - `c` _constant delay_ always the same timeout\n  - `v` _variable delay_ timeout grows with the attempts count (also using a random component)\n\n```coffeescript\nclient = new SphereClient {...}\nrepeater = new Repeater {attempts: 10}\n\nupdateTask = (payload) -\u003e client.products.byId(productId).update(payload)\nrepeater.execute -\u003e\n  updateTask(payload)\n, (e) -\u003e\n  if e.statusCode is 409\n    # this means a concurrent modification, so we retry to update with\n    # a new task by retrieving a new product version first\n    newTask = -\u003e # task must be a function\n      client.productProjections.staged(true).byId(productId).fetch()\n      .then (result) -\u003e\n        newPayload = _.extend payload, {version: result.body.version}\n        updateTask(newPayload)\n    # now we must resolve the recover function with the new task\n    # If we just want to retry the initial task then simply resolve an empty promise\n    # Promise.resolve()\n    Promise.resolve newTask\n  else\n    # we should not retry in this case, so simply bubble up the error\n    Promise.reject e\n```\n\n### Mixins\nCurrently following mixins are provided by `SphereUtils`:\n\n- `Qutils`\n  - `processList`\n\n#### Qutils\n\u003e Deprecated in favour of `Bluebird` promises\n\nA collections of Q utils (promise-based)\n\n```coffeescript\n{Qutils} = require 'sphere-node-utils'\n```\n\n##### `processList`\nProcess each element in the given list using the function `fn` (called on each iteration).\nThe function `fn` has to return a promise that should be resolved when all elements of the page are processed.\n\n```coffeescript\nlist = [{key: '1'}, {key: '2'}, {key: '3'}]\nprocessList list, (elems) -\u003e # elems is an array\n  doSomethingWith(elems) # it's a promise\n  .then -\u003e\n    # something else\n    anotherPromise()\n```\n\n\u003e Note that the argument passed to the process function is always an array, containing a number of elements defined by `maxParallel` option\n\nYou can pass some options as second argument:\n- `accumulate` whether the results should be accumulated or not (default `true`). If not, an empty array will be returned from the resolved promise.\n- `maxParallel` how many elements from the list will be passed to the process `fn` function (default `1`)\n\n\n## Examples\n_(Coming soon)_\n\n## Contributing\nIn lieu of a formal styleguide, take care to maintain the existing coding style. Add unit tests for any new or changed functionality. Lint and test your code using [Grunt](http://gruntjs.com/).\nMore info [here](CONTRIBUTING.md)\n\n## Releasing\nReleasing a new version is completely automated using the Grunt task `grunt release`.\n\n```javascript\ngrunt release // patch release\ngrunt release:minor // minor release\ngrunt release:major // major release\n```\n\n## License\nCopyright (c) 2014 SPHERE.IO\nLicensed under the [MIT license](LICENSE-MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcommercetools%2Fsphere-node-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcommercetools%2Fsphere-node-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcommercetools%2Fsphere-node-utils/lists"}