{"id":20445877,"url":"https://github.com/rapid7/r7insight_node","last_synced_at":"2025-04-13T05:36:19.610Z","repository":{"id":38416797,"uuid":"86822685","full_name":"rapid7/r7insight_node","owner":"rapid7","description":"node logging support for InsightOps","archived":false,"fork":false,"pushed_at":"2024-11-06T08:54:38.000Z","size":308,"stargazers_count":7,"open_issues_count":5,"forks_count":13,"subscribers_count":44,"default_branch":"master","last_synced_at":"2024-12-18T06:23:47.331Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rapid7.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-31T13:34:38.000Z","updated_at":"2024-11-06T08:54:42.000Z","dependencies_parsed_at":"2023-02-08T03:46:52.636Z","dependency_job_id":"817ea25b-b2ce-41db-8eaf-b32c2d8529be","html_url":"https://github.com/rapid7/r7insight_node","commit_stats":{"total_commits":40,"total_committers":12,"mean_commits":"3.3333333333333335","dds":0.575,"last_synced_commit":"6368d0915d700df4e583e82100365c87e5931d81"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rapid7%2Fr7insight_node","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rapid7%2Fr7insight_node/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rapid7%2Fr7insight_node/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rapid7%2Fr7insight_node/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rapid7","download_url":"https://codeload.github.com/rapid7/r7insight_node/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248670509,"owners_count":21142897,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T10:15:11.006Z","updated_at":"2025-04-13T05:36:19.588Z","avatar_url":"https://github.com/rapid7.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# r7insight_node: Insight Platform Client\n\nAllows you to send logs to the [Insight Platform](https://www.rapid7.com/products/)\n(or Logentries) account from Node.js.\n\nThis client is not backwards-compatible with [Le_Node](https://github.com/rapid7/le_node).  \nAn upgrade guide can be found on [the wiki](https://github.com/rapid7/r7insight_node/wiki/2.0-Migration-Guide).  \n\nThere’s a separate client intended for use in the browser, called\n[r7insight_js](https://github.com/rapid7/r7insight_js), which uses http and is optimized\nfor browser-specific logging needs.\n\n- [r7insight_node: Insight Platform Client](#r7insight_node-insight-platform-client)\n  - [Start](#start)\n  - [Development](#development)\n  - [Options](#options)\n    - [Required](#required)\n    - [Behavior](#behavior)\n    - [Log Processing Options`](#log-processing-options)\n    - [Other](#other)\n  - [Log Levels](#log-levels)\n  - [Events](#events)\n    - [Logger Events](#logger-events)\n      - [`'error'`](#error)\n      - [`'log'`](#log)\n      - [`'connected'` and `'disconnected'` and `'timed out'`](#connected-and-disconnected-and-timed-out)\n      - [`'drain'`, `'finish'`, `'pipe'`, and `'unpipe'`](#drain-finish-pipe-and-unpipe)\n      - [`'buffer drain'`](#buffer-drain)\n    - [RingBuffer Events](#ringbuffer-events)\n      - [`'buffer shift'`](#buffer-shift)\n  - [Log Entries](#log-entries)\n    - [Object Serialization](#object-serialization)\n    - [Optional Augmentation](#optional-augmentation)\n    - [Flattening Log Objects](#flattening-log-objects)\n  - [Methods](#methods)\n  - [Buffering](#buffering)\n  - [Connection Handling](#connection-handling)\n  - [Using as a Winston ‘Transport’](#using-as-a-winston-transport)\n    - [Pre-requisites](#pre-requisites)\n    - [Code Example](#code-example)\n  - [Using with Bunyan](#using-with-bunyan)\n  - [Using with Ts.ED Logger](#using-with-tsed-logger)\n\n## Start\n\n```javascript\nconst Logger = require('r7insight_node');\n\nconst logger = new Logger({ token: '\u003ctoken\u003e' , region: '\u003cregion\u003e'});\n\nlogger.warning(\"I'll put this over here, with the rest of the fire.\");\n```\n\n## Development\n\nWorkflow is as follows:\n- Fork the repository\n- Clone it locally\n- `npm install` for installing the packages\n- Add unit testing for desired functionality - `npm test` for testing\n- Work on functionality\n- Bump relevant version when finished `npm version [major|minor|patch]`\n- Update typescript definitions if necessary\n  * ```bash\n    # You can simply paste all this into your terminal\n    npm uninstall -g r7insight_node\n    npm pack\n    npm i -g r7insight_node-*.tgz\n    npm install -g dts-gen\n    dts-gen -m r7insight_node -f index.d.ts -o\n    ```\n- Push and open a pull request\n\n## Options\n\nThe options object you provide to the constructor only requires your access\ntoken, but you can configure its behavior further.\n\nAll of the following except `token`, `levels` and `secure` can also be\nconfigured after instantiation as settable properties on the client. They are\naccessors, though, and invalid values will be ignored.\n\n### Required\n\n - **token:** String. Log token for the Rapid7 Insight Platform.\n - **region**: The region of ingestion endpoint to be used. Examples: `eu`, `us` etc.\n\n### Behavior\n - **console:** If truthy, log events also get sent to `console.log`,\n   `console.warn` and `console.error` as appropriate. Default: `false`.\n - **levels**: Custom names for the 8 log levels and their corresponding\n   methods. More details on this below.\n - **minLevel**: The minimum level to actually record logs at. String or Number.\n   Defaults to 0.\n - **takeLevelFromLog**: If truthy, will take log message level from message. Default: `false`.  \n   E.g. if `true`:\n   ```javascript\n   // Rather than call different functions based on level:\n   logger.warn({message: 'hello'});\n   // You can call the same function with different levels within object:\n   logger.log({level: 'warn', message: 'hello'});\n   ```\n - **bufferSize**: The maximum number of log entries that may be queued in the \n   internal ring buffer for sending at a given moment. Default: `16192`.\n - **secure:** If truthy, uses a TLS connection. Default: `true`.\n - **inactivityTimeout:** The time, in milliseconds, that inactivity should warrant\n   closing the connection to the host until needed again. Defaults to 15 seconds.\n - **disableTimeout**: Sets the socket timeout to 0. Should not be used with \n   inactivityTimeout option.\n - **reconnectInitialDelay**: Initial wait time in milliseconds while reconnecting. \n   Default: `1000`\n - **reconnectMaxDelay**: Maximum wait time in milliseconds while reconnecting.\n   Default: `15 * 1000`\n - **reconnectBackoffStrategy**: Backoff strategy to be used while trying to reconnect.\n   It can be either `fibonacci` or `exponential`. Default: `fibonnacci`   \n\n\n### Log Processing Options\n - **flatten**: Convert objects into a single-level object where the values of\n   interior objects become dot-notation properties of the root object. Defaults\n   to `false`. More details on this below.\n - **flattenArrays**: If `flatten` is true, you can also indicate whether arrays\n   should be subject to the same process. Defaults to `true` if `flatten` is\n   `true`; otherwise meaningless.\n - **replacer**: A custom value-transform function to be used during JSON\n   serialization. Applied before error transformation.\n - **timestamp**: If truthy, prefix entries with an ISO timestamp (if strings)\n   or add the same as a property (if objects). Default: `false`.\n - **withLevel**: Will prepend (string) or add property (object) indicating the\n   log level. Default: `true`.\n - **withHostname**: Will prepend(string) or add property (object) indicating the \n   hostname from which the log was sent. Default: `false`.\n - **withStack**: If an object is or contains an `Error` object, setting this to\n   `true` will cause the stack trace to be included. Default: `false.`\n\n### Other\n - **host**: Optional host to send logs to. Normally you would not want to set this,\n   but it may be useful for mocking during tests. The value may be just the host\n   or the host with the port specified.\n - **port**: As above. This will default to 80 if `secure` is false, or 443 if\n   it’s true.\n - **debug**: Setting this to `true` will enable debug logging with a default stdout\n  logger.\n - **debugLogger**: Use this to override default stdout logger. New logger must\n  implement a `log` method.\n\n## Log Levels\n\nThe default log levels are:\n\n 0. debug\n 1. info\n 2. notice\n 3. warning\n 4. err\n 5. crit\n 6. alert\n 7. emerg\n\nYou can provision the constructor with custom names for these levels with either\nan array or an object hash:\n\n```javascript\n[ 'boring', 'yawn', 'eh', 'hey' ]\n\n{ boring: 0, yawn: 1, eh: 2, hey: 3 }\n```\n\nIn the former case, the index corresponds to the numeric level, so sparse arrays\nare valid. In either case, missing levels will be filled in with the defaults.\n\nThe `minLevel` option respects either level number (e.g. `2`) or the name (e.g.\n`'eh'`).\n\nThe level names each become methods on the client, which are just sugar for\ncalling `client.log(lvl, logentry)` with the first argument curried.\n\nSince these names will appear on the client, they can’t collide with existing\nproperties. Not that you’re particularly likely to try naming a log level\n‘hasOwnProperty’ or ‘_write’ but I figured I should mention it.\n\nSo the following three are equivalent:\n\n```javascript\nlogger.notice('my msg');\nlogger.log('notice', 'my msg');\nlogger.log(2, 'my msg');\n```\n\nIt’s also possible to forgo log levels altogether. Just call `log` with a single\nargument and it will be interpreted as the log entry. When used this way, the\n`minLevel` setting is ignored.\n\n## Events\n\n### Logger Events\n\nThese events are also exported in the `Logger`, so you can access them using `Logger.errorEvent`, `Logger.bufferDrainEvent` etc. Example:\n\n```javascript\n   logger.notice({ type: 'server', event: 'shutdown' });\n   logger.once(Logger.bufferDrainEvent, () =\u003e {\n      logger.closeConnection();\n      logger.on(Logger.disconnectedEvent, () =\u003e {\n        process.exit();\n      });\n   });\n```\n\n#### `'error'`\nThe client is an EventEmitter, so you should (as always) make sure you have a\nlistener on `'error'`. Error events can occur when there’s been a problem with\nthe connection or if a method was called with invalid parameters. Note that\nerrors that occur during instantiation, as opposed to operation, will **throw**.\n\n#### `'log'`\nTriggered when a log is about to be written to the underlying connection. The\nprepared log object or string is supplied as an argument.\n\n#### `'connected'` and `'disconnected'` and `'timed out'` \nThese indicate when a new connection to the host is established, destroyed or \ntimed out due to client side inactivity. Inactivity timeout is normal if the connection \nis inactive for a configurable period of time (see inactivityTimeout); it will \nbe reopened when needed again. Disconnection can be either a result of socket inactivity or a network failure.\n\n#### `'drain'`, `'finish'`, `'pipe'`, and `'unpipe'`\nThese are events inherited from `Writable`.\n\n#### `'buffer drain'`\nThis event is emitted when the underlying ring buffer is fully consumed and Socket.write callback called.\nThis can be useful when it’s time for the application to terminate but you want\nto be sure any pending logs have finished writing.\n\n```javascript\n   logger.notice({ type: 'server', event: 'shutdown' });\n   logger.once('buffer drain', () =\u003e {\n      logger.closeConnection();\n      logger.on('disconnected', () =\u003e {\n        process.exit();\n      });\n   });\n```\n\n### RingBuffer Events\n\n#### `'buffer shift'`\n\nBuffer shift event is emitted when the internal buffer is shifted due to reaching `bufferSize`\nof events in the buffer. This event may be listened for security/operations related reasons as\neach time this event is emitted, a log event will be discarded and discarded log event will\nnever make it to the Insight Platform.\n\n```javascript\nlogger.ringBuffer.on('buffer shift', () =\u003e {\n    // PagerDuty or send an email \n});\n```\n\n## Log Entries\n\nLog entries can be strings or objects. If the log argument is an array, it will\nbe interpretted as multiple log events.\n\n### Object Serialization\n\nIn the case of objects, the native JSON.stringify serialization is augmented in\nseveral ways. In addition to handling circular references, it will automatically\ntake care of a variety of objects and primitives which otherwise wouldn’t\nserialize correctly, like Error, RegExp, Set, Map, Infinity, NaN, etc.\n\nIf you choose to set `withStack` to true, errors will include their stacktraces\nas an array (so that they are not painful to look at). Be sure to turn on\n\"expand JSON\" (meaning pretty print) in the options in the Insight Platform:\n\n![stack trace as seen in logentries app][screen1]\n\nYou can adjust this further by supplying your own custom `replacer`. This is a\nstandard argument to JSON.stringify -- See [MDN: JSON \u003e Stringify \u003e The Replacer Parameter](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#The_replacer_parameter)\nfor details. In the event that you supply a custom replacer, it is applied\nprior to the built-in replacer described above so you can override its behavior.\n\n### Optional Augmentation\n\nTwo options are available, `timestamp` and `withLevel`, which will add data to\nyour log events. For objects, these are added as properties (non-mutatively).\nFor strings, these values are prepended. If the name of a property would cause\na collision with an existing property, it will be prepended with an underscore.\n\n### Flattening Log Objects\n\nIn some cases it will end up being easier to query your data if objects aren’t\ndeeply nested. With the `flatten` and `flattenArrays` options, you can tell the\nclient to transform objects like so:\n\n  * `{ \"a\": 1, \"b\": { \"c\": 2 } }` =\u003e `{ \"a\": 1, \"b.c\": 2 }`\n\nIf `flattenArrays` has not been set to false, this transformation will apply to\narrays as well:\n\n  * `{ \"a\": [ \"b\", { \"c\": 3 } ] }` =\u003e `{ \"a.0\": \"b\", \"a.1.c\": 3 }`\n\n## Methods\n\nIn addition to `log` and its arbitrary sugary cousins, you can call\n`closeConnection` to explicitly close an open connection if one exists; you\nmight wish to do this as part of a graceful exit. The connection will reopen if\nyou log further.\n\nAlso, because the client is actually a writable stream, you can call `write`\ndirectly. This gives you lower-level access to writing entries. It is in object\nmode, but this means it expects discreet units (one call = one entry), not\nactual objects; you should pass in strings. This is useful if you want to pipe\nstdout, for example.\n\n## Buffering\n\nIf there’s a problem with the connection (network loss or congestion),\nentries will be buffered in an internal ring buffer to a max of 16192(`bufferSize`)\nentries by default. After that, internal ring buffer will `shift` records\nto keep only last `bufferSize` number of records in memory. A log that indicates the\nbuffer was full will be sent to internal logger \"once\" this happens.\nIf `console` is true, these log entries will still display there, but they will\nnot make it to the Insight Platform.\n\nYou can adjust the maximum size of the buffer with the `bufferSize` option.\nYou’ll want to raise it if you’re dealing with very high volume (either a high\nnumber of logs per second, or when log entries are unusually long on average).\nOutside of these situations, exceeding the max buffer size is more likely an\nindication of creating logs in a synchronous loop (which seems like a bad idea).\n\n## Connection Handling\n\nIf the connection fails, it will keep retrying with a `fibonacci` backoff by default. \nConnection retry will start with a delay of `reconnectInitialDelay` and the delay between each retry \nwill go up to a maximum of `reconnectMaxDelay` with each retry in fibonacci sequence. \nBackoff strategy can be changed to `exponential` through constructor if necessary.\n\nA connection to the host does not guarantee that your logs are transmitting\nsuccessfully. If you have a bad token, there is no feedback from the server to\nindicate this. The only way to confirm that your token is working is to check\nthe live tail in InsightOps. I will investigate this further to see if there’s\nsome other means with which a token can be tested for validity.\n\n## Using as a Winston ‘Transport’\n\n### Pre-requisites\n\n- `winston` and `winston-transport` installed\n  * Check [package.json](./package.json) for currently supported version\n\n### Code Example\n\n```javascript\nconst winston = require('winston');\n\n// If Winston is included in your package.json dependencies,\n// you can just require the Insight Logger\n// to initialize it.\nrequire('r7insight_node');\n\nconst token = '00112233-4455-6677-8899-aabbccddeeff';\nconst transports = [];\n\ntransports.push(\n\tnew winston.transports.Console({\n\t\tformat: winston.format.simple(),\n\t\tlevel: 'debug',\n\t})\n);\n\ntransports.push(\n\tnew winston.transports.Insight({\n\t\ttoken,\n\t\tregion: 'eu',\n\t\tlevel: 'debug',\n\t})\n);\n\nconst logger = winston.createLogger({\n\ttransports,\n});\n\nlogger.info('hello there');\n```\n\n\nThe Insight client will place the transport constructor at `winston.transports`,\neven if Winston itself hasn’t yet been required.\n\n```javascript\nconst Logger = require('r7insight_node');\nconst winston = require('winston');\n\nassert(winston.transports.Insight);\n```\n\n- Winston is an optional dependency in `r7insight_node` and and if included it\nrequires `winston-transport` for the `InsightTransport` to extend it.\n\n- When adding a new Insight transport, the options argument passed to Winston’s\n`add` method supports the usual options in addition to those which are Winston-\nspecific. If custom levels are not provided, Winston’s defaults will be used.\n\n```javascript\nwinston.add(new winston.transports.Insight({ token: '\u003ctoken\u003e', region: '\u003cregion\u003e' }));\n```\n\n- In the hard-to-imagine case where you’re using Winston without including it in\npackage.json, you can explicitly provision the transport by first requiring\nWinston and then importing and calling `provisionWinston` like this:\n```javascript\nconst winston = require('winston');\n\nconst Logger = require('r7insight_node');\n\nLogger.provisionWinston();\n```\n\n## Using with Bunyan\n\nFor Bunyan it’s like so:\n\n```javascript\nconst bunyan = require('bunyan');\nconst Logger = require('r7insight_node');\n\nconst loggerDefinition = Logger.bunyanStream({ token: '\u003ctoken', region: '\u003cregion\u003e' });\n\n// One stream\nconst logger1 = bunyan.createLogger(loggerDefinition);\n\n// Multiple streams\nconst logger2 = bunyan.createLogger({\n\tname: 'my leg',\n\tstreams: [ loggerDefinition, otherLoggerDefinition ]\n});\n```\n\nAs with Winston, the options argument takes the normal constructor options (with\nthe exception of `timestamp`, which is an option you should set on Bunyan itself\ninstead). Bunyan uses six log levels, so the seventh and eighth, if provided,\nwill be ignored; by default Bunyan’s level names will be used.\n\nThe object returned by `bunyanStream` is the Bunyan logging ‘channel’ definition\nin total. If you want to futz with this you can -- you can change its `name` or\nget the `stream` object itself from here.\n\n\n## Using with Ts.ED Logger\n\nFor Ts.ED logger it's like so:\n\n```typescript\nimport {Logger} from \"@tsed/logger\";\nimport \"@tsed/logger-insight\";\n\nconst logger = new Logger(\"loggerName\");\n\nlogger.appenders.set(\"stdout\", {\n  type: \"insight\",\n  level: [\"info\"],\n  options: {\n    token: \"the token\",\n    region: \"us\"\n    // other options of insight\n  }\n});\n```\n\nAs with Winston, the options argument takes the normal constructor options.\n\nSee more details on [Ts.ED logger](https://logger.tsed.io/appenders/insight.html)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frapid7%2Fr7insight_node","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frapid7%2Fr7insight_node","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frapid7%2Fr7insight_node/lists"}