{"id":22800025,"url":"https://github.com/dadi/logger","last_synced_at":"2025-04-19T16:54:07.108Z","repository":{"id":10118378,"uuid":"56216414","full_name":"dadi/logger","owner":"dadi","description":null,"archived":false,"fork":false,"pushed_at":"2024-07-17T12:24:45.000Z","size":686,"stargazers_count":1,"open_issues_count":14,"forks_count":2,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-15T08:18:07.149Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dadi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-04-14T07:27:26.000Z","updated_at":"2024-01-22T09:38:17.000Z","dependencies_parsed_at":"2024-06-19T04:01:37.491Z","dependency_job_id":"fb457285-f6f9-48b0-823a-f1ed92af209b","html_url":"https://github.com/dadi/logger","commit_stats":{"total_commits":72,"total_committers":8,"mean_commits":9.0,"dds":0.4722222222222222,"last_synced_commit":"ed38b6e6563ddab6e3d965ff24c4738ee47ca171"},"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dadi%2Flogger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dadi%2Flogger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dadi%2Flogger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dadi%2Flogger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dadi","download_url":"https://codeload.github.com/dadi/logger/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249743112,"owners_count":21319053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-12T07:10:51.691Z","updated_at":"2025-04-19T16:54:07.092Z","avatar_url":"https://github.com/dadi.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DADI Logger\n\n[![npm (scoped)](https://img.shields.io/npm/v/@dadi/logger.svg?maxAge=10800\u0026style=flat-square)](https://www.npmjs.com/package/@dadi/logger)\n[![Coverage Status](https://coveralls.io/repos/github/dadi/logger/badge.svg?branch=master)](https://coveralls.io/github/dadi/logger?branch=master)\n[![Build Status](https://travis-ci.org/dadi/logger.svg?branch=master)](https://travis-ci.org/dadi/logger)\n[![JavaScript Style Guide](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square)](http://standardjs.com/)\n[![semantic-release](https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg?style=flat-square)](https://github.com/semantic-release/semantic-release)\n\n## Overview\n\n* [Install](#install)\n* [Configure](#configure)\n* [Using the logger](#using-the-logger)\n* [Namespacing](#namespacing-log-messages)\n* [HTTP request logging](#http-request-logging)\n* [Log rotation](#log-rotation)\n* [Viewing the logs](#viewing-the-logs)\n\n### Usage\n\n#### Install\n\n```\nnpm install @dadi/logger --save\n```\n\n#### Configure\n\nDADI Logger can be initialised with three parameters: log configuration,\nAWS configuration and run environment. Only the first parameter is required: environment\ndefaults to `development`.\n\nProperty|Description|Default|Example\n--------|-----------|-------|-------\nenabled|If true, logging is enabled using the following settings|false|true\nlevel|The threshold for writing to the log. Levels: `debug`, `info`, `warn`, `error`, `trace` |\"info\"|\"warn\"\nstream|The stream instance to write the log to||`process.stdout`\npath|The absolute or relative path to the directory for log files|\"./log\"|\"/var/log/\"\nfilename|The filename to use for logs, without extension| |\"web\"\nextension|The file extension to use for logs|\".log\"|\".txt\"\naccessLog| | | |\nenabled|If true, HTTP access logging is enabled. The log file name is similar to the setting used for normal logging, with the addition of 'access'. For example `web.access.log`|false|true\nkinesisStream|An AWS Kinesis stream to write to log records to| | |\n\n```js\nlet logConfig = {\n  enabled: true,\n  level: \"info\",\n  path: \"./log\",\n  filename: \"my_web_app\",\n  extension: \"log\",\n  accessLog: {\n    enabled: true,\n    kinesisStream: \"stream_name\"\n  }\n}\n\nlet awsConfig = {\n  accessKeyId: \"\",\n  secretAccessKey: \"\",\n  region: \"\"\n}\n\nconst logger = require('@dadi/logger')\nlogger.init(logConfig, awsConfig, 'production')\n```\n\n### Using the logger\n\nOnce the logger has been initialised it's as simple as requiring the module and calling a log method:\n\n```js\nconst logger = require('@dadi/logger')\n\nlogger.info('ImageHandler.get(): ' + req.url)\n```\n\n### Namespacing log messages\n\nLog messages can be \"namespaced\" based on any criteria you can think of, but we like to use a \"module\" namespace to make it easier to track down messages from different modules.\n\n```js\nconst logger = require('@dadi/logger')\n\nlogger.info({ module: 'ImageHandler' }, 'GET: ' + req.url)\n```\n\n### HTTP request logging\n\nYou can easily add an application-level middleware to log all HTTP requests. The `requestLogger` function will executed every time the app receives a request.\n\nDADI API, CDN and Web all use this HTTP request logger.\n\n#### Express\n\n```js\nconst express = require('express')\nconst app = express()\n\nconst logger = require('@dadi/logger')\nlogger.init(logConfig)\n\napp.use(logger.requestLogger)\n```\n\n#### Connect\n```js\nconst connect = require('connect')\nconst app = connect()\n\nconst logger = require('@dadi/logger')\nlogger.init(logConfig)\n\napp.use(logger.requestLogger)\n```\n\n#### Excluding sensitive parameters\n\nIt is possible to prevent sensitive querystring parameters from being written to log files by specifying an array of parameters in the configuration.\n\n```json\nlet logConfig = {\n  \"enabled\": true,\n  \"level\": \"info\",\n  \"path\": \"./log\",\n  \"filename\": \"my_web_app\",\n  \"filter\": [\"password\"],\n  \"extension\": \"log\",\n  \"accessLog\": {\n    \"enabled\": true\n  }\n}\n```\n\n```\n// Request URL:\n/profile?username=ed\u0026password=octopus\n\n// Written to log as:\n/profile?username=ed\u0026password=%5BFILTERED%5D\n```\n\n#### The HTTP log record\n\nThe request log contains a stream of JSON records. Each record contains a msg property containing details about the HTTP request, formatted using the nginx server log format.\n\n**Raw log record**\n```\n{\"name\":\"access\",\"hostname\":\"localhost\",\"pid\":3002,\"level\":30,\"msg\":\"127.0.0.1 - 2016-07-28T13:24:13+08:00 GET /news?page=3 HTTP/1.1 200 17529 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36\",\"time\":\"2016-07-28T05:24:13.460Z\",\"v\":0}\n```\n\n**Nginx log record, extracted from `msg`**\n```\n127.0.0.1 - 2016-07-28T13:24:13+08:00 GET /news?page=3 HTTP/1.1 200 17529 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36\n```\n\nThis record consists of the following fields:\n\n`remote address` - `time` `request` `status_code` `bytes_sent` `http_referer (optional)` `http_user_agent`\n\nFor example:\n\n* remote address: `127.0.0.1`\n* time: `2016-07-28T13:24:13+08:00`\n* request: `GET /news?page=3 HTTP/1.1`\n* status_code: `200`\n* bytes_sent: `17529`\n* http_referer (optional):\n* http_user_agent: `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36`\n\n### Log Rotation\n\n\u003e Automatic log rotation was removed from DADI Logger in v1.1.0 due to the number\nof applications relying on it that were being run on multiple CPU cores using Node's cluster module. **Log rotation should now be configured via your operating system.**\n\nThe below details are for a system running Ubuntu 14.04.\n\nYou can get detailed information about the `logrotate` tool from https://support.rackspace.com/how-to/understanding-logrotate-utility/\n\n**Example `logrotate` configuration file for a DADI Web log file**\n\nFile: `/etc/logrotate.d/web-production`\n\n```\n/data/apps/web/log/web.production.log {\n    su root root\n    daily\n    rotate 14\n    missingok\n    notifempty\n    copytruncate\n    compress\n}\n```\n\n### Viewing the logs\n\nDADI Logger uses [Bunyan](https://github.com/trentm/node-bunyan) to log errors and events. The Bunyan log output is a stream of JSON objects. A CLI tool is provided for pretty-printing Bunyan logs and for filtering.\n\n#### Install the CLI\n\nTo make reading the application logs easier install the Bunyan CLI tool globally:\n\n```\n$ npm install -g bunyan\n```\n\n#### Pipe the stdout log when running your app\n\n```\n$ npm start | bunyan\n```\n\n#### Formats\n\nA formatting option can be passed to the command to view shorter output. See the full range of options available [here](https://github.com/trentm/node-bunyan)\n\n```\n$ npm start | bunyan -o short\n```\n\n#### Pass the log contents to the CLI tool\n\n```\n$ tail log/web.log | bunyan\n```\n```\n[2015-10-27T09:14:01.856Z]  INFO: web/67025 on localhost: 5 rewrites/redirects loaded. (module=router)\n[2015-10-27T09:14:03.380Z]  INFO: web/67025 on localhost: Generating new access token for \"/home\" (module=auth)\n[2015-10-27T09:14:04.510Z]  INFO: web/67025 on localhost: Token received. (module=auth)\n[2015-10-27T09:14:04.517Z]  INFO: web/67025 on localhost: Generating new access token for datasource movies (module=auth/bearer)\n[2015-10-27T09:14:04.623Z]  INFO: web/67025 on localhost: https://127.0.0.1:3000/1.0/app/movies?count=3\u0026page=1\u0026filter={\"state\":\"published\"}\u0026fields={}\u0026sort={} (module=helper)\n[2015-10-27T09:14:04.643Z]  INFO: web/67025 on localhost: GET /home 200 65ms (module=router)\n[2015-10-27T09:16:46.331Z]  INFO: web/67025 on localhost: Server stopped, process exiting. (module=server)\n```\n\n#### Filtering logs\n\nLogs can be filtered by any valid Javascript condition. Here, `this` refers to the log record as JSON.\n\n**Example: filter logs by module**\n\n```\n$ tail -n30 log/web.log | bunyan -c 'this.module==\"router\"'\n```\n```\n[2015-10-27T09:14:01.856Z]  INFO: web/67025 on localhost: Rewrite module loaded. (module=router)\n[2015-10-27T09:14:01.856Z]  INFO: web/67025 on localhost: 5 rewrites/redirects loaded. (module=router)\n[2015-10-27T09:14:04.643Z]  INFO: web/67025 on localhost: GET /home 200 65ms (module=router)\n```\n\n**Example: filter logs by message contents**\n\n```\n$ tail log/web.log | bunyan -c 'this.msg.indexOf(\"GET\") \u003e -1' -o short\n```\n```\n09:11:57.618Z  INFO web: GET /home 200 460ms (module=router)\n09:13:18.325Z  INFO web: GET /home 200 2ms (module=router)\n```\n\n**Example: filter logs by level**\n\n```\n$ tail log/web.log | bunyan -l warn\n```\n```\n[2015-10-25T13:54:25.429Z]  WARN: web/58045 on localhost.local: log.stage() is deprecated and will be removed in a future release. Use log.debug(), log.info(), log.warn(), log.error(), log.trace() instead.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdadi%2Flogger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdadi%2Flogger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdadi%2Flogger/lists"}