{"id":19602687,"url":"https://github.com/qiwi/masker","last_synced_at":"2025-04-27T17:32:21.847Z","repository":{"id":41867850,"uuid":"266357461","full_name":"qiwi/masker","owner":"qiwi","description":"Composite data masking utility","archived":false,"fork":false,"pushed_at":"2025-04-14T11:01:56.000Z","size":5670,"stargazers_count":9,"open_issues_count":36,"forks_count":0,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-21T18:03:47.519Z","etag":null,"topics":["security"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/qiwi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-05-23T14:51:02.000Z","updated_at":"2025-01-16T09:44:41.000Z","dependencies_parsed_at":"2024-04-17T14:45:43.045Z","dependency_job_id":"6a30478a-1359-4d1c-b3e7-5bc0d37021df","html_url":"https://github.com/qiwi/masker","commit_stats":{"total_commits":509,"total_committers":8,"mean_commits":63.625,"dds":0.5245579567779961,"last_synced_commit":"1b3cf47948381e82cb775eb450cf7064b82c5d3d"},"previous_names":[],"tags_count":271,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qiwi%2Fmasker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qiwi%2Fmasker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qiwi%2Fmasker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qiwi%2Fmasker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/qiwi","download_url":"https://codeload.github.com/qiwi/masker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251178037,"owners_count":21548152,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["security"],"created_at":"2024-11-11T09:25:48.973Z","updated_at":"2025-04-27T17:32:20.068Z","avatar_url":"https://github.com/qiwi.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# @qiwi/masker\n\u003e Composite data masking utility\n\n[![CI](https://github.com/qiwi/masker/actions/workflows/ci.yaml/badge.svg?branch=master)](https://github.com/qiwi/masker/actions/workflows/ci.yaml)\n[![Maintainability](https://api.codeclimate.com/v1/badges/6205424ac673cb3f2bb8/maintainability)](https://codeclimate.com/github/qiwi/masker/maintainability)\n[![Test Coverage](https://api.codeclimate.com/v1/badges/6205424ac673cb3f2bb8/test_coverage)](https://codeclimate.com/github/qiwi/masker/test_coverage)  \n\n- [Digest](#digest)\n    - [Purpose](#purpose)\n    - [Status](#status)\n    - [Roadmap](#roadmap)\n    - [Key features](#key-features)\n- [Getting started](#getting-started)\n    - [Install](#install)\n    - [Default preset](#default-preset)\n    - [Custom pipeline](#custom-pipeline)\n    - [Masking schema](#masking-schema)\n    - [CLI](#cli)\n    - [Playground](#playground)\n- [Integration](#integration)\n    - [Console](#console)\n    - [Winston](#winston)\n- [Design](#design)\n- [Documentation](#documentation)\n- [Contributing](#contributing)\n- [Packages](#packages)\n- [License](#license)\n\n## Digest\n### Purpose\nImplement instruments, describe practices, contracts to solve sensitive data masking problem in JS/TS.\nFor secure logging, for public data output, for internal mimt-proxies (kuber sensitive-data-policy) and so on.\n\n### Status\n🚧 Work in progress / MVP#0 is available for testing  \n⚠️ **Not ready for production yet**\n\n### Roadmap\n- [x] Implement masking composer/processor\n- [x] Introduce (declarative?) masking directives: [schema](https://github.com/qiwi/masker/tree/master/packages/schema)\n- [x] Describe masking strategies and add masking utils\n- [x] Support logging tools integration\n\n### Key features\n* Both sync and async API\n* Declarative configuration\n* Deep customization\n* TS and Flow typings\n\n## Getting started\n### Install\nWith npm:\n```shell\nnpm install --save @qiwi/masker\n```\nor yarn:\n```shell\nyarn add @qiwi/masker\n```\n\n### Default preset\n```ts\nimport {masker} from '@qiwi/masker'\n\n// Suitable for most std cases: strings, objects, json strings, which may contain any standard secret keys/values or card PANs.\nmasker('411111111111111')       // Promise\u003c4111 **** **** 1111\u003e\nmasker.sync('4111111111111111') // 4111 **** **** 1111\n```\n\n### Custom pipeline\n```ts\nimport {masker, registry} from '@qiwi/masker'\n\nmasker.sync({\n  secret: 'foo',\n  nested: {\n    pans: [4111111111111111]\n  },\n  foo: 'str with printed password=foo and smth else',\n  json: 'str with json inside {\"secret\":\"bar\"} {\"4111111111111111\":\"bar\"}',\n}, {\n  registry,           // plugin storage\n  pipeline: [\n    'split',          // to recursively process object's children. The origin `pipeline` will be applied to internal keys and values\n    'pan',            // to mask card PANs\n    'secret-key',     // to conceal sensitive fields like `secret` or `token` (pattern is configurable)\n    'secret-value',   // to replace sensitive parts of strings like `token=foobar` (pattern is configurable)\n    'json',           // to find jsons in strings\n  ]\n})\n\n// result:\n{\n  secret: '***',      // secret-key\n  nested: { // split\n    pans: [ // split\n      '4111 **** **** 1111' // pan\n    ],  \n  },\n  foo: 'str with printed *** and smth else',  // secret-value\n  // json\n  // chunk#1: split, secret-key\n  // chunk#2: split, pan (applied to key!)\n  json: 'str with json inside {\"secret\":\"***\"} {\"4111 **** **** 1111\":\"bar\"}'\n}\n```\n\n### Masking schema\nDeclare masker directives over [json-schema](https://json-schema.org/). See [@qiwi/masker-schema](https://github.com/qiwi/masker/tree/master/packages/schema) for details.\n```ts\nimport {masker} from '@qiwi/masker';\n\nmasker.sync({\n  fo: 'fo',\n  foo: 'bar',\n  foofoo: 'barbar',\n  baz: 'qux',\n  arr:  [4111111111111111, 1234123412341234]\n}, {\n  pipeline: ['schema'],\n  schema: {\n    type: 'object',\n    properties: {\n      fo: {\n        type: 'string',\n        maskKey: ['plain']\n      },\n      foo: {\n        type: 'string',\n        maskKey: ['plain']\n      },\n      foofoo: {\n        type: 'string',\n        maskKey: ['strike'],\n        maskValue: ['plain']\n      },\n      arr: {\n        type: 'array',\n        items: {\n          type: 'number',\n          maskValue: ['pan']\n        }\n      }\n    }\n  }\n})\n\n// result:\n{\n  baz: 'qux',\n  arr: [ '4111 **** **** 1111', '1234123412341234' ],\n  '***': 'fo',\n  '***(2)': 'bar',\n  '******': '***',\n}\n```\n\n### CLI\n```shell script\nnpx masquer \"4111 1111 1111 1111\"\n# returns 4111 **** **** 1111\n```\n\n### Playground\n[codesandbox.io/s/qiwi-masker-sandbox-ngrnu](https://codesandbox.io/s/qiwi-masker-sandbox-ngrnu?file=/src/index.js)\n\n## Integration\n### Console\nOverride global console methods to print sensitive data free output to `stderr/stdout`:\n```ts\nimport {masker} from '@qiwi/masker'\n\n['log', 'info', 'error'].forEach(method =\u003e {\n  const _method = console[method]\n  console[method] = (...args: any[]) =\u003e _method(...args.map(masker))\n})\n```\n\n### Winston\nCreate a [custom masker formatter](https://github.com/winstonjs/winston#formats), then attach it to your reporter / transport:\n```ts\nconst winston = require('winston')\nconst {masker} = require('@qiwi/masker')\n\nconst logger = winston.createLogger({\n  levels: winston.config.syslog.levels,\n  transports: [\n    new winston.transports.Console({\n      format: winston.format.combine(\n        winston.format((info) =\u003e Object.assign(info, masker.sync(info)))(),\n        winston.format.json(),\n      ),\n    })\n  ]\n})\n\nlogger.log({\n  level: 'info',\n  message: {foo: 'bar', secret: 'foobar', pan: [4111111111111111, 1234123412341234]},\n})\n\n// stdout\n{\"level\":\"info\",\"message\":{\"foo\":\"bar\",\"secret\":\"***\",\"pan\":[\"4111 **** **** 1111\",\"1234123412341234\"]}}\n```\n[stackoverflow.com/how-to-make-a-custom-json-formatter-for-winston3-logger](https://stackoverflow.com/questions/51454523/how-to-make-a-custom-json-formatter-for-winston3-logger)\n\n## Design\n### Middleware\nThe masker bases on [the middleware pattern](https://www.google.com/search?q=middleware+pattern+javascript): it takes a piece of data and pushes it forward the `pipeline`. \nThe output of each `pipe` is the input for the next one. Each pipe is a dual interface data processor:\n```ts\nexport interface IMaskerPipe {\n  name: IMaskerPipeName\n  exec: IMaskerPipeAsync | IMaskerPipeDual\n  execSync: IMaskerPipeSync | IMaskerPipeDual,\n  opts?: IMaskerPipeOpts\n}\n```\n\n### True dynamic\nDuring the execution, every pipe handler takes full control of the `context`. It can override next steps, change the `executor` impl (replace, append hook, etc), \ncreate internal masker threads, parallelize invocation queues and sync them back together, and so on.\n\n### Context\nEach pipe is fed with a normalized context which consists of:\n```ts\nexport interface IMaskerPipeInput {\n  value: any                // value to process\n  _value?: any              // pipe result\n  id: IContextId            // ctx unique key\n  context: IMaskerPipeInput // ctx self ref\n  parentId?: IContextId     // parent ctx id\n  registry: IMaskerRegistry       // pipe registry attached to ctx\n  execute: IExecutor              // executor \n  sync: boolean                   // sync / async switch\n  mode: IExecutionMode            // lagacy sync switch\n  opts: IMaskerOpts               // current pipe options\n  pipe?: IMaskerPipeNormalized              // current pipe ref\n  pipeline: IMaskerPipelineNormalized       // actual pipeline\n  originPipeline: IMaskerPipelineNormalized // origin pipeline\n  [key: string]: any\n}\n```\n\n### Sync / async\nBoth. In different situations, each approach has pros and cons.\nFor this reason, the masker provides a choice:\n```ts\nmasker(data)                // async\nmasker.sync(data)           // sync\nmasker(data, {sync: true})  // sync\n```\n\n### Documentation\n* [JS/TS API](https://github.com/qiwi/masker/blob/master/API.md)\n* [CLI](https://github.com/qiwi/masker/blob/master/CLI.md)\n\n## Packages\nThere is also a bunch of plugins, that extend the available masking scenarios. Please follow their internal docs.\n\n| Package | Description | Version\n|---|---|---\n|[@qiwi/masker](https://github.com/qiwi/masker/tree/master/packages/facade)| Composite data masking utility with common pipeline preset | [![npm](https://img.shields.io/npm/v/@qiwi/masker/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker)\n|[masquer](https://github.com/qiwi/masker/tree/master/packages/cli)| CLI for [@qiwi/masker](https://github.com/qiwi/masker/tree/master/packages/facade) | [![npm](https://img.shields.io/npm/v/masquer/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/masquer)\n|[@qiwi/masker-common](https://github.com/qiwi/masker/tree/master/packages/common)| Masker common components: interfaces, executor, utils | [![npm](https://img.shields.io/npm/v/@qiwi/masker-common/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-common)\n|[@qiwi/masker-debug](https://github.com/qiwi/masker/tree/master/packages/debug)| Debug plugin to observe pipe effects | [![npm](https://img.shields.io/npm/v/@qiwi/masker-debug/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-debug)\n|[@qiwi/masker-infra](https://github.com/qiwi/masker/tree/master/packages/infra)| Infra package: build configs, tools, etc | [![npm](https://img.shields.io/npm/v/@qiwi/masker-infra/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-infra)\n|[@qiwi/masker-json](https://github.com/qiwi/masker/tree/master/packages/json)| Plugin to search and parse JSONs chunks in strings | [![npm](https://img.shields.io/npm/v/@qiwi/masker-json/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-json)\n|[@qiwi/masker-limiter](https://github.com/qiwi/masker/tree/master/packages/limiter)| Plugin to limit masking steps count and duration | [![npm](https://img.shields.io/npm/v/@qiwi/masker-limiter/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-limiter)\n|[@qiwi/masker-pan](https://github.com/qiwi/masker/tree/master/packages/pan)| Plugin to search and conceal [PANs](https://en.wikipedia.org/wiki/Payment_card_number) | [![npm](https://img.shields.io/npm/v/@qiwi/masker-pan/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-pan)\n|[@qiwi/masker-plain](https://github.com/qiwi/masker/tree/master/packages/plain)| Plugin to substitute any kind of data with `***` | [![npm](https://img.shields.io/npm/v/@qiwi/masker-plain/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-plain)\n|[@qiwi/masker-schema](https://github.com/qiwi/masker/tree/master/packages/schema)| Masker schema builder and executor | [![npm](https://img.shields.io/npm/v/@qiwi/masker-schema/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-schema)\n|[@qiwi/masker-secret-key](https://github.com/qiwi/masker/tree/master/packages/secret-key)| Plugin to hide sensitive data by key/path pattern match | [![npm](https://img.shields.io/npm/v/@qiwi/masker-secret-key/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-secret-key)\n|[@qiwi/masker-secret-value](https://github.com/qiwi/masker/tree/master/packages/secret-value)| Plugin to conceal substrings by pattern match | [![npm](https://img.shields.io/npm/v/@qiwi/masker-secret-value/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-secret-value)\n|[@qiwi/masker-split](https://github.com/qiwi/masker/tree/master/packages/split)| Executor hook to recursively process any object inners | [![npm](https://img.shields.io/npm/v/@qiwi/masker-split/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-split)\n|[@qiwi/masker-strike](https://github.com/qiwi/masker/tree/master/packages/strike)| Plugin to ~~strikethough~~ any non-space string chars | [![npm](https://img.shields.io/npm/v/@qiwi/masker-strike/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-strike)\n|[@qiwi/masker-trycatch](https://github.com/qiwi/masker/tree/master/packages/trycatch)| Executor hook to capture and handle exceptions | [![npm](https://img.shields.io/npm/v/@qiwi/masker-trycatch/latest.svg?label=\u0026color=09e)](https://www.npmjs.com/package/@qiwi/masker-trycatch)\n\n## Contributing\nFeel free to open any issues: for bugs, feature requests or questions. \nYou're always welcome to suggest a PR. Just fork this repo, write some code, add some tests and push your changes. \nAny feedback is appreciated.\n\n## License\n[MIT](https://github.com/qiwi/masker/blob/master/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqiwi%2Fmasker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqiwi%2Fmasker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqiwi%2Fmasker/lists"}