{"id":18317895,"url":"https://github.com/idiocc/nicer","last_synced_at":"2025-04-09T13:51:19.810Z","repository":{"id":65459166,"uuid":"195534345","full_name":"idiocc/nicer","owner":"idiocc","description":"An Http Multipart/Form-Data Request Body Parser.","archived":false,"fork":false,"pushed_at":"2019-07-29T17:57:37.000Z","size":3532,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-22T21:00:06.510Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.idio.cc","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/idiocc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-06T11:57:11.000Z","updated_at":"2019-12-29T10:54:34.000Z","dependencies_parsed_at":"2023-01-24T14:45:54.157Z","dependency_job_id":null,"html_url":"https://github.com/idiocc/nicer","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idiocc%2Fnicer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idiocc%2Fnicer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idiocc%2Fnicer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idiocc%2Fnicer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/idiocc","download_url":"https://codeload.github.com/idiocc/nicer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248054218,"owners_count":21039951,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T18:07:48.655Z","updated_at":"2025-04-09T13:51:19.787Z","avatar_url":"https://github.com/idiocc.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nicer\n\n[![npm version](https://badge.fury.io/js/nicer.svg)](https://npmjs.org/package/nicer)\n\n`nicer` is An Http _Multipart/Form-Data_ Request Body Parser. It can receive form data fields and files headers and stream their data.\n\n```sh\nyarn add nicer\n```\n\n\u003ctable\u003e\n\u003ctr\u003e\u003cth colspan=\"2\"\u003e\u003ca href=\"benchmark/default\"\u003eBenchmark\u003c/a\u003e\u003c/th\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e\n\n|  Library   |     1      |     2      |     3      |\n| ---------- | ---------- | ---------- | ---------- |\n| dicer      | 137.21mb/s | 100.65mb/s | 165.95mb/s |\n| multiparty | 25.78mb/s  | 29.48mb/s  | 29.60mb/s  |\n| nicer      | 106.55mb/s | 112.46mb/s | 134.24mb/s |\n| nicerc     | 114.42mb/s | 115.04mb/s | 116.57mb/s |\n\u003c/td\u003e\u003ctd\u003e\n\u003cdetails\u003e\n\u003csummary\u003e\u003ca name=\"stable-benchmark\"\u003eStable Benchmark\u003c/a\u003e (18 Jul)\u003c/summary\u003e\n\n|  Library   |     Max Speed     |\n| ---------- | --------- |\n| (1) dicer      | 92.93mb/s |\n| (2) nicerc     | 79.95mb/s |\n|     nicer      | 72.23mb/s |\n| (3) multiparty | 28.79mb/s |\n\u003c/details\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\u003ctd colspan=\"2\"\u003e\u003ca href=\"https://github.com/idiocc/nicer\"\u003e\u003cem\u003eNicer\u003c/em\u003e\u003c/a\u003e is comparable to the faster streaming parser, \u003ca href=\"https://github.com/idiocc/dicer\"\u003e\u003cem\u003eDicer\u003c/em\u003e\u003c/a\u003e since the real-world data (uploading 2 fields, 2 text files and 50 photos) is processed at speeds that are close to max 90mb/s. In the benchmark, \u003ccode\u003enicer\u003c/code\u003e is the source code of this package, whereas \u003ccode\u003enicerc\u003c/code\u003e is the \u003ca href=\"https://compiler.page\"\u003ecompiled JavaScript\u003c/a\u003e optimised Closure Compiler, which probably increases the speed by 5-10%.\u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\n\u003cp align=\"center\"\u003e\u003ca href=\"#table-of-contents\"\u003e\u003cimg src=\"/.documentary/section-breaks/0.svg?sanitize=true\"\u003e\u003c/a\u003e\u003c/p\u003e\n\n## Table Of Contents\n\n* [Stable Benchmark](#stable-benchmark)\n- [Table Of Contents](#table-of-contents)\n- [API](#api)\n- [`constructor(boundary: string): Nicer`](#constructorboundary-string-nicer)\n  * [`Nicer`](#type-nicer)\n  * [`Part`](#type-part)\n- [Errors](#errors)\n  * [Extra Buffer](#extra-buffer)\n- [Debug](#debug)\n- [Copyright](#copyright)\n\n\u003cp align=\"center\"\u003e\u003ca href=\"#table-of-contents\"\u003e\u003cimg src=\"/.documentary/section-breaks/1.svg?sanitize=true\"\u003e\u003c/a\u003e\u003c/p\u003e\n\n## API\n\nThe package is available by importing its default constructor function:\n\n```js\nimport Nicer from 'nicer'\n```\n\n\u003cp align=\"center\"\u003e\u003ca href=\"#table-of-contents\"\u003e\u003cimg src=\"/.documentary/section-breaks/2.svg?sanitize=true\"\u003e\u003c/a\u003e\u003c/p\u003e\n\n## `constructor(`\u003cbr/\u003e\u0026nbsp;\u0026nbsp;`boundary: string,`\u003cbr/\u003e`): Nicer`\n\nCreates a transform that emits objects with a header buffer and the body stream. The body stream is a pass-through so all data must be written as it comes, the request doesn't pause for data to be consumed. The header is a buffer which can be parsed more and/or decrypted, but it does not stream. The assumption is the headers are short therefore a header buffer is accumulated until `\\r\\n` is found. Just make sure to run behind _NginX_ then it should be alright.\n\n\u003ctable\u003e\n\u003ctr\u003e\u003cth\u003e\n\n__\u003ca name=\"type-nicer\"\u003e`Nicer`\u003c/a\u003e__: A stream that emits objects with a header buffer and the body PassThrough stream.\n\n|     Name      |      Type       |          Description           |\n| ------------- | --------------- | ------------------------------ |\n| __boundary*__ | \u003cem\u003estring\u003c/em\u003e | The mandatory field separator. |\n\u003c/th\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e\n\n```js\nimport { Transform } from 'stream'\nimport Nicer from 'nicer'\n\nconst detected = []\n\nawait http.startPlain((req, res) =\u003e {\n  const boundary = getBoundary(req, res)\n  console.log('Boundary detected: %s', boundary)\n  req.pipe(new Nicer({ boundary })).pipe(new Transform({\n    objectMode: true,\n    transform(obj, enc, next) {\n      const { header: HEADER, stream: STREAM } = obj\n\n      // to print in sync have to wait for all data\n      // since STREAM is a pass-through\n      let d = []\n      detected.push(['%s\\n====\\n', HEADER, d])\n\n      STREAM.on('data', (data) =\u003e {\n        d.push(data)\n      })\n      next()\n    },\n    final() {\n      res.statusCode = 200\n      res.end(JSON.stringify(detected))\n    },\n  }))\n})\n```\n\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003eA new instance of \u003cem\u003eNicer\u003c/em\u003e can be piped into by an \u003cem\u003ehttp.IncomingMessage\u003c/em\u003e stream in the Node.JS server. Then a transform stream must be created to listen for the data emitted by \u003cem\u003eNicer\u003c/em\u003e in object mode.\n\n__\u003ca name=\"type-part\"\u003e`Part`\u003c/a\u003e__: A part that gets emitted by _Nicer_.\n\n|    Name     |             Type             |          Description           |\n| ----------- | ---------------------------- | ------------------------------ |\n| __stream*__ | \u003cem\u003e!stream.PassThrough\u003c/em\u003e | The mandatory field separator. |\n| __header*__ | \u003cem\u003e!Buffer\u003c/em\u003e             | The header found before data.  |\n\n\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e\n\n```\nBoundary detected: u2KxIV5yF1y+xUspOQCCZopaVgeV6Jxihv35XQJmuTx8X3sh\n\nContent-Disposition: form-data; name=\"key\"\n====\n [ 'value' ] \n\n\nContent-Disposition: form-data; name=\"alan\"\n====\n [ 'watts' ] \n\n\nContent-Disposition: form-data; name=\"file\"; filename=\"test/fixture/test.txt\"\nContent-Type: application/octet-stream\n====\n [ 'a test file\\n' ]\n```\n\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003eThe data received by the 'transform' method, contains the { header, stream } properties. The data from the stream must be accumulated.\n\u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\n\u003cp align=\"center\"\u003e\u003ca href=\"#table-of-contents\"\u003e\u003cimg src=\"/.documentary/section-breaks/3.svg?sanitize=true\"\u003e\u003c/a\u003e\u003c/p\u003e\n\n## Errors\n\nThe errors are spawned when the buffer remaining the stream after the `final` event, and processed to extract the rest of the fields, still contains symbols different from \u003ckbd\u003e-\u003c/kbd\u003e\u003ckbd\u003e-\u003c/kbd\u003e (`[45,45]`).\n\n\u003ctable\u003e\n\u003ctr\u003e\u003cth\u003e\u003ca name=\"extra-buffer\"\u003eExtra Buffer\u003c/a\u003e \u003ca href=\"example/extra-buffer.js\"\u003e(\u003cem\u003eSource\u003c/em\u003e)\u003c/a\u003e\u003c/th\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e\n\n```http\n-------example\nContent-Disposition: form-data; name=\"key\"\n\ndata\n-------exampleWAT\n```\n\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003eThe data remaining after the last boundary detected after the \u003cem\u003efinal\u003c/em\u003e method is called does not have any meaning and is discarded. This is not the case with parts that arrived before the stream was closed, i.e., the file limit is not implemented.\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e\n\n```fs\nBoundary detected: -----example\n[!] Error Unexpected end of request body, wanted to see \"--\" but saw WA.\n    Detected Data:\n\nContent-Disposition: form-data; name=\"key\"\n====\n [ 'data' ]\n```\n\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003eThe parser will always check for the closing \u003ccode\u003e--\u003c/code\u003e and emit an error in the end, however the headers and data streams emitted by it, would have been all closed, i.e., the data can still be used.\u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\n\u003cp align=\"center\"\u003e\u003ca href=\"#table-of-contents\"\u003e\u003cimg src=\"/.documentary/section-breaks/4.svg?sanitize=true\"\u003e\u003c/a\u003e\u003c/p\u003e\n\n## Debug\n\nThe software can write debug information, when the `DEBUG=nicer` env variable is set.\n\n\u003ctable\u003e\n\u003ctr\u003e\u003cth\u003eDebug \u003ca href=\"example/debug.js\"\u003e(\u003cem\u003eSource\u003c/em\u003e)\u003c/a\u003e\u003c/th\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e\n\n```js\nimport { Writable } from 'stream'\nimport Nicer from '../src'\n\nconst detected = []\n\nawait http.startPlain((req, res) =\u003e {\n  const boundary = getBoundary(req, res)\n  console.log('Boundary detected: %s', boundary)\n  const nicer = new Nicer({ boundary })\n  const bt = new BufferTransform(50)\n\n  req.pipe(bt).pipe(nicer).pipe(new Writable({\n    objectMode: true,\n    write(obj, enc, next) {\n      const { header: HEADER, stream: STREAM } = obj\n      next()\n    },\n    final() {\n      res.statusCode = 200\n      res.end(JSON.stringify(detected))\n    },\n  }))\n})\n```\n\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003eThe transform method appends data to the left-over buffer (which can usually be small enough to accommodate [--boundary.length-1] symbols) and consumes data. The data is consumed by first trying to find the boundary in the new buffer. If this is possible, then depending on the state of the parser, the data found before the separator is either flushed in an existing data stream, or appended to the existing header, which can then lead to body-flushing.\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e\n\n```sh\nnicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +0ms\n  nicer 🔎  Finished boundary scan, buffer of length 50B left, separators found: 0 +2ms\n  nicer one consume safe consumed 0B and left 50B +1ms\n  nicer \u003cconcat-transform\u003e +1ms\n  nicer \u003cconcat-transform\u003e 100B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +0ms\n  nicer   ⭐  Found starting boundary at index 2 +0ms\n  nicer 🔎  Finished boundary scan, buffer of length 48B left, separators found: 1 +0ms\n  nicer one consume safe consumed 52B and left 48B +1ms\n  nicer \u003cconcat-transform\u003e +0ms\n  nicer \u003cconcat-transform\u003e 98B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +0ms\n  nicer 🔎  Finished boundary scan, buffer of length 98B left, separators found: 0 +0ms\n  nicer one consume safe consumed 46B and left 52B +0ms\n  nicer \u003cconcat-transform\u003e +0ms\n  nicer \u003cconcat-transform\u003e 102B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +0ms\n  nicer   🔛  Found boundary, data size 7B +0ms\n  nicer   🗒  Found header and data of size \u003c53B\u003e +1ms\n  nicer      Content-Disposition: form-data; name=\"key\" +0ms\n  nicer      value +0ms\n  nicer 🔎  Finished boundary scan, buffer of length 43B left, separators found: 1 +2ms\n  nicer one consume safe consumed 59B and left 43B +0ms\n  nicer \u003cconcat-transform\u003e +0ms\n  nicer \u003cconcat-transform\u003e 93B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +0ms\n  nicer 🔎  Finished boundary scan, buffer of length 93B left, separators found: 0 +0ms\n  nicer one consume safe consumed 41B and left 52B +0ms\n  nicer \u003cconcat-transform\u003e +1ms\n  nicer \u003cconcat-transform\u003e 102B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +0ms\n  nicer   🔛  Found boundary, data size 13B +0ms\n  nicer   🗒  Found header and data of size \u003c54B\u003e +0ms\n  nicer      Content-Disposition: form-data; name=\"alan\" +0ms\n  nicer      watts +0ms\n  nicer 🔎  Finished boundary scan, buffer of length 37B left, separators found: 1 +0ms\n  nicer one consume safe consumed 65B and left 37B +1ms\n  nicer \u003cconcat-transform\u003e +0ms\n  nicer \u003cconcat-transform\u003e 87B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +0ms\n  nicer 🔎  Finished boundary scan, buffer of length 87B left, separators found: 0 +0ms\n  nicer one consume safe consumed 35B and left 52B +0ms\n  nicer \u003cconcat-transform\u003e +0ms\n  nicer \u003cconcat-transform\u003e 102B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +11ms\n  nicer 🔎  Finished boundary scan, buffer of length 102B left, separators found: 0 +0ms\n  nicer       \u003cconcat-header\u003e +0ms\n  nicer       \u003cconcat-header\u003e 85B +1ms\n  nicer one consume safe consumed 50B and left 52B +0ms\n  nicer \u003cconcat-transform\u003e +0ms\n  nicer \u003cconcat-transform\u003e 102B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +1ms\n  nicer   🔛  Found boundary, data size 50B +0ms\n  nicer   🗒  Found header and data of size \u003c135B\u003e +0ms\n  nicer      Content-Disposition: form-data; name=\"file\"; filename=\"test/fixture/test.txt\" +0ms\n  nicer      Content-Type: ap... +0ms\n  nicer 🔎  Finished boundary scan, buffer of length 0B left, separators found: 1 +1ms\n  nicer one consume safe consumed 102B and left 0B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +0ms\n  nicer 🔎  Finished boundary scan, buffer of length 4B left, separators found: 0 +0ms\n  nicer one consume safe consumed 0B and left 4B +0ms\n  nicer 🔍  Staring boundary --u2KxIV5yF1y+x... scan +2ms\n  nicer 🔎  Finished boundary scan, buffer of length 4B left, separators found: 0 +1ms\n  nicer one consume safe consumed 0B and left 4B +0ms\n```\n\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003eAfter knowing what's left after the last found boundary, the \u003cem\u003eNicer\u003c/em\u003e parser takes only the safe amount of data to consume more which equals to the length of the boundary (including prior --), otherwise there might be a partial boundary leaking into the data stream. The remainder is saved as the new buffer, to which the following chunk in the transform method will be appended, and so on.\u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\n\u003cp align=\"center\"\u003e\u003ca href=\"#table-of-contents\"\u003e\u003cimg src=\"/.documentary/section-breaks/5.svg?sanitize=true\"\u003e\u003c/a\u003e\u003c/p\u003e\n\n## Copyright\n\n(c) [Idio][1] 2019\n\n[1]: https://idio.cc\n\n\u003cp align=\"center\"\u003e\u003ca href=\"#table-of-contents\"\u003e\u003cimg src=\"/.documentary/section-breaks/-1.svg?sanitize=true\"\u003e\u003c/a\u003e\u003c/p\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidiocc%2Fnicer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fidiocc%2Fnicer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidiocc%2Fnicer/lists"}