{"id":13410861,"url":"https://github.com/mafintosh/csv-parser","last_synced_at":"2025-05-12T15:23:00.043Z","repository":{"id":17086977,"uuid":"19852124","full_name":"mafintosh/csv-parser","owner":"mafintosh","description":"Streaming csv parser inspired by binary-csv that aims to be faster than everyone else","archived":false,"fork":false,"pushed_at":"2025-01-29T21:31:43.000Z","size":918,"stargazers_count":1455,"open_issues_count":57,"forks_count":139,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-04-23T17:13:19.412Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"moodle/moodle","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mafintosh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-05-16T09:58:45.000Z","updated_at":"2025-04-11T14:01:45.000Z","dependencies_parsed_at":"2024-06-18T11:10:47.917Z","dependency_job_id":"b31ab92d-8b1c-4b69-8b32-b5eea0a78f25","html_url":"https://github.com/mafintosh/csv-parser","commit_stats":{"total_commits":160,"total_committers":33,"mean_commits":4.848484848484849,"dds":0.59375,"last_synced_commit":"e712e324d49e7dee00dc66d3f5e65464ceed66e5"},"previous_names":[],"tags_count":43,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafintosh%2Fcsv-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafintosh%2Fcsv-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafintosh%2Fcsv-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafintosh%2Fcsv-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mafintosh","download_url":"https://codeload.github.com/mafintosh/csv-parser/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250477810,"owners_count":21437049,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T20:01:09.837Z","updated_at":"2025-04-23T17:13:26.486Z","avatar_url":"https://github.com/mafintosh.png","language":"JavaScript","readme":"[tests]: \thttp://img.shields.io/travis/mafintosh/csv-parser.svg\n[tests-url]: http://travis-ci.org/mafintosh/csv-parser\n\n[cover]: https://codecov.io/gh/mafintosh/csv-parser/branch/master/graph/badge.svg\n[cover-url]: https://codecov.io/gh/mafintosh/csv-parser\n\n[size]: https://packagephobia.now.sh/badge?p=csv-parser\n[size-url]: https://packagephobia.now.sh/result?p=csv-parser\n\n# csv-parser\n\n[![tests][tests]][tests-url]\n[![cover][cover]][cover-url]\n[![size][size]][size-url]\n\nStreaming CSV parser that aims for maximum speed as well as compatibility with\nthe [csv-spectrum](https://npmjs.org/csv-spectrum) CSV acid test suite.\n\n`csv-parser` can convert CSV into JSON at at rate of around 90,000 rows per\nsecond. Performance varies with the data used; try `bin/bench.js \u003cyour file\u003e`\nto benchmark your data.\n\n`csv-parser` can be used in the browser with [browserify](http://browserify.org/).\n\n[neat-csv](https://github.com/sindresorhus/neat-csv) can be used if a `Promise`\nbased interface to `csv-parser` is needed.\n\n_Note: This module requires Node v8.16.0 or higher._\n\n## Benchmarks\n\n⚡️ `csv-parser` is greased-lightning fast\n\n```console\n→ npm run bench\n\n  Filename                 Rows Parsed  Duration\n  backtick.csv                       2     3.5ms\n  bad-data.csv                       3    0.55ms\n  basic.csv                          1    0.26ms\n  comma-in-quote.csv                 1    0.29ms\n  comment.csv                        2    0.40ms\n  empty-columns.csv                  1    0.40ms\n  escape-quotes.csv                  3    0.38ms\n  geojson.csv                        3    0.46ms\n  large-dataset.csv               7268      73ms\n  newlines.csv                       3    0.35ms\n  no-headers.csv                     3    0.26ms\n  option-comment.csv                 2    0.24ms\n  option-escape.csv                  3    0.25ms\n  option-maxRowBytes.csv          4577      39ms\n  option-newline.csv                 0    0.47ms\n  option-quote-escape.csv            3    0.33ms\n  option-quote-many.csv              3    0.38ms\n  option-quote.csv                   2    0.22ms\n  quotes+newlines.csv                3    0.20ms\n  strict.csv                         3    0.22ms\n  latin.csv                          2    0.38ms\n  mac-newlines.csv                   2    0.28ms\n  utf16-big.csv                      2    0.33ms\n  utf16.csv                          2    0.26ms\n  utf8.csv                           2    0.24ms\n```\n\n## Install\n\nUsing npm:\n\n```console\n$ npm install csv-parser\n```\n\nUsing yarn:\n\n```console\n$ yarn add csv-parser\n```\n\n## Usage\n\nTo use the module, create a readable stream to a desired CSV file, instantiate\n`csv`, and pipe the stream to `csv`.\n\nSuppose you have a CSV file `data.csv` which contains the data:\n\n```\nNAME,AGE\nDaffy Duck,24\nBugs Bunny,22\n```\n\nIt could then be parsed, and results shown like so:\n\n``` js\nconst csv = require('csv-parser')\nconst fs = require('fs')\nconst results = [];\n\nfs.createReadStream('data.csv')\n  .pipe(csv())\n  .on('data', (data) =\u003e results.push(data))\n  .on('end', () =\u003e {\n    console.log(results);\n    // [\n    //   { NAME: 'Daffy Duck', AGE: '24' },\n    //   { NAME: 'Bugs Bunny', AGE: '22' }\n    // ]\n  });\n```\n\nTo specify options for `csv`, pass an object argument to the function. For\nexample:\n\n```js\ncsv({ separator: '\\t' });\n```\n\n## API\n\n### csv([options | headers])\n\nReturns: `Array[Object]`\n\n#### options\n\nType: `Object`\n\nAs an alternative to passing an `options` object, you may pass an `Array[String]`\nwhich specifies the headers to use. For example:\n\n```js\ncsv(['Name', 'Age']);\n```\n\nIf you need to specify options _and_ headers, please use the the object notation\nwith the `headers` property as shown below.\n\n#### escape\n\nType: `String`\u003cbr\u003e\nDefault: `\"`\n\nA single-character string used to specify the character used to escape strings\nin a CSV row.\n\n#### headers\n\nType: `Array[String] | Boolean`\n\nSpecifies the headers to use. Headers define the property key for each value in\na CSV row. If no `headers` option is provided, `csv-parser` will use the first\nline in a CSV file as the header specification.\n\nIf `false`, specifies that the first row in a data file does _not_ contain\nheaders, and instructs the parser to use the column index as the key for each column.\nUsing `headers: false` with the same `data.csv` example from above would yield:\n\n``` js\n[\n  { '0': 'Daffy Duck', '1': 24 },\n  { '0': 'Bugs Bunny', '1': 22 }\n]\n```\n\n_Note: If using the `headers` for an operation on a file which contains headers on the first line, specify `skipLines: 1` to skip over the row, or the headers row will appear as normal row data. Alternatively, use the `mapHeaders` option to manipulate existing headers in that scenario._\n\n#### mapHeaders\n\nType: `Function`\n\nA function that can be used to modify the values of each header. Return a `String` to modify the header. Return `null` to remove the header, and it's column, from the results. \n\n```js\ncsv({\n  mapHeaders: ({ header, index }) =\u003e header.toLowerCase()\n})\n```\n\n##### Parameters\n\n**header** _String_ The current column header.\u003cbr/\u003e\n**index** _Number_ The current column index.\n\n#### mapValues\n\nType: `Function`\n\nA function that can be used to modify the content of each column. The return value will replace the current column content.\n\n```js\ncsv({\n  mapValues: ({ header, index, value }) =\u003e value.toLowerCase()\n})\n```\n\n##### Parameters\n\n**header** _String_ The current column header.\u003cbr/\u003e\n**index** _Number_ The current column index.\u003cbr/\u003e\n**value** _String_ The current column value (or content).\n\n##### newline\n\nType: `String`\u003cbr\u003e\nDefault: `\\n`\n\nSpecifies a single-character string to denote the end of a line in a CSV file.\n\n#### quote\n\nType: `String`\u003cbr\u003e\nDefault: `\"`\n\nSpecifies a single-character string to denote a quoted string.\n\n#### raw\n\nType: `Boolean`\u003cbr\u003e\n\nIf `true`, instructs the parser not to decode UTF-8 strings.\n\n#### separator\n\nType: `String`\u003cbr\u003e\nDefault: `,`\n\nSpecifies a single-character string to use as the column separator for each row.\n\n#### skipComments\n\nType: `Boolean | String`\u003cbr\u003e\nDefault: `false`\n\nInstructs the parser to ignore lines which represent comments in a CSV file. Since there is no specification that dictates what a CSV comment looks like, comments should be considered non-standard. The \"most common\" character used to signify a comment in a CSV file is `\"#\"`. If this option is set to `true`, lines which begin with `#` will be skipped. If a custom character is needed to denote a commented line, this option may be set to a string which represents the leading character(s) signifying a comment line.\n\n#### skipLines\n\nType: `Number`\u003cbr\u003e\nDefault: `0`\n\nSpecifies the number of lines at the beginning of a data file that the parser should\nskip over, prior to parsing headers.\n\n#### maxRowBytes\n\nType: `Number`\u003cbr\u003e\nDefault: `Number.MAX_SAFE_INTEGER`\n\nMaximum number of bytes per row. An error is thrown if a line exeeds this value. The default value is on 8 peta byte.\n\n#### strict\n\nType: `Boolean`\u003cbr\u003e\nDefault: `false`\n\nIf `true`, instructs the parser that the number of columns in each row must match\nthe number of `headers` specified or throws an exception.\nif `false`: the headers are mapped to the column index\n   less columns: any missing column in the middle will result in a wrong property mapping!\n   more columns: the aditional columns will create a \"_\"+index properties - eg. \"_10\":\"value\"\n\n#### outputByteOffset\n\nType: `Boolean`\u003cbr\u003e\nDefault: `false`\n\nIf `true`, instructs the parser to emit each row with a `byteOffset` property.\nThe byteOffset represents the offset in bytes of the beginning of the parsed row in the original stream.\nWill change the output format of stream to be `{ byteOffset, row }`.\n\n## Events\n\nThe following events are emitted during parsing:\n\n### `data`\n\nEmitted for each row of data parsed with the notable exception of the header\nrow. Please see [Usage](#Usage) for an example.\n\n### `headers`\n\nEmitted after the header row is parsed. The first parameter of the event\ncallback is an `Array[String]` containing the header names.\n\n```js\nfs.createReadStream('data.csv')\n  .pipe(csv())\n  .on('headers', (headers) =\u003e {\n    console.log(`First header: ${headers[0]}`)\n  })\n```\n\n### Readable Stream Events\n\nEvents available on Node built-in\n[Readable Streams](https://nodejs.org/api/stream.html#stream_class_stream_readable)\nare also emitted. The `end` event should be used to detect the end of parsing.\n\n## CLI\n\nThis module also provides a CLI which will convert CSV to\n[newline-delimited](http://ndjson.org/) JSON. The following CLI flags can be\nused to control how input is parsed:\n\n```\nUsage: csv-parser [filename?] [options]\n\n  --escape,-e         Set the escape character (defaults to quote value)\n  --headers,-h        Explicitly specify csv headers as a comma separated list\n  --help              Show this help\n  --output,-o         Set output file. Defaults to stdout\n  --quote,-q          Set the quote character ('\"' by default)\n  --remove            Remove columns from output by header name\n  --separator,-s      Set the separator character (\",\" by default)\n  --skipComments,-c   Skip CSV comments that begin with '#'. Set a value to change the comment character.\n  --skipLines,-l      Set the number of lines to skip to before parsing headers\n  --strict            Require column length match headers length\n  --version,-v        Print out the installed version\n```\n\nFor example; to parse a TSV file:\n\n```\ncat data.tsv | csv-parser -s $'\\t'\n```\n\n## Encoding\n\nUsers may encounter issues with the encoding of a CSV file. Transcoding the\nsource stream can be done neatly with a modules such as:\n- [`iconv-lite`](https://www.npmjs.com/package/iconv-lite)\n- [`iconv`](https://www.npmjs.com/package/iconv)\n\nOr native [`iconv`](http://man7.org/linux/man-pages/man1/iconv.1.html) if part\nof a pipeline.\n\n## Byte Order Marks\n\nSome CSV files may be generated with, or contain a leading [Byte Order Mark](https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8). This may cause issues parsing headers and/or data from your file. From Wikipedia:\n\n\u003eThe Unicode Standard permits the BOM in UTF-8, but does not require nor recommend its use. Byte order has no meaning in UTF-8.\n\nTo use this module with a file containing a BOM, please use a module like [strip-bom-stream](https://github.com/sindresorhus/strip-bom-stream) in your pipeline:\n\n```js\nconst fs = require('fs');\n\nconst csv = require('csv-parser');\nconst stripBom = require('strip-bom-stream');\n\nfs.createReadStream('data.csv')\n  .pipe(stripBom())\n  .pipe(csv())\n  ...\n```\n\nWhen using the CLI, the BOM can be removed by first running:\n\n```console\n$ sed $'s/\\xEF\\xBB\\xBF//g' data.csv\n```\n\n## Meta\n\n[CONTRIBUTING](./.github/CONTRIBUTING)\n\n[LICENSE (MIT)](./LICENSE)\n","funding_links":[],"categories":["Packages","JavaScript","Repository","包","Parsing","目录","node","Utilities","file format (文件格式)","Number"],"sub_categories":["Parsing","解析","解析工具","file parse","React Components"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmafintosh%2Fcsv-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmafintosh%2Fcsv-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmafintosh%2Fcsv-parser/lists"}