{"id":13447492,"url":"https://github.com/sharpart555/nexline","last_synced_at":"2025-03-22T01:31:04.397Z","repository":{"id":34984418,"uuid":"194287576","full_name":"sharpart555/nexline","owner":"sharpart555","description":"Read file, stream, string, buffer line by line without putting them all in memory. It supports cool features like `custom line separator`, `various encodings`, `reverse mode`, `iterable protocol`","archived":false,"fork":false,"pushed_at":"2022-12-30T18:12:49.000Z","size":581,"stargazers_count":16,"open_issues_count":6,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-28T13:10:32.688Z","etag":null,"topics":["buffer","encodings","file","line","lineseparator","next","nodejs","npm","read","readline","reverse","split","stream"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/nexline","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sharpart555.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-28T14:31:29.000Z","updated_at":"2023-07-19T17:17:12.000Z","dependencies_parsed_at":"2023-01-15T11:29:16.338Z","dependency_job_id":null,"html_url":"https://github.com/sharpart555/nexline","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharpart555%2Fnexline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharpart555%2Fnexline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharpart555%2Fnexline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharpart555%2Fnexline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sharpart555","download_url":"https://codeload.github.com/sharpart555/nexline/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244893414,"owners_count":20527585,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["buffer","encodings","file","line","lineseparator","next","nodejs","npm","read","readline","reverse","split","stream"],"created_at":"2024-07-31T05:01:19.222Z","updated_at":"2025-03-22T01:31:03.949Z","avatar_url":"https://github.com/sharpart555.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# Nexline\n[![npm](https://img.shields.io/npm/v/nexline.svg)](https://www.npmjs.com/package/nexline)\n[![npm](https://img.shields.io/node/v/nexline.svg)](https://www.npmjs.com/package/nexline)\n[![npm](https://img.shields.io/npm/dt/nexline.svg)](https://www.npmjs.com/package/nexline)\n[![GitHub license](https://img.shields.io/github/license/sharpart555/nexline.svg)](https://github.com/sharpart555/nexline/blob/master/LICENSE)\n\n\n\nRead file, stream, string, buffer line by line without putting them all in memory.\\\nIt supports cool features below.\n\n* Lightweight\n* Iterable protocol\n* Handle large file with small memory footprint\n* Support various input types (file, stream, string, buffer)\n* Support various encodings\n* Support reverse mode\n* Support custom, multiple line separators\n* Support multiple inputs\n\n## Why I made this?\nNode.js's default readline module is great but it's `pause()` method does not work immediately.\\\nI made wrapper to fix that issue before, but it was not enough.\\\nEven if `pause()` works immediately, it is still inconvenient.\\\nIf I want to execute async function over line by line in large file, I have to call `pause()` and `resume()` at every line.\\\nI needed better way to do that without putting them all in memory.\n\n## Breaking changes in 1.0.0\n* `lineSeparator` default value is changed from `['\\n', '\\r\\n']` to `'\\n'` for better performance\n  * If you want to handle both CRLF and LF, set `lineSeparator: ['\\n', '\\r\\n']`\n* Support reverse mode.\n* Support file descriptor as input\n* Improve performance\n* Optimize memory usage\n\n## Install with npm\nRequired Node.js version \u003e= **8.0.0**\n```\nnpm install nexline\n```\n \n## How to use\n### Use `next()` method\n```js\nconst nexline = require('nexline');\n\nasync function main () {\n  const nl = nexline(...);\n  \n  while(true) {\n    const line = await nl.next();\n    console.log(line);\n    if (line === null) break; // If all data is read, returns null\n  }\n}\n```\n\n### Use as iterator ( Requires node \u003e= 10 )\n```js\nconst nexline = require('nexline');\n\nasync function main () {\n  const nl = nexline(...);\n\n  // nexline is iterable\n  for await (const line of nl) console.log(line);\n}\n```\n\n### Use file as input\n* Don't forget to close file descriptor after finish\n* You can use `autoCloseFile: true` to close file descriptor automatically\n```js\nconst nl = nexline({\n  input: fs.openSync(path_to_file, 'r'),\n  // autoCloseFile: true,\n});\n```\n\n### Use stream as input\n```js\nconst nl = nexline({\n  input: fs.createReadStream(path_to_file),\n});\n```\n\n### Use string as input\n```js\nconst nl = nexline({\n  input: 'foo\\nbar\\nbaz',\n});\n```\n\n### Use buffer as input\n```js\nconst nl = nexline({\n  input: Buffer.from('foo\\nbar\\nbaz'),\n});\n```\n\n### Use multiple input\n```js\nconst nl = nexline({\n  input: [\n    fs.openSync(path_to_file1, 'r'),\n    fs.createReadStream(path_to_file2),\n    'foo\\nbar\\nbaz',\n    Buffer.from('foo\\nbar\\nbaz'),\n  ],\n});\n```\n\n### Use other encodings\n[See encodings supported by iconv-lite](https://github.com/ashtuchkin/iconv-lite/wiki/Supported-Encodings)\n```js\nconst nl = nexline({\n  input: fs.openSync(path_to_file, 'r'), \n  encoding: 'cp949',\n});\n```\n\n### Use other lineSeparator\n```js\nconst nl = nexline({\n  input: 'foo;bar;baz', \n  lineSeparator: ';',\n});\n```\n\n### Use multiple lineSeparator\n```js\nconst nl = nexline({\n  input: 'foo\\r\\nbar\\nbaz', \n  lineSeparator: ['\\n', '\\r\\n'], // You can handle both LF and CRLF like this.\n});\n```\n\n### Reverse mode\n```js\nconst nl = nexline({\n  input: fs.openSync(path_to_file, 'r'), // NOTE: You cannot use stream in reverse mode. \n  reverse: true, \n});\n```\n\n## Methods\n| Name          |  Description    |\n| ------------- | --------------- |\n| next()        | **async**, It returns line until all data is read, after then returns `null`  |\n| close()        | Close nexline |\n\n## Options\n| Name          | Default                     |  Description    |\n| ------------- | --------------------------- | --------------- |\n| input         | undefined                   | **Required.** File descriptor, stream, string, buffer, You can provide multiple inputs using array |\n| lineSeparator | '\\n'                         | Any string more than one character. You can provide multiple line separator using array |\n| encoding      | 'utf8'                      | [See encodings supported by iconv-lite](https://github.com/ashtuchkin/iconv-lite/wiki/Supported-Encodings) |\n| reverse       | false                       | Reverse mode, **Cannot use this option with stream input** because stream cannot be read from end. Use file descriptor instead |\n| autoCloseFile | false                       | Automatically close file descriptor after all data is read |\n\n## Contribution\nPull requests and stars are always welcome. For bugs and feature requests, [please create an issue](https://github.com/sharpart555/nexline/issues/new)\n\n## License\nCopyright \u0026copy; 2019 Youngho Jin. Released under the [MIT License](https://github.com/sharpart555/nexline/blob/master/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharpart555%2Fnexline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsharpart555%2Fnexline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharpart555%2Fnexline/lists"}