{"id":18204456,"url":"https://github.com/3p3r/csv-query-stream","last_synced_at":"2025-04-07T18:53:45.380Z","repository":{"id":73994294,"uuid":"604835781","full_name":"3p3r/csv-query-stream","owner":"3p3r","description":"Query large compressed CSV documents using NodeJS streams.","archived":false,"fork":false,"pushed_at":"2023-02-24T03:48:59.000Z","size":4379,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-13T20:37:48.604Z","etag":null,"topics":["csv","fast","query","stream","streaming","zip"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/3p3r.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-21T22:22:26.000Z","updated_at":"2023-02-23T20:56:33.000Z","dependencies_parsed_at":"2023-07-06T21:01:28.493Z","dependency_job_id":null,"html_url":"https://github.com/3p3r/csv-query-stream","commit_stats":{"total_commits":1,"total_committers":1,"mean_commits":1.0,"dds":0.0,"last_synced_commit":"6996904bbf41a39497fa7f80e63516739e62e79a"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/3p3r%2Fcsv-query-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/3p3r%2Fcsv-query-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/3p3r%2Fcsv-query-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/3p3r%2Fcsv-query-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/3p3r","download_url":"https://codeload.github.com/3p3r/csv-query-stream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247713296,"owners_count":20983682,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","fast","query","stream","streaming","zip"],"created_at":"2024-11-03T11:04:24.105Z","updated_at":"2025-04-07T18:53:45.358Z","avatar_url":"https://github.com/3p3r.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# csv-query-stream\n\nQuery large compressed CSV documents using NodeJS streams.\n\n## Use Case\n\n```sh\n$ npm install csv-query-stream\n```\n\nIn mission critical applications, sometimes even the extra head space of SQLite\nindices can be too much. In that case, data can be saved directly to text files\nand queried with this module, while at the same time being inside a Zip archive\nand never get unpacked.\n\nThis module uses two stream to achieve this:\n\n1. a stream to read the Zip file and seek to the CSV file's position in it\n2. a stream to read the CSV file and query inside it\n\nUsage of streams allows low memory overhead and fast processing.\n\n## Data Format\n\nThe following assumptions are made about your data when using this module:\n\n- Your data is in CSV or TSV file(s)\n- Every row of data is unique in its own file\n- Your data file(s) are inside a Zip archive at the root level\n- Every row of data is monotonic, meaning row's ID is its line number minus 1\n- First row of data is a header row\n\nSample data is checked in under the `test/` directory.  \nAPI usage is pretty straightforward. See `test/` for examples.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F3p3r%2Fcsv-query-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F3p3r%2Fcsv-query-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F3p3r%2Fcsv-query-stream/lists"}