{"id":24060755,"url":"https://github.com/koma-private/csvs","last_synced_at":"2025-05-12T19:22:51.326Z","repository":{"id":271329414,"uuid":"913092269","full_name":"koma-private/csvs","owner":"koma-private","description":"csvs (CSV Sql) is a command-line tool that simplifies working with CSV or TSV files by enabling SQL queries through an embedded SQLite engine. Binaries for macOS, Windows \u0026 Linux.","archived":false,"fork":false,"pushed_at":"2025-04-11T14:56:55.000Z","size":3058,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-11T15:43:10.241Z","etag":null,"topics":["csv","csv-converter","csv-export","csv-import","csv-parser","cui","linux","macos","sql","sqlite","tsv","tsv-parser","windows"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/koma-private.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-07T02:41:58.000Z","updated_at":"2025-04-11T14:57:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"88e414c5-e9a8-44c3-bfbe-ee5af192132c","html_url":"https://github.com/koma-private/csvs","commit_stats":null,"previous_names":["koma-private/csvs"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koma-private%2Fcsvs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koma-private%2Fcsvs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koma-private%2Fcsvs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koma-private%2Fcsvs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/koma-private","download_url":"https://codeload.github.com/koma-private/csvs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253806231,"owners_count":21967124,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","csv-converter","csv-export","csv-import","csv-parser","cui","linux","macos","sql","sqlite","tsv","tsv-parser","windows"],"created_at":"2025-01-09T07:14:55.537Z","updated_at":"2025-05-12T19:22:51.318Z","avatar_url":"https://github.com/koma-private.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# csvs\n\n**csvs** (**CSV** **S**ql) is a command-line tool that simplifies working with CSV or TSV files by enabling SQL queries\nthrough an embedded [SQLite](https://www.sqlite.org/) engine. It is ideal for data analysts and developers who need\nSQL's flexibility to manage text-based data efficiently.\n\n[![GitHub Release](https://img.shields.io/github/actions/workflow/status/koma-private/csvs/release.yml)](https://github.com/koma-private/csvs)\n[![GitHub Tag](https://img.shields.io/github/v/tag/koma-private/csvs)](https://github.com/koma-private/csvs)\n[![Crates.io Version](https://img.shields.io/crates/v/csvs)](https://crates.io/crates/csvs)\n\n![Banner of executing csvs](assets/usage.banner.png)\n![Banner of interactive mode](assets/interactive.banner1.png)\n![Banner of interactive mode](assets/interactive.banner2.png)\n\n- [Features](#features)\n    - [SQL Power for CSV Files](#sql-power-for-csv-files)\n    - [Automatic Encoding Detection](#automatic-encoding-detection)\n    - [Decide Data Type for Each Column](#decide-data-type-for-each-column)\n    - [Multi-File Handling](#multi-file-handling)\n    - [Customizable Output](#customizable-output)\n    - [Interactive Mode](#interactive-mode)\n    - [Multi-Statement Query Support](#multi-statement-query-support)\n- [Usage](#usage)\n    - [Interactive Mode](#interactive-mode-1)\n    - [Command Options](#command-options)\n- [Example](#example)\n- [SQL Query Notes](#sql-query-notes)\n    - [Mapping CSVs to Table Names](#mapping-csvs-to-table-names)\n    - [Quoting Columns with Special Characters](#quoting-columns-with-special-characters)\n    - [`--in-no-header` Option](#--in-no-header-option)\n    - [Execute Multiple Statements in a Single Query](#execute-multiple-statements-in-a-single-query)\n- [Error Handling](#error-handling)\n- [Build](#building-csvs)\n- [Limitations](#limitations)\n- [Acknowledgments](#acknowledgments)\n- [License](#license)\n\n---\n\n## Features\n\n### SQL Power for CSV Files\n\nRun advanced SQL queries, including `JOIN`, `GROUP BY`, `SUM()`, or `COUNT()` on CSV data.\nGain unparalleled flexibility to query, filter, sort, group, and combine data compared to traditional spreadsheet tools.\n**csvs** also supports regular expressions in SQL queries. Refer to [Regular Expressions Document](regexp.md).\n\n### Automatic Encoding Detection\n\nEliminate encoding issues with automatic detection of character encodings. Avoid garbled text and broken queries\neffortlessly.\n\n### Decide Data Type for Each Column\n\n**csvs** scans CSV rows to determine the most appropriate data types for SQLite tables. This dynamic analysis ensures\ncompatibility and precision, even when handling nullable fields.\n\nSee [Decide Data Type for Each Column](decide_data_type.md) and [Validating Number Document](validating_number.md) for\ndetails.\n\n### Multi-File Handling\n\nCombine data from multiple CSV or TSV files by creating a temporary SQLite database using `--in-file`. Easily perform\nSQL joins across files in seconds.\n\nCommon use cases:\n\n- Merging datasets from separate files.\n- Cross-referencing data using SQL joins.\n\n### Customizable Output\n\nExport query results as:\n\n- **CSV or TSV**: Ideal for data sharing or further processing.\n- **SQLite Database**: Retain results as `.db` files for future queries.\n\nControl delimiters, headers, and quoting styles to suit your needs.\n\n### Interactive Mode\n\nExplore datasets interactively without specifying queries upfront. Features include:\n\n- Browsing imported tables.\n- Ad-hoc query execution.\n- Previewing and saving query results.\n\n### Multi-Statement Query Support\n\nExecute multiple SQL statements in a single command. Transform and query data across multiple steps, with only the final\nresult displayed.\n\n---\n\n## Usage\n\n- Display help:\n\n```shell\ncsvs --help\n```\n\n### Interactive Mode\n\nStart **csvs** in *interactive mode* when neither `--query` nor `--source` is specified. This mode allows you to:\n\n- View imported tables.\n- Preview table content.\n- Save query results to files interactively.\n\nSee [Interactive Mode Guide](interactive_mode.md)\n\n![Animation of interactive mode](assets/interactive.gif)\n\n### Command Options\n\nSee [Command Options Guide](command_options.md)\n\n![Animation of executing csvs](assets/usage.gif)\n\n---\n\n## Example\n\n- Display version:\n\n```shell\ncsvs --version\n```\n\n- Select specific fields from `./data/address.csv` and save the results to `picked.csv`:\n\n```shell\ncsvs -i ./data/address.csv -q 'SELECT \"city\",\"town\",\"phone\" FROM \"address.csv\"' -o picked.csv\n```\n\n- Process CSV data from `STDIN`:\n\n```shell\ncsvs -q 'SELECT \"city\",\"town\",\"phone\" FROM \"stdin\"' \u003c ./data/address.csv \u003e picked.csv\n```\n\nor\n\n```shell\ncat ./data/address.csv | csvs -q 'SELECT \"city\",\"town\",\"phone\" FROM \"stdin\"' \u003e picked.csv\n```\n\n- Perform SQL joins across multiple files:\n\n```shell\ncsvs -i ./left.csv -i ./right.tsv -q 'SELECT * FROM \"left.csv\" AS l JOIN \"right.tsv\" AS r ON l.\"name\"=r.\"name\"'\n```\n\n- Leverage SQLite functions like `UPPER()`, `COUNT()`, etc., and export results to a SQLite database:\n\n```shell\ncsvs -i people.csv -q 'SELECT \"city\",COUNT(*) FROM \"people.csv\" GROUP BY \"city\" ORDER BY COUNT(*) DESC' --out-database out.db\n```\n\n- Start in *interactive mode*:\n\n```shell\ncsvs -i MOCK_DATA.csv\n```\n\n---\n\n## SQL Query Notes\n\n### Mapping CSVs to Table Names\n\nFile names provided with `--in-file` map directly to SQLite tables (e.g. `./sample/address.csv` becomes `\"address.csv\"`,\nand `data.2024.csv` becomes `\"data.2024.csv\"`).\n\n### Quoting Columns with Special Characters\n\nColumns or table names with spaces, punctuation, or reserved words must be quoted. Example:\n\n```sql\nSELECT \"first name\", \"last name\"\nFROM \"contacts.csv\"\n```\n\n### `--in-no-header` Option\n\nIf specified, column names default to \"c1\", \"c2\", \"c3\", etc., for header-less CSV files.\n\n### Execute Multiple Statements in a Single Query\n\nSeparate SQL statements with semicolons to execute multiple queries in sequence. Only the result of the final query is\ndisplayed.\n\nExample:\n\n```sql\nSELECT \"first name\"\nFROM \"contacts.csv\";\nSELECT \"age\"\nFROM \"contacts.csv\"; \n```\n\n---\n\n## Error Handling\n\nSee [Error Handling Guide](error_handling.md)\n\n---\n\n## Building csvs\n\nSee [Build Guide](build.md)\n\n---\n\n## Limitations\n\n- *Interactive mode* cannot be invoked when CSV data is provided via `STDIN`. Use `--in-file` to specify CSV files\n  instead.\n- Large files may require significant RAM since **csvs** loads entire files into memory when `--out-database` is not\n  specified.\n- CSV files with names starting with `sqlite_` cannot be used with `--in-file` due to SQLite's reserved naming\n  convention.\n\n## Acknowledgments\n\n**csvs** relies on the following open-source projects:\n\n- [SQLite](https://www.sqlite.org/)\n- [anyhow](https://github.com/dtolnay/anyhow)\n- [chardetng](https://github.com/hsivonen/chardetng)\n- [clap](https://github.com/clap-rs/clap)\n- [clap-help](https://github.com/Canop/clap-help)\n- [csv](https://github.com/BurntSushi/rust-csv)\n- [encoding_rs](https://github.com/hsivonen/encoding_rs)\n- [encoding_rs_rw](https://github.com/LiosK/encoding_rs_rw)\n- [indicatif](https://github.com/console-rs/indicatif)\n- [lazy-regex](https://github.com/Canop/lazy-regex)\n- [ratatui](https://github.com/ratatui/ratatui)\n- [r2d2](https://github.com/sfackler/r2d2)\n- [r2d2_sqlite](https://github.com/ivanceras/r2d2-sqlite)\n- [rusqlite](https://github.com/rusqlite/rusqlite)\n- [smashquote](https://github.com/ionizedgirl/smashquote)\n- [sqlparser](https://github.com/apache/datafusion-sqlparser-rs)\n- [tracing](https://github.com/tokio-rs/tracing)\n- [tracing-logfmt](https://github.com/EmbarkStudios/tracing-logfmt)\n- [tuirealm](https://github.com/veeso/tui-realm)\n- And many more! See the full list in the source repository.\n\n## License\n\n**csvs** is licensed under the [MIT license](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoma-private%2Fcsvs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkoma-private%2Fcsvs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoma-private%2Fcsvs/lists"}