https://github.com/devtin/csv-tr
cli utility to transform and sort csv files
https://github.com/devtin/csv-tr
cli csv filter sort transform
Last synced: 7 months ago
JSON representation
cli utility to transform and sort csv files
- Host: GitHub
- URL: https://github.com/devtin/csv-tr
- Owner: devtin
- Created: 2021-12-04T19:09:16.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2024-03-21T15:20:34.000Z (about 2 years ago)
- Last Synced: 2025-02-08T00:38:24.350Z (about 1 year ago)
- Topics: cli, csv, filter, sort, transform
- Language: JavaScript
- Homepage:
- Size: 258 KB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# csv-tr
Utility to manipulate long csv files
## Manifesto
Manipulating long csv files using graphic user interfaces might not be the most efficient at times. I want an utility that
helps me manipulate long csv files. This utility should:
- [Filter rows](#filtering-rows)
- [Transform rows](#transforming-rows)
- [Select specific columns](#selecting-specific-columns)
- [Exclude specific columns](#excluding-specific-columns)
- [Sort by columns](#sorting-by-column)
**Glossary of terms**
- **Column**: header column name of a csv file
- **Index**: line number of a csv file
- **Row**: line of a csv file
## Installation
```shell
npm install --global csv-tr
```
using `yarn`
```shell
yarn add global csv-tr
```
using `npx`
```shell
npx csv-tr --help
```
## CLI usage
```text
Usage: csv-tr [options] [source | input stream]
transforms given csv file or stream and outputs the result
Options:
-V, --version output the version number
-o, --only output only specified columns (comma separated). Not to be used with --exclude.
-e, --exclude exclude specified columns (comma separated). Not to be used with --only.
-t, --transform transform rows by given JavaScript expression. Ej: -t "row.email = row.email.toLowerCase()"
-f, --filter filter rows by given JavaScript file or expression. Ej: -f "row.state === 'FL'"
-s, --sort <[sort-column]:[sort-order: 1=ASC | -1=DESC]> sorts rows by column:order (comma separated) Ej: -s "firstName:1,lastName:-1"
-h, --help display help for command
```
### Input sample
Imagine a CSV file called `contacts.csv` with content
```csv
name,email,state
Juan,juan@gmail.com,FL
Miguel,miguel@hotmail.com,NY
Jesus,jesus@gmail.com,NY
```
### Filtering rows
Filtering values using the option `--filter` or `-f` followed by a JavaScript expression that will be evaluated
against a function that must return boolean, and looks like:
```js
(row, index) => { return /* your JavaScript expression */ }
```
Alternatively a file that exports a function with the same signature is also accepted:
```js
// my-filter-file
module.exports = (row/* , index */) => {
return /@gmail.com$/i.test(row.email)
}
```
Each `row` is nothing but a line of the csv file represented as a JSON object streamed by
csv-parser.
The `index` value is given row number starting at `0`. Meaning `index` of the first row (which is not the column names header)
equals `0`.
```shell
csv-tr contacts.csv -f '/@gmail.com$/i.test(row.email)' > gmail-contacts.csv
```
`gmail-contacts.csv` would look like:
```csv
name,email,state
Juan,juan@gmail.com,FL
Jesus,jesus@gmail.com,NY
```
**Slicing a given range of rows using the `index` value**
```shell
csv-tr contacts.csv -f 'index > 0 && index < 2'
```
Would output:
```csv
name,email,state
Miguel,miguel@hotmail.com,NY
```
### Transforming rows
Transforming values using the option `--transform` or `-t` followed by a JavaScript expression that will be evaluated
against a function that looks like.
```js
(row, index) => {
/* your js mutations go here */
}
```
Alternatively a file that exports a function with the same signature is also accepted:
```js
// my-transform-file
module.exports = (row/* , index */) => {
row.name = row.name.toUpperCase()
row.email = row.email.toUpperCase()
row.initial = row.name[0]
return row
}
```
Using the same `contacts.csv` [input sample](#input-sample).
```shell
csv-tr contacts.csv -t 'row.name = row.name.toUpperCase(); row.email = row.email.toUpperCase(); row.initial = row.name[0]' > contacts-uppercase.csv
```
`contacts-uppercase.csv` would look like:
```csv
name,email,state,initial
JUAN,JUAN@GMAIL.COM,FL,J
MIGUEL,MIGUEL@HOTMAIL.COM,NY,M
JESUS,JESUS@GMAIL.COM,NY,J
```
### Selecting specific columns
Using the same `contacts.csv` [input sample](#input-sample).
```shell
csv-tr contacts.csv -o email,state > contact-email-state.csv
```
`contact-email-state.csv` would look like:
```csv
email,state
juan@gmail.com,FL
miguel@hotmail.com,NY
jesus@gmail.com,NY
```
### Excluding specific columns
Using the same `contacts.csv` [input sample](#input-sample).
```shell
csv-tr contacts.csv -e email,state > contact-names.csv
```
`contact-names.csv` would look like:
```csv
name
Juan
Miguel
Jesus
```
### Sorting by column
Using the same `contacts.csv` [input sample](#input-sample), imagine sorting by `state` -> `DESC` and `name` -> `ASC`:
Sorting will guess numeric and date kind of values, and treat them accordingly.
```shell
csv-tr contacts.csv -s state:-1,name:1
```
Would output:
```csv
name,email,state
Jesus,jesus@gmail.com,NY
Miguel,miguel@hotmail.com,NY
Juan,juan@gmail.com,FL
```
## API Usage
Using all above's examples at once with the same `contacts.csv` [input sample](#input-sample)
```js
const fs = require('fs');
const { csvTr, sort, csvStringify } = require('csv-tr');
// un-comment any or multiple of the options below, run it and then take a look at result.csv
csvTr(fs.createReadStream('./tests/contacts.csv'), {
// filter: (row, index) => { return /@gmail.com$/i.test(row.email) },
// transform: (row, index) => { row.name = row.name.toUpperCase(); row.email = row.email.toUpperCase(); return row },
// only: ['email', 'state'],
// exclude: ['email', 'state'],
}).pipe(csvStringify()).pipe(fs.createWriteStream('result.csv'))
// SORTING
// mind sorting buffers all rows
const csvStreamToSort = csvTr(fs.createReadStream('contacts.csv'))
sort(csvStreamToSort, { state: -1, name: 1 }).then(sortedStream => {
sortedStream.pipe(csvStringify()).pipe(fs.createWriteStream('result-sorted.csv'))
})
```
`result-sorted.csv` would look like:
```csv
name,email,state
Jesus,jesus@gmail.com,NY
Miguel,miguel@hotmail.com,NY
Juan,juan@gmail.com,FL
```
* * *
## License
[MIT](https://opensource.org/licenses/MIT)
© 2021 Martin Rafael