Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/adamgordonbell/csvquote
https://github.com/adamgordonbell/csvquote
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/adamgordonbell/csvquote
- Owner: adamgordonbell
- License: mit
- Created: 2021-11-17T15:18:14.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2021-12-01T15:46:10.000Z (almost 3 years ago)
- Last Synced: 2024-07-10T18:17:40.450Z (2 months ago)
- Language: Go
- Size: 14.6 KB
- Stars: 31
- Watchers: 3
- Forks: 1
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- my-awesome-github-stars - adamgordonbell/csvquote - (Go)
README
# csvquote
_smart and simple CSV processing on the command line_
Are you looking for a way to process CSV data with standard UNIX shell commands?
Are you running into problems with embedded commas and newlines that mess
everything up?
Do you wish there was some way to add some CSV intelligence to these UNIX tools?
* awk, sed
* cut, join
* head, tail
* sort, uniq
* wc, split
This program can be used at the start and end of a text processing pipeline
so that regular unix command line tools can properly handle CSV data that
contain commas and newlines inside quoted data fields.
Without this program, embedded special characters would be incorrectly
interpretated as separators when they are inside quoted data fields.
By using csvquote, you temporarily replace the special characters inside quoted
fields with harmless nonprinting characters that can be processed as data by
regular text tools. At the end of processing the text, these nonprinting
characters are restored to their previous values.
In short, csvquote wraps the pipeline of UNIX commands to let them work on
clean data that is consistently separated, with no ambiguous special
characters present inside the data fields.
By default, the program expects to use these as special characters:
" quote character
, field delimiter
\n record separator
It is possible to specify different characters for the field and record
separators, such as tabs or pipe symbols.
Note that the quote character can be contained inside a quoted field
by repeating it twice, eg.
field1,"field2, has a comma in it","field 3 has a ""Quoted String"" in it"
Typical usage of csvquote is as part of a command line pipe, to permit
the regular unix text-manipulating commands to avoid misinterpreting
special characters found inside fields. eg.
csvquote foobar.csv | cut -d ',' -f 5 | sort | uniq -c | csvquote -u
or taking input from stdin,
cat foobar.csv | csvquote | cut -d ',' -f 7,4,2 | csvquote -u
other examples:
csvquote -t foobar.tsv | wc -l
csvquote -q "'" foobar.csv | sort -t, -k3 | csvquote -u
csvquote foobar.csv | awk -F, '{sum+=$3} END {print sum}'
## Installation
To install using go, clone the repo and then build it:
```
> go build -o csvquote cmd/cvsquote/main.go
> cp ./csvquote /usr/local/bin
```
To install using Earthly (linux):
```
earthly github.com/adamgordonbell/csvquote+build
cp ./csvquote /usr/local/bin
```
Install on MacOS (X86)
```
earthly github.com/adamgordonbell/csvquote+for-darwin-amd64
cp ./csvquote /usr/local/bin
```
Install on Windows
```
earthly github.com/adamgordonbell/csvquote+for-windows-amd64
# Then add to your path
```
## History
This is a fork of original version in C by Dan Brown found here:
https://github.com/dbro/csvquote
More specifically, this is a fork of a fork of the C version. It is based on a GoLang version Dan wrote at some point. Go makes cross-compilation easier.