Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/vweevers/tabular-stream
Detects tabular data (spreadsheets, dsv or json, 20+ different formats) and emits normalized objects.
https://github.com/vweevers/tabular-stream
csv dsv json ndjson nodejs ods qpw spreadsheets stream sylk tabular-data tsv xls xlsx
Last synced: 13 days ago
JSON representation
Detects tabular data (spreadsheets, dsv or json, 20+ different formats) and emits normalized objects.
- Host: GitHub
- URL: https://github.com/vweevers/tabular-stream
- Owner: vweevers
- License: mit
- Created: 2015-05-25T08:46:25.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2023-11-01T13:58:09.000Z (about 1 year ago)
- Last Synced: 2025-01-09T07:34:26.191Z (13 days ago)
- Topics: csv, dsv, json, ndjson, nodejs, ods, qpw, spreadsheets, stream, sylk, tabular-data, tsv, xls, xlsx
- Language: JavaScript
- Homepage:
- Size: 233 KB
- Stars: 36
- Watchers: 3
- Forks: 2
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# tabular-stream
> Note: the core of `tabular-stream` 1.0 has moved to [`detect-tabular`](https://www.npmjs.com/package/detect-tabular). This is now a data normalizer on top of that.
**[Detects tabular data](https://www.npmjs.com/package/detect-tabular) (spreadsheets, dsv or json, 20+ different formats) and emits objects. Ensures all rows [have the same keys, optionally transforms keys](https://www.npmjs.com/package/map-tabular-keys) and tries to [coerce values to numbers](https://www.npmjs.com/package/coerce-tabular). Spreadsheets and DSV must have a header.**
[](https://www.npmjs.org/package/tabular-stream)
[](https://www.npmjs.org/package/tabular-stream)
[](https://github.com/vweevers/tabular-stream/actions/workflows/test.yml)
[](https://standardjs.com)
## Example
```
npm i tabular-stream snake-case format-data
```
```js
const tabular = require('tabular-stream')
const fs = require('fs')
const snake = require('snake-case').snakeCase
const format = require('format-data')
fs.createReadStream('test/air_pollution_nl.xlsx')
.pipe(tabular(snake))
.pipe(format('json'))
.pipe(process.stdout)
```
**Need a CLI doing just this?** Jump to [`tabular-cli`](https://www.npmjs.com/package/tabular-cli), which pairs `tabular-stream` with `format-data` to convert tabular data to json, ndjson, dsv or sse. For example:
```
tabular -k snake-case -o tsv < input.xls > output.tsv
```
## API
### `tabular([keys || options])`
Returns a duplex stream - give it any tabular data, get back objects. `(keys)` is a shorthand for `({ keys: keys })`. The available options are:
#### `function keys`
An optional function to [transform and/or filter keys](https://www.npmjs.com/package/map-tabular-keys). Receives a single argument, for every key of the first row. Everything at [`change-case`](https://www.npmjs.com/package/change-case) works well. If it returns an empty string or anything other than a string, the key is ignored (i.e. not included in the emitted objects).
```js
function keys(key) {
if (key === 'useless') return false
return key.toUpperCase()
}
```
#### `mixed defaultValue`
Fallback value to use for `null` and `undefined` values. **Default is `0`**.
#### `boolean bare`
Whether to emit null prototype objects via `Object.create(null)` or plain javascript objects **(the default)**.
#### Other
Other options are passed as-is to [`spreadsheet-stream`](https://github.com/vweevers/spreadsheet-stream) (if applicable). NB. Because the binary spreadsheets formats are not streamable, `spreadsheet-stream` will buffer the whole thing in memory. As a safe-guard you can set the `maxSize` option (in bytes): `tabular({ maxSize: 1024 * 1024 })`. See [`spreadsheet-stream`](https://github.com/vweevers/spreadsheet-stream) for details.
## Install
With [npm](https://npmjs.org) do:
```
npm install tabular-stream
```
## License
[MIT](LICENSE).
Test data © Statistics Netherlands, The Hague/Heerlen.