{"id":20489454,"url":"https://github.com/nodys/spreadstream","last_synced_at":"2025-06-27T15:08:01.692Z","repository":{"id":39097202,"uuid":"114016161","full_name":"nodys/spreadstream","owner":"nodys","description":"Pipe your csv to google spreadsheet","archived":false,"fork":false,"pushed_at":"2023-03-04T02:30:11.000Z","size":1298,"stargazers_count":10,"open_issues_count":8,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-27T15:07:55.645Z","etag":null,"topics":["csv","spreadsheet","stream"],"latest_commit_sha":null,"homepage":"https://nodys.github.io/spreadstream","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nodys.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-12-12T16:55:59.000Z","updated_at":"2022-06-19T10:39:05.000Z","dependencies_parsed_at":"2025-04-13T16:34:49.700Z","dependency_job_id":"f17e4c4a-eb86-430e-b039-58fb93c23ee2","html_url":"https://github.com/nodys/spreadstream","commit_stats":null,"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"purl":"pkg:github/nodys/spreadstream","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodys%2Fspreadstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodys%2Fspreadstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodys%2Fspreadstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodys%2Fspreadstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nodys","download_url":"https://codeload.github.com/nodys/spreadstream/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodys%2Fspreadstream/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262279181,"owners_count":23286551,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","spreadsheet","stream"],"created_at":"2024-11-15T17:13:00.878Z","updated_at":"2025-06-27T15:08:01.672Z","avatar_url":"https://github.com/nodys.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# spreadstream\n[![Build Status](https://travis-ci.org/nodys/spreadstream.svg?branch=master)](https://travis-ci.org/nodys/spreadstream) [![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com) [![Clussh on npm](https://img.shields.io/npm/v/spreadstream.svg)](https://www.npmjs.com/package/spreadstream)\n\nPipe data from and to google spreadsheet\n\n---\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://nodys.github.io/spreadstream/spreadstream.png\" alt=\"clussh\"\u003e\n\u003c/p\u003e\n\nRead\n```sh\nspreadstream \u003e data.csv\nspreadstream --json \u003e data.ndjson\n```\n\nWrite\n```sh\ncat data.csv | spreadstream\ncat data.ndjson | spreadstream --json\n```\n\n---\n\n##  Install\n\n```sh\nnpm install -g spreadstream\n```\n\nThen:\n\n1. [Create a Google authentication token](#google-authentication-token) and make sure the account (or the service account) can access the spreadsheets you plan to use.\n2. Create a [rc file](#rc-file) with your authentication token\n\n\n## Configuration\n\n### Google authentication token\nYou need to create a Google authentication token for the Google Sheet Api: either a [service account](#create-a-service-account) or a [OAuth 2 token](#create-oauth2-token) to use your own account.\n\nOnce created, put your credentials in a [rc file](#rc-file) as described below.\n\n\n### Rc file\nThe rc file must contain the `credential` key with the google authentication token created previously. You can add any other spreadstream options (see `spreadstream --help`). The location of the rc file depend on your needs: either at a [standard rc file path](https://www.npmjs.com/package/rc) or specified using the `--settings` option.\n\n*You can too try the new (alpha) spreadstream config generator which supports Oauth2 authentication: `spreadstream init`*\n\n**Exemple:**\n\n```js\n// .spreadstreamrc\n{\n  // Your google authentication token created previously:\n  \"credential\": {\n    \"type\": \"service_account\",\n    \"project_id\": \"xxxxx\",\n    \"private_key_id\": \"xxxxx\",\n    \"private_key\": \"xxxxx\",\n    \"client_email\": \"xxxxx@xxxxx\",\n    \"client_id\": \"xxxxx\",\n    \"auth_uri\": \"https://accounts.google.com/o/oauth2/auth\",\n    \"token_uri\": \"https://accounts.google.com/o/oauth2/token\",\n    \"auth_provider_x509_cert_url\": \"https://www.googleapis.com/oauth2/v1/certs\",\n    \"client_x509_cert_url\": \"https://www.googleapis.com/robot/v1/metadata/x509/xxxxx\"\n  },\n  // You can set default value for any spreadstream command line options...\n  \"id\": \"spreadsheet id\", // The document id from the spreadsheet url\n  \"sheet\": \"My Sheet\",    // The sheet title in your document\n  \"json\": true\n  // ...\n}\n```\n\n## Usage\n\nThe examples below depends on the availability of a [rc file](#rc-file) containing a valid `credential`, a spreadsheet document `id` and a `sheet` title.\n\n### Write data to Google Spreadsheet\n\n```sh\n# Pipe csv to spreadstream (append new rows to the sheet)\ncat mydocument.csv | spreadstream\n\n# Or read a csv file\nspreadstream --input mydocument.csv\n\n# Choose the document and the sheet (or use configuration file):\ncat mydocument.csv | spreadstream --id=\"ya29.GlsiBTHclgwXhCs3dJZHp\" --sheet \"My Sheet\"\n\n# Clear sheet first (replace)\ncat mydocument.csv | spreadstream --replace\n\n# Pipe line delimited json instead of csv\ncat mydocument.ndjson | spreadstream --json\n```\n\n### Read data from Google Spreadsheet\n\n```sh\n# Read sheet\nspreadstream\n\n# Choose the document and the sheet (or use configuration file):\nspreadstream --id=\"ya29.GlsiBTHclgwXhCs3dJZHp\" --sheet \"My Sheet\"\n\n# Limit reading range with A1 notation\n# https://developers.google.com/sheets/api/guides/concepts#a1_notation\nspreadstream --range=\"A:C\"\nspreadstream --range=\"A1:C4\"\nspreadstream --range=\"1:4\"\nspreadstream --range=\"My Sheet!1:4\" # Override sheet\n\n# Change csv output (same options than for input):\nspreadstream --csv-separator \";\"\n\n# output as line delimited json instead of csv\nspreadstream --json\n\n# Write output to a file\nspreadstream \u003e myfile.csv\nspreadstream --output myfile.csv\n```\n\n### Options\n\nSee too `spreadstream --help` for detailed command line usage, options and default values.\n\nThe [API](#api) use the camelCase version for the dashed options names. Every option can be set in the [rc file](#rc-file) using either camelCase or kebab-case format.\n\n- `--id`\n   Identifier of the spreadsheet document\n   \u003csmall\u003eThe spreadsheet document id is the long unique identifier in the URL of the document looking like `Dh9CsT4eXiTeKQLWZLpM..`\u003c/small\u003e\n- `--sheet`\n   Title of the sheet in the document\n- `--replace`\n   Write in overwrite mode: replace the content of the sheet (the default behavior is to append new rows at the end)\n- `--verbose`\n  Be verbose about what is done (on stderr)\n- `--value-input`\n  Determines how input data should be interpreted (default: `USER_ENTERED`) ([more](https://developers.google.com/sheets/api/reference/rest/v4/ValueInputOption))\n- `--major-dimension`\n  Indicates which dimension read operation should apply to (default: `ROWS`) ([more](https://developers.google.com/sheets/api/reference/rest/v4/Dimension))\n- `--value-render`\n  Determines how values should be rendered in the the output while reading (default: `FORMATTED_VALUE`) ([more](https://developers.google.com/sheets/api/reference/rest/v4/valueRender))\n- `--date-time-render`\n   Determines how dates should be rendered in the the while reading (default: `SERIAL_NUMBER`) ([more](https://developers.google.com/sheets/api/reference/rest/v4/dateTimeRender))\n- `--max-buffer`\n   Buffer max size before flushing to spreadsheet (default: `1000`).\n   How many row of data should be sent at once to the spreadsheet while writing.\n   A lower value would negatively impact speed and API usage limits, but it will produce atomic update of your document when used with light and slow stream of data (eg. a line-delimited-json log producer).\n- `--range`\n   Fore reading: The A1 notation of the values to retrieve. Default is to select the whole sheet.\n   Exemples: `A1:D3` a 4x3 range, `A:D` the four first columns, `12:30` for lines from 12 to 30\n- `--csv-separator`\n   Csv separator (both for the parser and the writer).\n   Default to auto-detect and `,`\n- `--csv-quote`\n   Csv quote (both for the parser and the writer)\n   Default to auto-detect and `\"`\n- `--csv-escape`\n   Csv quote escaping (both for the parser and the writer)\n   Default to auto-detect and `\"\"`\n- `--csv-newline`\n   Csv new line character\n   Default to auto-detect and `\\n`\n- `--read-headers` (default: true)\n  The first chunk in the input feed should be used as headers (prefix with `--no-` to disable)\n  If false:\n  - When reading from json, this option has no effect.\n  - When reading from csv, the first line is not interpreted as headers (and default headers are generated)\n  - When reading from spreadsheet, the first line is not interpreted as headers (and default headers are generated)\n- `--write-headers` (default: true)\n  The first chunk in the output feed should include headers\n  If false:\n  - When writing to json, this option has no effect.\n  - When writing to csv the headers are omitted\n  - When writing to spreadsheet the headers are omitted\n- `--noheaders` (default: false)\n  Alias for `--no-read-headers` and `--no-write-headers`\n- `--json`\n   Use [new line delimited json](http://ndjson.org/) parser and writer instead of csv as input and output.\n- `--classic-json`\n   Input / output format should use classic json serializer and parser instead of line delimited json (read and produce a json array)\n- `--input`\n   Set input file. Default is to read from stdin outside of a tty environment.\n   Reading from stdin can be forced by setting this option to `-`.\n- `--output`\n   Set output file. Default is to write to stdout.\n   Writing to stdout can be forced by setting this option to `-`.\n- `completion`\n   Generate bash completion code (`spreadstream completion \u003e\u003e $HOME/.bashrc`)\n\n## Api\n\n```js\nconst spreadstream = require('spreadstream')\n\nconst config = {\n  // See below about creating a service account (required)\n  credentials: require('./google-credentials.json')  \n\n  // Spreadsheet id (see document's url) shared with the service account (required)\n  id: '1jLPcDv0UaYIDh9CsT4eXiMeKQLWZLpMyAA7FjkO7Z3X',\n\n  // The sheet title (required)\n  sheet: 'My Sheet',\n\n  // Clear sheet before adding value (default: false)\n  // If true, every values in the sheet will be removed first.\n  // The default behavior (replace: false) is to append rows at the bottom of\n  // the sheet.\n  // Headers are pushed only when replace is true or when the sheet is new.\n  replace: false,\n\n  // Input value option:\n  // - USER_ENTERED: The values will be parsed as if the user typed them into the UI (the default)\n  // - RAW: The values will be stored as-is.\n  // See https://developers.google.com/sheets/api/reference/rest/v4/ValueInputOption\n  valueInputOption: spreadstream.USER_ENTERED,\n\n  // How many row must be keept in the stream buffer before flushing\n  // data to the document (default: 5000)\n  maxBuffer: 5000,\n\n  // Verbose mode (default: false)\n  verbose: false\n}\n\n// Create a stream\nconst stream = spreadstream(config)\n\n// Pipe an object stream or write directly to the stream.\n// The spreadstream stream supports to kind of object stream: Array or Object.\n// Note that you can not push a mixed stream of array and object.\n\n// Stream of Array:\n// Each array is a row in the sheet. The first row will be interpreted has\n// headers (unless readHeaders if false)\nconst stream1 = spreadstream(config)\nstream1.write(['foo', 'bar'])\nstream1.write(['4', '2'])\nstream1.write(['7', '10'])\nstream1.end()\n\n\n// Stream of Object\nconst stream2 = spreadstream(config)\nstream2.write({ foo: 4, bar: 2 })\nstream2.write({ foo: 7, bar: 10 })\nstream2.end()\n\n\n// Pipe data from stdin (eg. with ndjson):\nprocess.stdin\n  .pipe(require('ndjson').parse())\n  .pipe(spreadstream(config))\n\n// Read a document\nspreadstream.readDocument(config).then(values =\u003e console.log(values))\n\n```\n\n## Create a service account\n\n1. Go to the [Google Developers Console](https://console.developers.google.com/project)\n2. Select your project or create a new one (and then select it)\n3. Enable the Google Sheets API for your project\n   - Search for \"sheet\"\n   - In the sidebar on the left, expand __APIs \u0026 auth__ \u003e __APIs__\n   - Click on \"[Google Sheets API](https://console.developers.google.com/apis/api/sheets.googleapis.com/overview)\"\n   - Click the blue \"Enable API\" button\n4. Create a service account for your project\n   - In the sidebar on the left, expand __APIs \u0026 auth__ \u003e __Credentials__\n   - Click blue \"Create credentials\" button\n   - Select the \"Create service account key\" option\n   - Select \"New service account\"\n   - Keep the \"JSON\" key type option selected\n   - Click blue \"Create\" button and complete the form\n   - Your JSON key file is generated and downloaded to your machine\n     (__it is the only copy!__)\n   - Note your service account's *email address* (also available in the JSON\n     key file)\n5. Add the content of the JSON file in your spreadstream rc file (eg. `./.spreadstreamrc` or `~/.config/spreadstream/config`):\n   ```json\n   {\n     \"credential\": {\n        \"type\": \"service_account\",\n        \"project_id\": \"xxxxx\",\n        \"private_key_id\": \"xxxxx\",\n        \"private_key\": \"xxxxx\",\n        \"client_email\": \"xxxxx@xxxxx\",\n        \"client_id\": \"xxxxx\",\n        \"auth_uri\": \"https://accounts.google.com/o/oauth2/auth\",\n        \"token_uri\": \"https://accounts.google.com/o/oauth2/token\",\n        \"auth_provider_x509_cert_url\": \"https://www.googleapis.com/oauth2/v1/certs\",\n        \"client_x509_cert_url\": \"https://www.googleapis.com/robot/v1/metadata/x509/xxxxx\"\n      }\n   }\n   ```\n6. Share the doc (or docs) with your service account using the email\n   noted above\n\n## Create Oauth2 token\n\n1. Go to the [Google Developers Console](https://console.developers.google.com/project)\n2. Select your project or create a new one (and then select it)\n3. Enable the Google Sheets API for your project\n   - Search for \"sheet\"\n   - In the sidebar on the left, expand __APIs \u0026 auth__ \u003e __APIs__\n   - Click on \"[Google Sheets API](https://console.developers.google.com/apis/api/sheets.googleapis.com/overview)\"\n   - Click the blue \"Enable API\" button\n4. Create a Oauth credential\n   - In the sidebar on the left, expand __APIs \u0026 auth__ \u003e __Credentials__\n   - Click blue \"Create credentials\" button\n   - Select the \"OAuth 2.0 client IDs\" option\n   - Select \"Other\" and give a name\n   - Click blue \"Create\" button\n   - Copy the client id (`CLIENT_ID` below) and the secret key (`CLIENT_SECRET` below)\n5. Edit you spreadstream rc file (eg. `~/.config/spreadstream/config`):\n   ```json\n   {\n     \"type\": \"oauth2\",\n     \"client_id\": \"CLIENT_ID\",\n     \"client_secret\": \"CLIENT_SECRET\",\n     \"tokens\": { }\n   }\n   ```\n6. Complete the\n\n\n*(credit: [node-google-spreadsheet]( https://github.com/theoephraim/node-google-spreadsheet/blob/master/README.md#service-account-recommended-method))*\n\n---\n\nLicense: [MIT](./LICENSE) - Novadiscovery\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodys%2Fspreadstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnodys%2Fspreadstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodys%2Fspreadstream/lists"}