{"id":21655902,"url":"https://github.com/koltyakov/sppull","last_synced_at":"2025-10-19T15:33:45.072Z","repository":{"id":45078187,"uuid":"66185962","full_name":"koltyakov/sppull","owner":"koltyakov","description":"📎 Download files from SharePoint document libraries using Node.js without hassles","archived":false,"fork":false,"pushed_at":"2024-09-06T08:59:25.000Z","size":954,"stargazers_count":47,"open_issues_count":12,"forks_count":15,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-03T18:23:52.733Z","etag":null,"topics":["developer-tools","downloader","javascript","nodejs","sharepoint","sharepoint-online"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/koltyakov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-08-21T07:07:42.000Z","updated_at":"2024-12-30T22:22:42.000Z","dependencies_parsed_at":"2024-02-19T21:27:51.284Z","dependency_job_id":"c0bfc37f-0be7-42be-b79e-646fb8df191d","html_url":"https://github.com/koltyakov/sppull","commit_stats":{"total_commits":143,"total_committers":8,"mean_commits":17.875,"dds":0.6433566433566433,"last_synced_commit":"73276dee1ab200278b5cc8a762dd488edf0ba73a"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koltyakov%2Fsppull","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koltyakov%2Fsppull/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koltyakov%2Fsppull/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/koltyakov%2Fsppull/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/koltyakov","download_url":"https://codeload.github.com/koltyakov/sppull/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243573441,"owners_count":20312883,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["developer-tools","downloader","javascript","nodejs","sharepoint","sharepoint-online"],"created_at":"2024-11-25T08:37:44.237Z","updated_at":"2025-10-19T15:33:40.037Z","avatar_url":"https://github.com/koltyakov.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SPPull - simple client to pull and download files from SharePoint\r\n\r\n[![NPM](https://nodei.co/npm/sppull.png?mini=true\u0026downloads=true\u0026downloadRank=true\u0026stars=true)](https://nodei.co/npm/sppull/)\r\n\r\n[![npm version](https://badge.fury.io/js/sppull.svg)](https://badge.fury.io/js/sppull)\r\n[![Downloads](https://img.shields.io/npm/dm/sppull.svg)](https://www.npmjs.com/package/sppull)\r\n![Build Status](https://koltyakov.visualstudio.com/SPNode/_apis/build/status/sppull?branchName=master)\r\n[![Gitter](https://badges.gitter.im/gitterHQ/gitter.png)](https://gitter.im/sharepoint-node/Lobby)\r\n\r\nNode.js module for downloading files from SharePoint document libraries.\r\n\r\n## New in version 2.7.0\r\n\r\n- [Breaking change]: Removed legacy module.exports for straightforward usage in modern JS and TypeScript\r\n\r\n```typescript\r\nimport SPPull from 'sppull';\r\nconst sppull = SPPull.download; // \u003c-- this is an entry point\r\n```\r\n\r\n- TSLint replaced with ESLint\r\n- Integration tests are migrated to new version of Mocha and fixed\r\n\r\n## New in version 2.2.0\r\n\r\nFiles streaming download:\r\n\r\n- Download files of any supported size\r\n- Effective memmory consumption while fetching large files\r\n\r\n## New in version 2.1.0\r\n\r\nPerformance in SPO and HTTPS environments is improved. Download on multiple objects is x2 faster now!\r\n\r\n## New in version 2.0.6\r\n\r\nSmart re-download mechanism. Existing files with no changes are ignored from the download.\r\n\r\n## Supported SharePoint versions\r\n\r\n- SharePoint Online\r\n- SharePoint 2013\r\n- SharePoint 2016\r\n- SharePoint 2019\r\n\r\n## How to use\r\n\r\n### Install\r\n\r\n```bash\r\nnpm install sppull --save-dev\r\n```\r\n\r\n### Demo\r\n\r\n![How it works](http://koltyakov.ru/images/sppull-demo.gif)\r\n\r\n### Usage\r\n\r\n```javascript\r\nconst { SPPull } = require('sppull');\r\n\r\nconst context = {/*...*/};\r\nconst options = {/*...*/};\r\n\r\nSPPull.download(context, options)\r\n  .then(successHandler)\r\n  .catch(errorHandler);\r\n```\r\n\r\n#### Arguments\r\n\r\n##### Context\r\n\r\n- `siteUrl` - SharePoint site (SPWeb) url [string, required]\r\n- `creds`\r\n  - `username` - user name for SP authentication [string, optional in case of some auth methods]\r\n  - `password` - password [string, optional in case of some auth methods]\r\n  ...\r\n\r\n**Additional authentication options:**\r\n\r\nSince communication module (sp-request), which is used in sppull, had received additional SharePoint authentication methods, they are also supported in sppull.\r\n\r\n- SharePoint On-Premise (Add-In permissions)\r\n  - `clientId`\r\n  - `issuerId`\r\n  - `realm`\r\n  - `rsaPrivateKeyPath`\r\n  - `shaThumbprint`\r\n- SharePoint On-Premise (NTLM handshake - more commonly used scenario):\r\n  - `username` - username without domain\r\n  - `password`\r\n  - `domain` / `workstation`\r\n- SharePoint Online (Add-In permissions):\r\n  - `clientId`\r\n  - `clientSecret`\r\n- SharePoint Online (SAML based with credentials - more commonly used scenario):\r\n  - `username` - user name for SP authentication [string, required]\r\n  - `password` - password [string, required]\r\n- ADFS user credantials:\r\n  - `username`\r\n  - `password`\r\n  - `relyingParty`\r\n  - `adfsUrl`\r\n\r\nFor more information please check node-sp-auth [credential options](https://github.com/s-KaiNet/node-sp-auth#params) and [wiki pages](https://github.com/s-KaiNet/node-sp-auth/wiki).\r\n\r\n##### Options\r\n\r\n- `spRootFolder` - root folder in SharePoint to pull from [string, required]\r\n- `dlRootFolder` - local root folder where files and folders will be saved to [string, optional, default: `./Downloads`]\r\n- `spBaseFolder` - base folder path which is omitted then saving files locally [string, optional, default: equals to spRootFolder]\r\n- `recursive` - to pull all files and folders recursively [boolean, optional, default: `true`]\r\n- `ignoreEmptyFolders` - to ignore local creation of SharePoint empty folders [boolean, optional, default: `true`]\r\n- `foderStructureOnly` - to ignore files, recreate only folders' structure [boolean, optional, default: `false`]\r\n- `strictObjects` - array of files and folders relative paths within the `spRootFolder` to proceed explicitly, [array of strings, optional]\r\n- `camlCondition` - SharePoint CAML conditions to use [string, optional]\r\n- `spDocLibUrl` - SharePoint document library URL [string, mandatory with `camlCondition`]\r\n- `metaFields` - array of internal field names to request along with the files [array of strings, optional]\r\n- `createEmptyFolders` - to create empty folders along with documents download task [boolean, optional, default: `true`]\r\n- `omitFolderPath` - folder path pattern which is omitted from final downloaded files path [string, optional]\r\n- `muteConsole` - to mute console messages during transport queries to SharePoint API [boolean, optional, default: `false`]\r\n- `shouldDownloadFile` - a callback to decide if a file should be downloaded based on the metadata [optional, (props) =\u003e boolean, default: undefined]\r\n\r\n#### Overloads / cases\r\n\r\n- All files with folder structure from spRootFolder\r\n- Files from spRootFolder folder, first hierarchy level only\r\n- Folders structure from spRootFolder without files\r\n- Files based on array of paths provided strictly [works with array of files only right now]\r\n- Files based on CAML query conditions\r\n- Pull for documents metadata to use it in callback's custom logic\r\n\r\nUse case scenarios can be found on the [Scenarios](https://github.com/koltyakov/sppull/tree/master/docs/Scenarios.md) page. This page suggests combinations of options which are optimal for certain use cases.\r\n\r\n#### successHandler\r\n\r\nCallback gets called upon successful files download.\r\n\r\n#### errorHandler\r\n\r\nCallback gets executed in case of exception inside `sppull`. Accepts error object as first argument for callback.\r\n\r\n## Samples\r\n\r\nRefer to the [Scenarios](https://github.com/koltyakov/sppull/tree/master/docs/Scenarios.md) page for suggested options combinations available with `sppull`.\r\n\r\n### Basic usage\r\n\r\nTypeSctipt:\r\n\r\n```typescript\r\nimport { AuthConfig as SPAuthConfigurator } from 'node-sp-auth-config';\r\nimport SPPull, { ISPPullOptions, ISPPullContext } from 'sppull';\r\n\r\nnew SPAuthConfigurator().getContext().then(({ siteUrl, authOptions }) =\u003e {\r\n\r\n  const context: ISPPullContext = {\r\n    siteUrl: context.siteUrl,\r\n    ...context.authOptions\r\n  } as any;\r\n\r\n  const options: ISPPullOptions = {\r\n    spRootFolder: 'Shared%20Documents',\r\n    dlRootFolder: './Downloads/Documents'\r\n  };\r\n\r\n  SPPull.download(context, options);\r\n\r\n}).catch(console.log);\r\n```\r\n\r\nor ES6:\r\n\r\n```javascript\r\nconst { SPPull } = require('sppull');\r\n\r\nconst context = {\r\n  siteUrl: \"http://contoso.sharepoint.com/subsite\",\r\n  creds: {\r\n    username: \"user@contoso.com\",\r\n    password: \"_Password_\",\r\n    online: true\r\n  }\r\n};\r\n\r\nconst options = {\r\n  spRootFolder: \"Shared%20Documents/Contracts\",\r\n  dlRootFolder: \"./Downloads/Contracts\"\r\n};\r\n\r\n/*\r\n * All files will be downloaded from http://contoso.sharepoint.com/subsite/Shared%20Documents/Contracts folder\r\n * to __dirname + /Downloads/Contracts folder.\r\n * Folders structure will remain original as it is in SharePoint's target folder.\r\n*/\r\nSPPull.download(context, options)\r\n  .then((downloadResults) =\u003e {\r\n    console.log(\"Files are downloaded\");\r\n    console.log(\"For more, please check the results\", JSON.stringify(downloadResults));\r\n  })\r\n  .catch((err) =\u003e {\r\n    console.log(\"Core error has happened\", err);\r\n  });\r\n```\r\n\r\n### Passwords storage\r\n\r\nTo eliminate any local password storing if preferable to use any two-way hashing technique, like [cpass](https://github.com/koltyakov/cpass).\r\n\r\n## Inspiration and references\r\n\r\nThis project was inspired by [spsave](https://github.com/s-KaiNet/spsave) by [Sergei Sergeev](https://github.com/s-KaiNet) and [gulp-spsync](https://github.com/wictorwilen/gulp-spsync) by [Wictor Wilén](https://github.com/wictorwilen) projects.\r\n\r\nSPPull depends heavily on [sp-request](https://github.com/s-KaiNet/sp-request) module and use it to send REST queries to SharePoint.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoltyakov%2Fsppull","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkoltyakov%2Fsppull","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoltyakov%2Fsppull/lists"}