https://github.com/emnudge/index-scraper
Scrape website indexes using puppeteer
https://github.com/emnudge/index-scraper
Last synced: 3 months ago
JSON representation
Scrape website indexes using puppeteer
- Host: GitHub
- URL: https://github.com/emnudge/index-scraper
- Owner: EmNudge
- Created: 2020-01-12T00:38:09.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2021-06-05T19:05:29.000Z (about 5 years ago)
- Last Synced: 2026-02-18T04:35:13.500Z (4 months ago)
- Language: TypeScript
- Size: 15.6 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# index-scraper
This is a scraper for index sites. They are file systems and, as such, will force you to go in them one by one in order to download.
This is a script built with puppeteer and typescript that allows you to mass-download those files.
# Usage
1. download node if you haven't already
2. navigate to the directory
3. run `npm install`
4. create a `.env` file with the keys `URL` and `FOLDER`(optional) pointing to the correct url and folder to output to in the downloads folder
5. run `npm run start`
6. find files in downloads/``