https://github.com/harrystevens/scraperama
Scrape html, csv, json files from the internet.
https://github.com/harrystevens/scraperama
Last synced: 3 months ago
JSON representation
Scrape html, csv, json files from the internet.
- Host: GitHub
- URL: https://github.com/harrystevens/scraperama
- Owner: HarryStevens
- License: mit
- Created: 2020-11-05T18:48:44.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2024-03-22T21:15:15.000Z (about 1 year ago)
- Last Synced: 2024-04-14T07:30:37.389Z (about 1 year ago)
- Language: JavaScript
- Size: 66.4 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# scraperama
Scrape files from the internet.
## Installation
```bash
npm i scraperama -S
```
## Usage
```js
const scraperama = require("scraperama");
scraperama.csv("file.csv", json => {
console.log(json);
});
scraperama.html("file.html", $ => {
console.log($("body").html());
});
scraperama.json("file.json", json => {
console.log(json);
});
scraperama.text("file.txt", text => {
console.log(text);
});
```
To download any old file:
```js
scraperama.download(
"path/to/file.zip", // URL
`${__dirname}/local/file.zip`, // local file path
(pct) => { process.stdout.write(`\r${pct.toFixed(1)}%`); }, // log percentage downloaded
() => { console.log("Done!"); } // callback function
);
```
To get an object's file size:
```js
scraperama.filesize(object);
```
To throttle a function:
```js
const logThrottled = scraperama.throttle(console.log, 500);
Array.from({ length: 10 }).forEach((_, i) => logThrottled(i));
```