https://github.com/jcblw/snapscrape

scrapper for nodejs with middleware
https://github.com/jcblw/snapscrape

Last synced: 16 days ago
JSON representation

scrapper for nodejs with middleware

• Host: GitHub
• URL: https://github.com/jcblw/snapscrape
• Owner: jcblw
• Created: 2013-06-23T00:27:34.000Z (over 11 years ago)
• Default Branch: master
• Last Pushed: 2017-05-12T14:16:07.000Z (over 7 years ago)
• Last Synced: 2024-10-11T14:38:55.931Z (about 1 month ago)
• Language: JavaScript
• Size: 4.88 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
### Snapscape it

scraping the web with node on request.

#### Use it.

```javascript

var scrapper = require("snapscrape");

scrapper.scrape("http://google.com", function(res, body){

	console.log(arguments);

});

```

cool beans

#### Add to it

```javascript

scrapper.use(/*middleware*/);

//comes with a htmlparser

scrapper.use(scrapper.htmlParser);

scrapper.scrape("http://google.com", function(res, body){

	// see googles javascripts

	console.log(res.html("body").text());

});

```

#### Build for it

the middleware is very similiar to connects middleware

```javascript

module.exports = function(response, body, next){

	next();	

};

```

`response` is the response from the server your pulling from

`body` is the body from server, or aka the goods

`next` is a function called to pass payload to next middleware function or callback;

###### to come

probably hooking this into a promise system so that you can have nice errors;

```javascript

var error = function(){

	console.log("you fail")

}

scrapper.scrape("FAIL", function(res, body){

	

}, err);

```

and test!