https://github.com/jun85664396/auto-web-crawler
auto web crawler , nodeJs
https://github.com/jun85664396/auto-web-crawler
Last synced: about 1 year ago
JSON representation
auto web crawler , nodeJs
- Host: GitHub
- URL: https://github.com/jun85664396/auto-web-crawler
- Owner: jun85664396
- Created: 2014-10-05T09:05:51.000Z (over 11 years ago)
- Default Branch: master
- Last Pushed: 2015-06-17T05:35:34.000Z (about 11 years ago)
- Last Synced: 2025-04-16T16:54:01.939Z (about 1 year ago)
- Language: JavaScript
- Homepage:
- Size: 205 KB
- Stars: 4
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
auto-web-crawler
================
###0.0.5 Update
+ add sorting
##Installation
npm install awc
##Usage
var awc = require("awc");
var getIndex = function(callback){
//Write your code
//ex) var idx = [8911,8912,8913,8914,8915]; //url Suffixes.
callback(idx);
};
awc.run({
"minute":1, // If Minute Parameter is null,Minute Default Value is 60.
"url":"https://github.com/joyent/node/issues/", //Idx Default Prefix.
"idxFunc":getIndex, //Index List
"rule":{"title":"$('.gh-header-title').text().trim()"}, //Jquery Selector Rule.
"done":function(res){
console.log(res); //Return Value { '0': { title: 'events.markdown document improvements\n #8911' } ...}
//Write your code
}
});
##Test
$ node test.js
##Parameter
+ minute `int`
+ url `String`
+ idxFunc `Function` // Require parameter callback
+ rule `Hash`
#use
+ jsdom
+ fs