https://github.com/you21979/node-crawler.io

https://github.com/you21979/node-crawler.io

Last synced: 4 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/you21979/node-crawler.io
• Owner: you21979
• License: mit
• Created: 2014-10-08T08:06:02.000Z (about 10 years ago)
• Default Branch: master
• Last Pushed: 2015-08-30T17:18:49.000Z (about 9 years ago)
• Last Synced: 2024-08-10T23:16:30.996Z (3 months ago)
• Language: JavaScript
• Size: 184 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# node-crawler.io

web crawler i/o

## install

```

npm install crawler.io

```

## example

### Simple Usage

```

var CookieSession = require('crawler.io').CookieSession;

var c = new CookieSession("UserAgent : (I am a Bot.)");

c.get('http://www.yahoo.co.jp/').then(function(res){

    console.log(res)

}).catch(function(err){

    console.log("err")

})

```

### Login Session

```

var CookieSession = require('crawler.io').CookieSession;

var c = new CookieSession("UserAgent : (I am a Bot.)");

var loginParam = {

    name:process.env['ID'],

    password:process.env['PASS'],

};

c.get('https://www.hatena.ne.jp/').then(function(res){

    return c.post('https://www.hatena.ne.jp/login', loginParam).then(function(res){

        if( res.match(/が違います/) ){

            throw new Error('login error');

        }

        return c.get('http://b.hatena.ne.jp/my')

    }).then(function(res){

        console.log(res)

    })

}).catch(function(err){

    console.log("err")

})

```

command

```

ID=username PASS=password node login.js

```

## license

MIT