https://github.com/roccomuso/is-baidu

Verify that a request is from Baidu crawlers using DNS verification
https://github.com/roccomuso/is-baidu

baidu crawler dns ip js nodejs verification

Last synced: about 1 month ago
JSON representation

Verify that a request is from Baidu crawlers using DNS verification

• Host: GitHub
• URL: https://github.com/roccomuso/is-baidu
• Owner: roccomuso
• License: mit
• Created: 2018-05-01T19:35:00.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2019-01-05T17:18:13.000Z (almost 6 years ago)
• Last Synced: 2024-11-09T17:47:24.739Z (about 2 months ago)
• Topics: baidu, crawler, dns, ip, js, nodejs, verification
• Language: JavaScript
• Homepage:
• Size: 3.91 KB
• Stars: 3
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# is-baidu

[![NPM Version](https://img.shields.io/npm/v/is-baidu.svg)](https://www.npmjs.com/package/is-baidu)

[![Build Status](https://travis-ci.org/roccomuso/is-baidu.svg?branch=master)](https://travis-ci.org/roccomuso/is-baidu)

![node](https://img.shields.io/node/v/is-baidu.svg)

[![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com)

> Verify that a request is from Baidu crawlers using Baidu's DNS verification steps

You may wish to verify that a web crawler accessing your server is Baidu (or another Baidu user-agent) and not spammers or other bots scraping your site while claiming to be the Baidu crawler. Since you cannot rely on the `User-Agent` header which is easily spoofed, you need to use DNS look up to verify that the IP address belongs to Baidu.

## Install

`npm install --save is-baidu`

## Example

```javascript

const isBaidu = require('is-baidu')

let ip = '123.125.71.87'

isBaidu(ip).then((outcome) => {

  if (outcome) {

    // it's baidu.

  }

}).catch(console.error)

```

### Example with express

```javascript

app.enable('trust proxy')

app.use((req, res, next) => {

  let ip = req.ip || req.connection.remoteAddress

  isBaidu(ip).then(outcome => {

    if (outcome) {

      res.status(404).text('Nothing to scan') // block baidu crawler

    } else {

      next() // it's a user

    }

  })

})

```

## Tests

`npm test`

## License

MIT

## Author

Rocco Musolino [@roccomuso](https://twitter.com/roccomuso)