Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jefferyhus/es6-crawler-detect
:spider: This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.
https://github.com/jefferyhus/es6-crawler-detect
bots crawler detection es6-javascript spider
Last synced: 4 days ago
JSON representation
:spider: This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.
- Host: GitHub
- URL: https://github.com/jefferyhus/es6-crawler-detect
- Owner: JefferyHus
- License: mit
- Created: 2017-10-31T15:24:18.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2024-12-06T07:05:27.000Z (20 days ago)
- Last Synced: 2024-12-15T18:07:19.033Z (11 days ago)
- Topics: bots, crawler, detection, es6-javascript, spider
- Language: TypeScript
- Homepage:
- Size: 3.29 MB
- Stars: 90
- Watchers: 4
- Forks: 29
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# Crawler Detect
[](https://deepscan.io/dashboard#view=project&tid=16465&pid=19756&bid=518343)
[](https://badge.fury.io/js/es6-crawler-detect)
[](https://github.com/JefferyHus/es6-crawler-detect/issues)
## About
This Library is an ES6 version of the original PHP class @[CrawlerDetect](https://github.com/JayBizzle/Crawler-Detect), it helps you detect bots/crawlers and spiders only by scanning the user-agent string or from the global `request.headers`.
## Installation
`npm install es6-crawler-detect`
## Usage
### ECMAScript 6 (ES6)
```javascript
'use strict';
const express = require('express')
const { Crawler, middleware } = require('es6-crawler-detect')
const app = express()
app.get('your/route', function async (request, response) {
// create a new Crawler instance
var CrawlerDetector = new Crawler(request)
// check the current visitor's useragent
if ( CrawlerDetector.isCrawler() )
{
// true if crawler user agent detected
}
// or check a user agent string
if ( CrawlerDetector.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)') )
{
// true if crawler user agent detected
}
// Output the name of the bot that matched (if any)
response.send(CrawlerDetector.getMatches())
})
/**
* Or by using the middleware
*/
app.use(middleware((request, reponse) => {
// do something here
// e.g. console.log(request.body)
// e.g. return response.status(403).send('Forbidden')
}))
app.get('/crawler', function async (request, response) {
// or check a user agent string
request.Crawler.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)')
// Output the name of the bot that matched (if any)
response.send(request.Crawler.getMatches())
})
```
### TypeScript
```typescript
import { Crawler, middleware } from 'es6-crawler-detect'
const CrawlerDetector = new Crawler()
// check the current visitor's useragent
if ( CrawlerDetector.isCrawler() )
{
// true if crawler user agent detected
}
// or check a user agent string
if ( CrawlerDetector.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)') )
{
// true if crawler user agent detected
}
// Output the name of the bot that matched (if any)
console.log(CrawlerDetector.getMatches())
/**
* Or by using the middleware
*/
middleware((request, reponse) => {
// do something here
// e.g. console.log(request.body)
// e.g. return response.status(403).send('Forbidden')
})
```
## Contributing
If you find a `bot/spider/crawler` user agent that CrawlerDetect fails to detect, please submit a pull request with the regex pattern added to the `data` array in `./crawler/crawlers.ts`.
