{"id":17789669,"url":"https://github.com/jefferyhus/es6-crawler-detect","last_synced_at":"2025-05-16T10:07:50.209Z","repository":{"id":26490038,"uuid":"109014856","full_name":"JefferyHus/es6-crawler-detect","owner":"JefferyHus","description":":spider: This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.","archived":false,"fork":false,"pushed_at":"2025-04-28T19:09:25.000Z","size":3516,"stargazers_count":92,"open_issues_count":0,"forks_count":29,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-16T10:07:45.053Z","etag":null,"topics":["bots","crawler","detection","es6-javascript","spider"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JefferyHus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":null,"patreon":"jefferyhus","open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2017-10-31T15:24:18.000Z","updated_at":"2025-04-16T00:37:51.000Z","dependencies_parsed_at":"2023-11-18T23:21:32.332Z","dependency_job_id":"dda232dd-806e-42aa-8938-f681262d31bf","html_url":"https://github.com/JefferyHus/es6-crawler-detect","commit_stats":{"total_commits":101,"total_committers":11,"mean_commits":9.181818181818182,"dds":"0.38613861386138615","last_synced_commit":"74a2fb236b845dc1188dc8c75888f901e3c45fd5"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JefferyHus%2Fes6-crawler-detect","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JefferyHus%2Fes6-crawler-detect/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JefferyHus%2Fes6-crawler-detect/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JefferyHus%2Fes6-crawler-detect/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JefferyHus","download_url":"https://codeload.github.com/JefferyHus/es6-crawler-detect/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254509476,"owners_count":22082891,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bots","crawler","detection","es6-javascript","spider"],"created_at":"2024-10-27T10:37:36.818Z","updated_at":"2025-05-16T10:07:45.199Z","avatar_url":"https://github.com/JefferyHus.png","language":"TypeScript","funding_links":["https://patreon.com/jefferyhus"],"categories":[],"sub_categories":[],"readme":"# Crawler Detect\n\n\u003cp align=\"center\"\u003e\u003ca href=\"http://crawlerdetect.io/\" target=\"_blank\"\u003e\u003cimg src=\"https://cloud.githubusercontent.com/assets/340752/23082173/1bd1a396-f550-11e6-8aba-4d3c75edea2f.png\" width=\"321\" height=\"219\" /\u003e\u003c/a\u003e\u003cbr\u003e\u003cbr\u003e\n\u003ca href=\"http://crawlerdetect.io/\" target=\"_blank\"\u003ecrawlerdetect.io\u003c/a\u003e\n\u003cbr\u003e\u003cbr\u003e\n\u003c/p\u003e\n\n[![DeepScan grade](https://deepscan.io/api/teams/16465/projects/19756/branches/518343/badge/grade.svg)](https://deepscan.io/dashboard#view=project\u0026tid=16465\u0026pid=19756\u0026bid=518343)\n![Static Badge](https://img.shields.io/badge/TypeScript-3178C6?style=flat\u0026logo=typescript\u0026logoColor=fff)\n[![npm version](https://badge.fury.io/js/es6-crawler-detect.svg?icon=si%3Anpm)](https://badge.fury.io/js/es6-crawler-detect)\n![NPM Downloads](https://img.shields.io/npm/dy/es6-crawler-detect)\n[![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/JefferyHus/es6-crawler-detect/issues)\n\n## About\n\nThis Library is an ES6 version of the original PHP class @[CrawlerDetect](https://github.com/JayBizzle/Crawler-Detect), it helps you detect bots/crawlers and spiders only by scanning the user-agent string or from the global `request.headers`.\n\n## Installation\n\n`npm install es6-crawler-detect`\n\n## Usage\n\n### ECMAScript 6 (ES6)\n\n```javascript\n'use strict';\n\nconst express = require('express')\nconst { Crawler, middleware } = require('es6-crawler-detect')\n\nconst app = express()\n\napp.get('your/route', function async (request, response) {\n  // create a new Crawler instance\n  var CrawlerDetector = new Crawler(request)\n  \n  // check the current visitor's useragent\n  if ( CrawlerDetector.isCrawler() )\n  {\n    // true if crawler user agent detected\n  }\n  \n  // or check a user agent string\n  if ( CrawlerDetector.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)') )\n  {\n    // true if crawler user agent detected\n  }\n  \n  // Output the name of the bot that matched (if any)\n  response.send(CrawlerDetector.getMatches())\n})\n\n/**\n * Or by using the middleware\n*/\napp.use(middleware((request, reponse) =\u003e {\n  // do something here\n  // e.g. console.log(request.body)\n  // e.g. return response.status(403).send('Forbidden')\n}))\n\napp.get('/crawler', function async (request, response) {\n  // or check a user agent string\n  request.Crawler.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)')\n  \n  // Output the name of the bot that matched (if any)\n  response.send(request.Crawler.getMatches())\n})\n```\n\n### TypeScript\n\n```typescript\nimport { Crawler, middleware } from 'es6-crawler-detect'\n\nconst CrawlerDetector = new Crawler()\n\n// check the current visitor's useragent\nif ( CrawlerDetector.isCrawler() )\n{\n  // true if crawler user agent detected\n}\n\n// or check a user agent string\nif ( CrawlerDetector.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)') )\n{\n  // true if crawler user agent detected\n}\n\n// Output the name of the bot that matched (if any)\nconsole.log(CrawlerDetector.getMatches())\n\n/**\n * Or by using the middleware\n*/\nmiddleware((request, reponse) =\u003e {\n  // do something here\n  // e.g. console.log(request.body)\n  // e.g. return response.status(403).send('Forbidden')\n})\n\n```\n\n## Contributing\n\nIf you find a `bot/spider/crawler` user agent that CrawlerDetect fails to detect, please submit a pull request with the regex pattern added to the `data` array in `./crawler/crawlers.ts`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjefferyhus%2Fes6-crawler-detect","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjefferyhus%2Fes6-crawler-detect","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjefferyhus%2Fes6-crawler-detect/lists"}