Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/georgea93/crawley

crawler depth es6 javascript node nodejs nodejs-web-crawler npm npm-module npm-package robots-txt sitemap web yarn

Last synced: about 11 hours ago
JSON representation

nodejs web crawler

Host: GitHub
URL: https://github.com/georgea93/crawley
Owner: GeorgeA93
Created: 2017-01-19T12:09:31.000Z (almost 8 years ago)
Default Branch: master
Last Pushed: 2017-02-24T10:03:26.000Z (over 7 years ago)
Last Synced: 2024-11-16T09:14:46.741Z (4 days ago)
Topics: crawler, depth, es6, javascript, node, nodejs, nodejs-web-crawler, npm, npm-module, npm-package, robots-txt, sitemap, web, yarn
Language: JavaScript
Homepage:
Size: 70.3 KB
Stars: 3
Watchers: 2
Forks: 0
Open Issues: 1