https://github.com/pmuens/crawler

Multi-threaded Web crawler with support for custom fetching and persisting logic
https://github.com/pmuens/crawler

crawler crawler-engine rust rust-lang web-crawler web-crawling

Last synced: 20 days ago
JSON representation

Multi-threaded Web crawler with support for custom fetching and persisting logic

• Host: GitHub
• URL: https://github.com/pmuens/crawler
• Owner: pmuens
• Created: 2020-10-18T09:31:56.000Z (about 4 years ago)
• Default Branch: master
• Last Pushed: 2020-10-18T09:32:16.000Z (about 4 years ago)
• Last Synced: 2023-03-24T08:11:43.709Z (over 1 year ago)
• Topics: crawler, crawler-engine, rust, rust-lang, web-crawler, web-crawling
• Language: Rust
• Homepage:
• Size: 53.7 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0

Awesome Lists containing this project