https://github.com/schollz/linkcrawler

Cross-platform persistent and distributed web crawler :link:
https://github.com/schollz/linkcrawler

crawler hyperlinks web

Last synced: about 2 months ago
JSON representation

Cross-platform persistent and distributed web crawler :link:

• Host: GitHub
• URL: https://github.com/schollz/linkcrawler
• Owner: schollz
• License: mit
• Created: 2017-03-08T17:57:34.000Z (almost 8 years ago)
• Default Branch: master
• Last Pushed: 2017-09-09T14:56:30.000Z (over 7 years ago)
• Last Synced: 2024-11-01T16:40:21.259Z (about 2 months ago)
• Topics: crawler, hyperlinks, web
• Language: Go
• Homepage:
• Size: 59.6 KB
• Stars: 111
• Watchers: 9
• Forks: 9
• Open Issues: 0

Awesome Lists containing this project