Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/antoinegagne/treewalker
A web crawler in Erlang that respects `robots.txt`.
https://github.com/antoinegagne/treewalker
crawler erlang webcrawler
Last synced: 13 days ago
JSON representation
A web crawler in Erlang that respects `robots.txt`.
- Host: GitHub
- URL: https://github.com/antoinegagne/treewalker
- Owner: AntoineGagne
- License: other
- Created: 2019-12-30T00:25:40.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2023-02-27T18:39:55.000Z (over 1 year ago)
- Last Synced: 2023-03-02T19:51:10.353Z (over 1 year ago)
- Topics: crawler, erlang, webcrawler
- Language: Erlang
- Homepage:
- Size: 46.9 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 6