https://github.com/shavit/crawlero
Distributed web crawlers. Fault tolerance, user-agent randomizer, RabbitMQ, Tor, PostgreSQL.
https://github.com/shavit/crawlero
crawler marketing-automation marketing-tools pbn proxy rabbitmq tor
Last synced: 6 months ago
JSON representation
Distributed web crawlers. Fault tolerance, user-agent randomizer, RabbitMQ, Tor, PostgreSQL.
- Host: GitHub
- URL: https://github.com/shavit/crawlero
- Owner: shavit
- Created: 2017-11-30T13:44:13.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-01-17T22:30:36.000Z (over 7 years ago)
- Last Synced: 2023-03-06T08:31:23.973Z (over 2 years ago)
- Topics: crawler, marketing-automation, marketing-tools, pbn, proxy, rabbitmq, tor
- Language: Go
- Homepage:
- Size: 7.81 KB
- Stars: 14
- Watchers: 4
- Forks: 0
- Open Issues: 0