Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/matteoredaelli/ebot
Ebot, an Opensource Web Crawler built on top of a nosql database (apache couchdb, riak), AMQP database (rabbitmq), webmachine and mochiweb. Ebot is written in Erlang and it is a very scalable, distribuited and highly configurable web cawler. See wiki pages for more details
https://github.com/matteoredaelli/ebot
Last synced: 3 months ago
JSON representation
Ebot, an Opensource Web Crawler built on top of a nosql database (apache couchdb, riak), AMQP database (rabbitmq), webmachine and mochiweb. Ebot is written in Erlang and it is a very scalable, distribuited and highly configurable web cawler. See wiki pages for more details
- Host: GitHub
- URL: https://github.com/matteoredaelli/ebot
- Owner: matteoredaelli
- License: gpl-3.0
- Created: 2010-04-04T07:11:47.000Z (about 14 years ago)
- Default Branch: master
- Last Pushed: 2011-04-08T19:54:55.000Z (about 13 years ago)
- Last Synced: 2024-01-18T19:24:33.380Z (5 months ago)
- Language: Erlang
- Homepage: http://www.redaelli.org/matteo-blog/projects/ebot
- Size: 5.54 MB
- Stars: 326
- Watchers: 28
- Forks: 54
- Open Issues: 10
-
Metadata Files:
- Readme: README
- License: LICENSE.txt
Lists
- awesome-crawler - ebot - A scalable, distribuited and highly configurable web cawler. (Erlang)
- awesome-crawler-cn - ebot - 一个分布式，高可扩展的网络爬虫. (Erlang)
- awesome-web-scraper - ebot - Opensource Web Crawler built on top of a nosql database (apache couchdb, riak), AMQP database (rabbitmq), webmachine and mochiweb. (Erlang)
- awesome-crawler - ebot - A scalable, distribuited and highly configurable web cawler. (Erlang)
- awesome-crawlers - ebot - 07-05 | A scalable, distribuited and highly configurable web cawler. | (Erlang)
README
EBOT (http://www.redaelli.org/matteo-blog/projects/ebot/)
USAGE: see the wiki homepage (http://wiki.github.com/matteoredaelli/ebot/) and ebot_test.erl for details
LICENSE: GPL v3+, see file LICENSE
Copyright (c) 2010, 2011 - Matteo Redaelli at libero dot it
Tested with Linux (Ubuntu and Debian) and Erlang R13B03 and R14A
*******************************************************************************
* DOWNLOAD "TAG" releases and not the latest sources that could be unstable...
*******************************************************************************