https://github.com/montferret/ferret-server

Advanced declarative web scraping
https://github.com/montferret/ferret-server

crawling data-mining ferret-server golang query-language scraper scraping scraping-api server

Last synced: 1 day ago
JSON representation

Advanced declarative web scraping

• Host: GitHub
• URL: https://github.com/montferret/ferret-server
• Owner: MontFerret
• License: apache-2.0
• Created: 2018-09-26T16:49:37.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2023-02-25T00:30:40.000Z (over 1 year ago)
• Last Synced: 2024-11-04T15:59:02.857Z (11 days ago)
• Topics: crawling, data-mining, ferret-server, golang, query-language, scraper, scraping, scraping-api, server
• Language: Go
• Homepage:
• Size: 359 KB
• Stars: 30
• Watchers: 8
• Forks: 6
• Open Issues: 15
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Ferret Server



	

		

	

	

		

	

	

		

	

	

		

	

	

		

	

	

		

	



Server for advanced web scraping.    

[Open API defintion.](https://next.stoplight.io/montferret/ferret-server/)

# Features

- Scripts persistence

- Scraped data persistence

- Script execution scheduling

- Integration with 3rd party systems

- Web Hooks

- Security

# WIP

Be aware, that the project is under heavy development.    

There is no documentation and some things may change in the final release.

# Installation

### Binary

You can download latest binaries from [here](https://github.com/MontFerret/ferret-server/releases).

### Source code

#### Production

* Go >=1.11

* Chrome or Docker

* ArangoDB

#### Development

* [Go Swagger](https://www.github.com/go-swagger/go-swagger)

* GNU Make

* ANTLR4 >=4.7.1

## Quick start

```sh

ferret-server --db=http://0.0.0.0:8529

```