https://github.com/antchfx/antch

Antch, a fast, powerful and extensible web crawling & scraping framework for Go
https://github.com/antchfx/antch

crawler crawling framework golang scraping web-crawler web-spider

Last synced: about 2 months ago
JSON representation

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

• Host: GitHub
• URL: https://github.com/antchfx/antch
• Owner: antchfx
• License: mit
• Created: 2017-09-28T05:44:17.000Z (about 7 years ago)
• Default Branch: master
• Last Pushed: 2020-05-31T15:12:21.000Z (over 4 years ago)
• Last Synced: 2024-07-31T20:52:23.138Z (4 months ago)
• Topics: crawler, crawling, framework, golang, scraping, web-crawler, web-spider
• Language: Go
• Homepage:
• Size: 56.6 KB
• Stars: 258
• Watchers: 16
• Forks: 41
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-go - antch - A fast, powerful and extensible web crawling & scraping framework. (Miscellaneous / Uncategorized)
• zero-alloc-awesome-go - antch - A fast, powerful and extensible web crawling & scraping framework. (Miscellaneous / Uncategorized)
• go-awesome - Antch
• awesome-go - antch - Antch, a fast, powerful and extensible web crawling & scraping framework for Go - ★ 110 (Miscellaneous)
• awesome-go-extra - antch - 09-28T05:44:17Z|2020-05-31T15:12:21Z| (Microsoft Office / Uncategorized)
• awesome-go-zh - antch

README

        
Antch

====

[![Build Status](https://travis-ci.org/antchfx/antch.svg?branch=master)](https://travis-ci.org/antchfx/antch)

[![Coverage Status](https://coveralls.io/repos/github/antchfx/antch/badge.svg?branch=master)](https://coveralls.io/github/antchfx/antch?branch=master)

[![Go Report Card](https://goreportcard.com/badge/github.com/antchfx/antch)](https://goreportcard.com/report/github.com/antchfx/antch)

[![GoDoc](https://godoc.org/github.com/antchfx/antch?status.svg)](https://godoc.org/github.com/antchfx/antch)

**Antch**, inspired by [Scrapy](https://scrapy.org/). If you're familiar with scrapy, 

you can quickly get started.

Antch is a fast, powerful and extensible web crawling & scraping framework for Go, used 

to crawl websites and extract structured data from their pages.

Get Started

====

[Getting Started](https://github.com/antchfx/antch-getstarted)

Follow the Getting Started instructions to start your first spider.

- [English](https://github.com/antchfx/antch/wiki/Getting-Started)

- [简体中文](https://github.com/antchfx/antch/wiki/%E5%BF%AB%E9%80%9F%E5%85%A5%E9%97%A8)

Features

====

- Polite, highly concurrent web crawler.

- Powerful and customizable HTTP middleware.

- Item data pipeline for the web spider.

- Built-in proxy support (HTTP, HTTPS, SOCKS5).

- Built-in XPath query support for HTML/XML documents.

- Easy to use and integrate with your project.

Examples

===

[BingWallpaper](https://github.com/zhengchun/bingwallpaper) - Bing daily wallpaper.

Documentation

====

See [https://github.com/antchfx/antch/wiki](https://github.com/antchfx/antch/wiki)