https://github.com/ferventdesert/Hawk

visualized crawler & ETL IDE written with C#/WPF
https://github.com/ferventdesert/Hawk

Last synced: 9 months ago
JSON representation

visualized crawler & ETL IDE written with C#/WPF

• Host: GitHub
• URL: https://github.com/ferventdesert/Hawk
• Owner: ferventdesert
• License: apache-2.0
• Created: 2016-04-02T07:54:41.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2019-12-21T10:26:40.000Z (almost 6 years ago)
• Last Synced: 2024-10-29T17:49:42.244Z (about 1 year ago)
• Language: C#
• Size: 39.1 MB
• Stars: 3,159
• Watchers: 287
• Forks: 974
• Open Issues: 67
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-crawler - Hawk - Advanced Crawler and ETL tool written in C#/WPF. (C#)
• awesome-crawler-cn - Hawk -  用 C#/WPF开发的网络爬虫，具有简单的ETL功能. (C#)

README

          

Hawk: Advanced ETL & Crawler Stream Engine 

***

Welcome to use Hawk! Hawk can graphically crawle webpage, clean, process and save data without programming， written in C#/WPF, open source according to the GPL protocol.

- Github : https://github.com/ferventdesert/Hawk

- Documentation: https://ferventdesert.github.io/Hawk/

- Project Respository: https://github.com/ferventdesert/Hawk-Projects/Hawk3

- Download:  https://github.com/ferventdesert/Hawk/releases

- Download(chs): https://gitee.com/deserthawk/Hawk/attach_files

## Introduction

The meaning of Hawk is "Eagle", which can kill prey efficiently and accurately. Its design idea comes from Lisp language , and its function mimics awk in Unix.

The key features are as follows:

- Intelligent analysis of web content without programming.

- WYSIWYG, conversion, filtering & storage with visually drag and drop.

- Parallel processing & high speed

- Support mutli-file & database: xml, csv, sqlite,mongodb...

- Tasks can be save, pause, restart & reused.

- Focus on crawler but power is far beyond that.

![splash](https://upload-images.jianshu.io/upload_images/9608527-2185c11706ccf618.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

Fast & Smart webpage crawling :

![1.gif-1001.8kB](https://raw.githubusercontent.com/ferventdesert/Hawk/master/Docs/imgs/1.gif-1001.8kB.gif)

WYSIWYG ETL:

![2.gif](https://raw.githubusercontent.com/ferventdesert/Hawk/master/Docs/imgs/2.gif)

****

欢迎使用Hawk! HAWK无需编程，可见即所得的图形化数据采集和清洗工具，依据GPL协议开源。

- 项目主页：https://github.com/ferventdesert/Hawk

- 项目文档： https://ferventdesert.github.io/Hawk/

- 示例工程文件： https://github.com/ferventdesert/Hawk-Projects/Hawk3

- 下载地址： https://github.com/ferventdesert/Hawk/releases

- 国内下载地址:  https://gitee.com/deserthawk/Hawk/attach_files

## 介绍

Hawk的含义为“鹰”，能够高效，准确地捕杀猎物。它的思想来源于Lisp语言，功能模仿了Linux工具awk。

特点如下：

- 智能分析网页内容，无需编程

- 所见即所得，可视化拖拽，快地实现转换和过滤等数据清洗操作

- 能从各类数据库和文件实现导入导出

- 任务可以被保存和复用

- 其最适合的领域是爬虫和数据清洗，但其威力远超于此。