Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ferventdesert/Hawk

visualized crawler & ETL IDE written with C#/WPF
https://github.com/ferventdesert/Hawk

Last synced: 13 days ago
JSON representation

visualized crawler & ETL IDE written with C#/WPF

Awesome Lists containing this project

README

        

Hawk: Advanced ETL & Crawler Stream Engine

***

Welcome to use Hawk! Hawk can graphically crawle webpage, clean, process and save data without programming, written in C#/WPF, open source according to the GPL protocol.

- Github : https://github.com/ferventdesert/Hawk
- Documentation: https://ferventdesert.github.io/Hawk/
- Project Respository: https://github.com/ferventdesert/Hawk-Projects/Hawk3
- Download: https://github.com/ferventdesert/Hawk/releases
- Download(chs): https://gitee.com/deserthawk/Hawk/attach_files
## Introduction

The meaning of Hawk is "Eagle", which can kill prey efficiently and accurately. Its design idea comes from Lisp language , and its function mimics awk in Unix.

The key features are as follows:

- Intelligent analysis of web content without programming.
- WYSIWYG, conversion, filtering & storage with visually drag and drop.
- Parallel processing & high speed
- Support mutli-file & database: xml, csv, sqlite,mongodb...
- Tasks can be save, pause, restart & reused.
- Focus on crawler but power is far beyond that.

![splash](https://upload-images.jianshu.io/upload_images/9608527-2185c11706ccf618.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

Fast & Smart webpage crawling :

![1.gif-1001.8kB](https://raw.githubusercontent.com/ferventdesert/Hawk/master/Docs/imgs/1.gif-1001.8kB.gif)

WYSIWYG ETL:

![2.gif](https://raw.githubusercontent.com/ferventdesert/Hawk/master/Docs/imgs/2.gif)

****

欢迎使用Hawk! HAWK无需编程,可见即所得的图形化数据采集和清洗工具,依据GPL协议开源。

- 项目主页:https://github.com/ferventdesert/Hawk
- 项目文档: https://ferventdesert.github.io/Hawk/
- 示例工程文件: https://github.com/ferventdesert/Hawk-Projects/Hawk3
- 下载地址: https://github.com/ferventdesert/Hawk/releases
- 国内下载地址: https://gitee.com/deserthawk/Hawk/attach_files

## 介绍
Hawk的含义为“鹰”,能够高效,准确地捕杀猎物。它的思想来源于Lisp语言,功能模仿了Linux工具awk。

特点如下:

- 智能分析网页内容,无需编程
- 所见即所得,可视化拖拽,快地实现转换和过滤等数据清洗操作
- 能从各类数据库和文件实现导入导出
- 任务可以被保存和复用
- 其最适合的领域是爬虫和数据清洗,但其威力远超于此。