{"id":19241648,"url":"https://github.com/qwqcode/nacollector","last_synced_at":"2025-09-16T08:30:54.839Z","repository":{"id":38009302,"uuid":"98152060","full_name":"qwqcode/Nacollector","owner":"qwqcode","description":"Nacollector WEB data collection platform.","archived":false,"fork":false,"pushed_at":"2022-12-08T03:44:24.000Z","size":6720,"stargazers_count":194,"open_issues_count":2,"forks_count":67,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-01-02T16:13:45.854Z","etag":null,"topics":["cef","cefsharp","collector","download-manager","downloader","gome","image-downloader","js-terminal","material-design","suning","taobao","tmall","webapp"],"latest_commit_sha":null,"homepage":"https://qwqaq.com/65c99d88.html","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/qwqcode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-24T05:27:06.000Z","updated_at":"2024-07-24T09:37:12.000Z","dependencies_parsed_at":"2023-01-24T10:15:30.032Z","dependency_job_id":null,"html_url":"https://github.com/qwqcode/Nacollector","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qwqcode%2FNacollector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qwqcode%2FNacollector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qwqcode%2FNacollector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qwqcode%2FNacollector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/qwqcode","download_url":"https://codeload.github.com/qwqcode/Nacollector/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233231609,"owners_count":18645069,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cef","cefsharp","collector","download-manager","downloader","gome","image-downloader","js-terminal","material-design","suning","taobao","tmall","webapp"],"created_at":"2024-11-09T17:12:06.254Z","updated_at":"2025-09-16T08:30:48.744Z","avatar_url":"https://github.com/qwqcode.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"./docs/NacollectorLogo_2.png\"\u003e\u003c/p\u003e\n\n# Nacollector\n\n[![](https://img.shields.io/github/release/qwqcode/Nacollector.svg?style=flat-square)](https://github.com/qwqcode/Nacollector/releases/latest) ![](https://img.shields.io/badge/NET-%3E%3D%204.6.2-green.svg?style=flat-square) [![](https://img.shields.io/github/downloads/qwqcode/Nacollector/total.svg?style=flat-square)](https://github.com/qwqcode/Nacollector/releases) [![](https://img.shields.io/github/last-commit/qwqcode/Nacollector.svg?style=flat-square)](https://github.com/qwqcode/Nacollector/commits) [![](https://img.shields.io/github/issues/qwqcode/Nacollector.svg?style=flat-square)](https://github.com/qwqcode/Nacollector/issues) [![](https://img.shields.io/github/issues-pr/qwqcode/Nacollector.svg?style=flat-square)](https://github.com/qwqcode/Nacollector/pulls) [![](https://img.shields.io/gitter/room/qwqcode/Nacollector.svg?style=flat-square)](https://gitter.im/Nacollector/community) [![](https://img.shields.io/badge/%24-donate-ff69b4.svg?style=flat-square)](https://github.com/qwqcode/donate-qwqaq)\n\n**Nacollector：一款高效便捷的采集器聚合客户端**\n\n旨在采集各类网络资源，为数据处理提供丰富支持，使繁杂的采集任务得以轻松且高效地完成。\n\nNacollector 提供了一套从用户界面到采集器功能的开发一体化解决方案，助你快速开发和定制专属的数据采集工作流程。\n\n## 特性\n\n- **友好的用户界面**：通过 Web 技术构建用户界面，基于 [CefSharp](https://github.com/cefsharp/CefSharp) 嵌入 Chromium 浏览器，实现浏览器的集成和定制，满足采集器的开发需求。\n- **丰富的 C# 生态系统**：无缝接入 C# 开发者生态系统，支持 .NET 及系统 API 调用，为采集器开发提供丰富支持。\n- **终端窗口及日志管理**：提供日志 API，实时跟踪和管理采集任务的运行状态与异常。\n- **多任务并行处理能力**：支持多任务管理，可同时运行多个采集任务异步执行，提高工作效率。\n- **下载资源管理器**：内置下载管理，便于保存和整理采集的资源。\n- **完善的 Cookie 管理**：可以手动导入、自动填充、保存 Cookie，支持正则表达式匹配规则，便于处理需要登录的网站。\n- **数据本地离线储存**：隐私至上，采集数据本地管理、离线保存，安全可靠。\n- **多类型资源预览**：快速预览图片、视频、音频、文本等资源，便于检查采集结果。\n- **任务配置界面（启动台）**：提供界面配置采集任务参数，便于启动和管理任务。\n- **灵活网络连接管理**：支持设置代理，适应不同网络环境。\n- **升级机制及版本管理**：支持在线和离线更新软件版本，为软件快速升级迭代提供支持。\n- **可扩展设计**：采用模块化设计，可通过插件添加新功能。\n- **清晰的界面布局**：使用 Material Design 设计风格，界面简单易用。\n- **电商数据采集**：针对电商平台采集的商品信息优化，例如获取商品详情、价格、库存、评价等数据。\n\n## 内置采集器\n\n采集器代码存储于 `NacollectorSpiders` 目录下，目前支持的采集器有：\n\n**电商采集功能**\n\n- 商品详情页图片解析\n  - 支持网站： 淘宝、天猫、苏宁、国美\n  - 支持图片类型：主图、分类图、详情图\n  - 支持即时预览 显示 URL\n  - 支持下载单张图片 右键另存为\n  - 支持下载所有图片 打包为压缩文件并保存\n- 商品详情页视频抓取\n- 淘宝店铺搜索卖家ID名采集\n  - 支持忽略天猫店铺\n- 天猫供销平台分销商一键邀请\n  - 支持卖家账号登录 得到 Cookie（也可以手动输入 Cookie 字符串）\n- 天猫供销平台分销商一键撤回\n\n**资讯采集器**\n\n- 新闻资讯采集\n  - 支持网站：新浪新闻、腾讯新闻、网易新闻、搜狐新闻、百度新闻、知乎文章、推酷、简书、博客园文章\n\n**图片采集器**\n\n- 通用图片采集工具\n  - 提供 Chrome 插件采集图片（仿花瓣、堆糖）\n- 新浪微博图片采集\n- Pinterest 图片采集\n\n**小说采集器**\n\n- 小说采集\n  - 支持网站：起点中文网、晋江文学城、纵横中文网\n\n**视频采集器**\n\n- 优酷视频采集\n- YouTube 视频采集\n\n\u003e 注：采集器仅供学习交流使用，请合理使用并尊重版权。\n\n## 技术特点\n\n本项目基于 **C# + CefSharp**，通过嵌入 Chromium 实现了浏览器的集成和定制开发。与 Electron.js 相比，Nacollector 在以下方面具有一些优势：\n\n- **.NET 集成**：拥有丰富的 C# 开发者生态系统，与现有 C# 项目自然配合。\n- **轻量快速**：Nacollector 启动速度快，内存占用低，同时无需依赖 Node.js 环境。\n- **系统调用**：轻松调用操作系统 API，访问系统资源，方便开发者进行功能扩展。\n- **浏览器定制**：可深度定制 Chromium 功能，灵活满足多样化的业务需求。\n\n## 软件截图\n\n| 主界面 | 终端界面 |\n|-|-|\n| \u003cimg width=\"350\" src=\"./docs/screenshots/home.png\"\u003e | \u003cimg width=\"350\" src=\"./docs/screenshots/terminal.png\"\u003e |\n\n| 终端日志 1 | 终端日志 2 |\n|-|-|\n| \u003cimg width=\"350\" src=\"./docs/screenshots/terminal1.png\"\u003e | \u003cimg width=\"350\" src=\"./docs/screenshots/terminal2.png\"\u003e |\n\n| 终端日志 3 | 终端日志 4 |\n|-|-|\n| \u003cimg width=\"350\" src=\"./docs/screenshots/terminal3.png\"\u003e | \u003cimg width=\"350\" src=\"./docs/screenshots/terminal4.png\"\u003e |\n\n| 终端日志 5 | 终端日志 6 |\n|-|-|\n| \u003cimg width=\"350\" src=\"./docs/screenshots/terminal5.png\"\u003e | \u003cimg width=\"350\" src=\"./docs/screenshots/terminal6.png\"\u003e |\n\n| 操作演示 | Cookie 获取器 |\n|-|-|\n| \u003cimg width=\"350\" src=\"./docs/screenshots/action.gif\"\u003e | \u003cimg width=\"350\" src=\"./docs/screenshots/cookie_getter.png\"\u003e |\n\n| 多任务管理 | 下载管理 1 |\n|-|-|\n| \u003cimg width=\"350\" src=\"./docs/screenshots/tasks.png\"\u003e | \u003cimg width=\"350\" src=\"./docs/screenshots/downloading.png\"\u003e |\n\n| 下载管理 2 | 设置界面 |\n|-|-|\n| \u003cimg width=\"350\" src=\"./docs/screenshots/downloading1.png\"\u003e | \u003cimg width=\"350\" src=\"./docs/screenshots/settings.png\"\u003e |\n\n| 安装程序 1 | 安装程序 2 |\n|-|-|\n| \u003cimg width=\"350\" src=\"./docs/screenshots/installer.png\"\u003e | \u003cimg width=\"350\" src=\"./docs/screenshots/installer-2.png\"\u003e |\n\n| 软件更新视频演示 | 软件更新界面 |\n|-|-|\n| \u003cvideo width=\"350\" src=\"https://github.com/user-attachments/assets/a683e83e-fcab-4d78-bb1d-5bba8d057d46\" controls autoplay loop\u003e\u003c/video\u003e | \u003cimg width=\"350\" src=\"./docs/screenshots/updater.png\"\u003e |\n\n| 程序安装演示视频 |\n|-|\n| \u003cvideo width=\"600\" src=\"https://github.com/user-attachments/assets/5eb3f08e-9cef-4100-9a5f-0aadf26d186e\" controls autoplay loop\u003e\u003c/video\u003e |\n\n## Stargazers over time\n\n[![Stargazers over time](https://starchart.cc/qwqcode/Nacollector.svg)](https://starchart.cc/qwqcode/Nacollector)\n\n## Development\n\n### Dependencies\n- NET \u003e= 4.6.2\n- [CefSharp](https://github.com/cefsharp/CefSharp)\n- [CsQuery](https://github.com/jamietre/CsQuery)\n- Selenium.WebDriver\n\n### Get the sources\n\nIn order to make development easier, the frontend is included in the backend as a [git submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules).\n\nThen, in order to tinker with the sources, start by getting both repos at once with:\n\n``` bash\ngit clone --recurse-submodules https://github.com/qwqcode/Nacollector.git\n```\n\n\u003e NOTE: since it is a submodule, when developing the frontend remember to update the backend repo accordingly.\n\n### Quick Start\n\n```bash\n# 1. clone\ngit clone --recurse-submodules https://github.com/qwqcode/Nacollector.git\ncd Nacollector\n\n# 2. copy config files\ncp ./Nacollector/GlobalConstant.cs.example ./Nacollector/GlobalConstant.cs\n\n# 3. download `https://github.com/qwqcode/Nacollector/releases/download/1.3.0.0/CefSharp_v69.7z` to `./CefSharp/` and unpack\n\n# 4. open .sln by vs\nstart Nacollector.sln\n```\n\n## 版权声明\n\n仅限个人学习使用，禁止商业用途。请阅读 [《Nacollector 用户使用许可协议》](./LICENSE)。\n\n[Nacollector](https://github.com/qwqcode/Nacollector) Copyright (C) 2018 [qwqaq.com](https://qwqaq.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqwqcode%2Fnacollector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqwqcode%2Fnacollector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqwqcode%2Fnacollector/lists"}