{"id":17140084,"url":"https://github.com/mrxujiang/crawel","last_synced_at":"2025-04-13T10:21:33.878Z","repository":{"id":99154177,"uuid":"263803505","full_name":"MrXujiang/crawel","owner":"MrXujiang","description":"基于Apify+node+react搭建的有点意思的爬虫平台","archived":false,"fork":false,"pushed_at":"2020-05-14T03:42:22.000Z","size":9529,"stargazers_count":41,"open_issues_count":0,"forks_count":16,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-12T19:23:27.540Z","etag":null,"topics":["apify","crawler","node","puppeteer","react","react-hooks","umi","umi3"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MrXujiang.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-05-14T03:17:00.000Z","updated_at":"2024-09-02T06:32:24.000Z","dependencies_parsed_at":"2023-06-04T05:15:33.813Z","dependency_job_id":null,"html_url":"https://github.com/MrXujiang/crawel","commit_stats":{"total_commits":2,"total_committers":2,"mean_commits":1.0,"dds":0.5,"last_synced_commit":"c288eebf741d239e11c4780b2d7239d58b32a79f"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrXujiang%2Fcrawel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrXujiang%2Fcrawel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrXujiang%2Fcrawel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrXujiang%2Fcrawel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MrXujiang","download_url":"https://codeload.github.com/MrXujiang/crawel/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248696232,"owners_count":21147093,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apify","crawler","node","puppeteer","react","react-hooks","umi","umi3"],"created_at":"2024-10-14T20:13:52.044Z","updated_at":"2025-04-13T10:21:33.855Z","avatar_url":"https://github.com/MrXujiang.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"## 基于Apify+node+react搭建的有点意思的爬虫平台\n\u003e 该平台可以用来爬取任意网页的数据, 网页全屏截图(网页快照), 并对爬取到的数据进行管理, 操作等.开发者也可以自行进行扩展, 作为爬虫管理平台的基础服务.\n\n## 技术实现\n* 前端采用umi集成开发环境, 框架选用react+antd;\n* 后端采用nodejs + koa, 爬虫框架采用Apify;\n* node多进程开发模式\n\n## 项目截图\n\u003cimg src=\"./1.png\" width=800 /\u003e\n\u003cimg src=\"./2.png\" width=800 /\u003e\n\n## 开源交流与反馈\n\u003cimg src=\"./wx.png\" /\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrxujiang%2Fcrawel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmrxujiang%2Fcrawel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrxujiang%2Fcrawel/lists"}