{"id":21085304,"url":"https://github.com/nekolr/slime","last_synced_at":"2025-05-16T11:31:11.205Z","repository":{"id":39761746,"uuid":"365988704","full_name":"nekolr/slime","owner":"nekolr","description":"🍰 A visual crawler management platform","archived":true,"fork":false,"pushed_at":"2023-09-04T06:07:52.000Z","size":2603,"stargazers_count":69,"open_issues_count":4,"forks_count":29,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-25T04:54:02.520Z","etag":null,"topics":["crawler","spider","visual-crawler","websocket"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nekolr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-10T09:31:34.000Z","updated_at":"2024-08-23T12:24:17.000Z","dependencies_parsed_at":"2024-11-19T20:44:59.237Z","dependency_job_id":null,"html_url":"https://github.com/nekolr/slime","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nekolr%2Fslime","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nekolr%2Fslime/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nekolr%2Fslime/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nekolr%2Fslime/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nekolr","download_url":"https://codeload.github.com/nekolr/slime/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254521789,"owners_count":22084981,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","spider","visual-crawler","websocket"],"created_at":"2024-11-19T20:31:38.731Z","updated_at":"2025-05-16T11:31:06.188Z","avatar_url":"https://github.com/nekolr.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/nekolr/slime\"\u003e\u003cimg src=\"slime.svg\" width=\"128\" height=\"128\" alt=\"logo\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/nekolr/slime/blob/master/LICENSE\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/license/nekolr/slime?style=flat-square\" alt=\"license\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://hub.docker.com/r/nekolr/slime\"\u003e\n    \u003cimg src=\"https://img.shields.io/docker/v/nekolr/slime?label=docker%20image\u0026style=flat-square\" alt=\"docker image\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/nekolr/slime/releases\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/v/release/nekolr/slime?style=flat-square\u0026color=blueviolet\u0026include_prereleases\" alt=\"release\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/nekolr/slime/tags\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/v/tag/nekolr/slime?style=flat-square\" alt=\"tag\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/nekolr/slime\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/stars/nekolr/slime?style=flat-square\" alt=\"stars\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n## 介绍\n一个可视化的爬虫平台。以流程图的方式配置爬虫，基本上无需编写代码即可完成工作。\n\n本项目源自开源项目 [spider-flow](https://github.com/ssssssss-team/spider-flow) ，本着学习的目的，在它的基础上进行了重构，修复了一些问题，并增加了一些新的功能。\n\n## 特性\n- 支持 xpath 和 css 选择器\n- 支持选择器提取、正则提取、json 提取等\n- 支持 Cookie 自动管理\n- 支持抓取由 js 动态渲染的页面\n- 支持代理\n- 支持多数据源\n- 内置常用的字符串、日期、文件、加解密等函数\n- 支持结果保存至多目的地（数据库、csv 文件等）\n- 支持插件扩展（自定义执行器，自定义函数等）\n- 支持任务日志\n- 支持爬虫可视化调试\n\n新增的特性：\n\n- 采用内置数据库 [H2](https://github.com/h2database/h2database) ，做到开箱即用\n- 支持同步执行，对于执行结果有顺序要求的可以使用该功能\n- 增加随机 User-Agent（数据来自：[useragentstring.com](http://useragentstring.com/pages/useragentstring.php) ）\n- 增加代理的管理界面，方便进行代理的手工添加（不推荐）和启动代理的自动管理功能\n- 增加身份认证机制，默认账号和密码都是 slime，可以通过环境变量 `SLIME_USERNAME` 和 `SLIME_PASSWORD` 修改\n- 增加 Feed 解析功能\n- 增加延迟执行节点\n- 通过远程 WebDriver 来操纵浏览器（本地不用再加载驱动，更省心）\n\n## 感谢\n[spider-flow](https://github.com/ssssssss-team/spider-flow) - 新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。\n\n## 免责声明\n请勿使用本项目进行任何可能会违反法律规定和道德约束的工作。如您选择使用本项目，即代表您遵守此声明，作者不承担由于您违反此声明所带来的任何法律风险和损失。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnekolr%2Fslime","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnekolr%2Fslime","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnekolr%2Fslime/lists"}