{"id":20768367,"url":"https://github.com/ainilili/seeker","last_synced_at":"2025-07-12T02:07:42.671Z","repository":{"id":105569553,"uuid":"120174067","full_name":"ainilili/seeker","owner":"ainilili","description":"超文本解析工具+网络爬虫工具","archived":false,"fork":false,"pushed_at":"2018-02-04T12:58:04.000Z","size":26,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-18T06:43:31.479Z","etag":null,"topics":["html","xml"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ainilili.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-04T10:14:01.000Z","updated_at":"2018-11-14T06:22:35.000Z","dependencies_parsed_at":"2024-03-25T12:49:21.963Z","dependency_job_id":null,"html_url":"https://github.com/ainilili/seeker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ainilili%2Fseeker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ainilili%2Fseeker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ainilili%2Fseeker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ainilili%2Fseeker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ainilili","download_url":"https://codeload.github.com/ainilili/seeker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243098224,"owners_count":20235990,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["html","xml"],"created_at":"2024-11-17T11:38:15.159Z","updated_at":"2025-03-11T19:29:13.015Z","avatar_url":"https://github.com/ainilili.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 介绍\n\n#### 为什么取名Seeker\n\n我创作Seeker的初衷是为了开发一个可视化爬虫系统，而Seeker作为其中的组件之一英译的含义是`n. 探求者；搜查人`，我认为这个名字很适合它，所以决定定名为Seeker~\n\n#### Seeker的作用\n\nSeeker作为组件之一，它所处的角色就是解析+搜索，它可以将一个超文本解析成一个可操作的容器，我们使用这个容器来获取我们想要的信息。\n\n#### Seeker的结构\n\nSeeker目前由一下几个组块总成：\n\n\u003e *   Document：文本存储结构及字典\n\u003e     \n\u003e     \n\u003e *   Regex ：负责提供正则支持\n\u003e     \n\u003e     \n\u003e *   Scanner ：负责扫描\n\u003e     \n\u003e     \n\u003e *   Searcher ：负责搜索\n\u003e     \n\u003e     \n\u003e *   Http ：负责Http请求\n\u003e     \n\u003e     \n\u003e *   Stream ：负责数据流处理\n\u003e     \n\u003e     \n\u003e *   Plan：定制计划\n\u003e     \n\u003e     \n\u003e *   Starter：负责计划的启动\n\u003e     \n\u003e     \n\u003e *   StarterHelper：快速启动\n\n其中**Scanner**的作用在其中处于核心的地位，超文本解析处理全靠它，不过很遗憾的是目前解析算法的时间复杂度和空间复杂度都是`O(n^2)`，这个待优化中。\n\n# 详解\n\n在使用Seeker之前，我将详细介绍一下前面所讲的几个组块没块的协作流程。\n\n### Document\n\n#### DomBean\n\nDomBean（文本bean）一块是负责储存Scanner扫描后的数据，它的数据结构是一个普通树（tree），比如以下文本\n\n```\n\u003c!DOCTYPE html\u003e\n\u003chtml lang=\"en\"\u003e\n    \u003chead\u003e\n        \u003cmeta charset=\"UTF-8\"\u003e\n        \u003ctitle\u003eTitle\u003c/title\u003e\n    \u003c/head\u003e\n    \u003cbody\u003e\n\n    \u003c/body\u003e\n\u003c/html\u003e\n\n```\n\n存储后的结构如下\n\n![image.png](http://nginx.ikuvn.com:1024/images/261504761175853.jpg)\n\n每一个标签都相当于一个DomBean，它的子标签会存储在它自身的一个List中，他的标签属性会存储在自身的map当中，附带着存储的还有他的标签名和其他一些有用的属性。\n\n#### DomHelper\n\n它可以将DomBean解析成一个html文档（字符串），可以更好的查看当前DomBean的结构\n\n#### DomMember\n\n负责标签的注册，其中分为三个注册商\n\n\u003e *   MEMBERS_NORMAL\n\u003e     \n\u003e     \n\u003e *   MEMBERS_SPECIAL\n\u003e     \n\u003e     \n\u003e *   MEMBERS_OTHER\n\n**MEMBERS_NORMAL** 面对的群体是普通的标签，类似于`\u003cdiv\u003e\u003c/div\u003e`带标签体的标签和`\u003cinput/\u003e`自封闭的标签\n\n**MEMBERS_SPECIAL** 面的的是`\u003cimg\u003e`可以不封闭的标签\n\n**MEMBERS_OTHER** 面对的是`\u003cscript\u003e\u003c/script\u003e`和`\u003cstyle\u003e\u003c/style\u003e`这种不属于Html范畴的标签\n\n在Scanner中这三个注册商非常重要，它们在其中负责对标签类型的验证以达到用不同的方式去解析文档~\n\n### Regex\n\n这一块主要负责一些正则表达式的定义和提供一个对外可以直接使用的正则方法，忽略不讲了\n\n### Scanner\n\nScanner对于整个Seeker服务来讲是非常重要的，对外它解析外来文本，对内提供解析结果存储在DomBean结构中来对其他服务提供数据，它的解析流程图如下\n\n![image.png](http://nginx.ikuvn.com:1024/images/261504764970368.jpg)\n\n通过上述算法及细节处理可以适应多种情景文档的读取\n\n### Seacher\n\n负责提供搜索服务，搜索对象就是Scanner层解析后的DomBean对象，下面是实现的其中一个搜索器之一`NicoSeacher`\n\n```\n        /**\n\t * 通过前后缀及属性名搜索\n\t * @param prefix\n\t * @param paramName\n\t * @param paramValue\n\t * @param domBeans\n\t * @param bank\n\t * @param searchingUnique\n\t */\nprivate void searchingAssemble(String prefix, String paramName, String paramValue, List\u003cDomBean\u003e domBeans, List\u003cDomBean\u003e bank, boolean searchingUnique, DomBean[] tmpDomBeans){\n\t\tif(domBeans == null) return;\n\t\tif(tmpDomBeans != null) domBeans = Arrays.asList(tmpDomBeans);\n\t\tIterator\u003cDomBean\u003e items = domBeans.iterator();\n\t\twhile(items.hasNext()){\n\t\t\tDomBean domBean = items.next();\n\t\t\tif(((domBean.get(paramName) != null \n\t\t\t\t\t\u0026\u0026 domBean.get(paramName).equals(paramValue)) \n\t\t\t\t\t|| StringUtils.isBlank(paramName) )\n\t\t\t\t\t\u0026\u0026 ( domBean.getPrefix().equals(prefix) || StringUtils.isBlank(prefix) )){\n\t\t\t\tbank.add(domBean);\n\t\t\t\tif(searchingUnique){\n\t\t\t\t\treturn;\n\t\t\t\t}\n\t\t\t}\n\t\t\tif(this.globalCheck){\n\t\t\t\tsearchingAssemble(prefix, paramName, paramValue, domBean.getDomProcessers(), bank, searchingUnique, null);\n\t\t\t}\n\t\t}\n\t}\n```\n\n这一部分可以通过实现`com.nico.seeker.searcher.SeekerSearcher`类来换成自己的搜索器\n\n### HTTP\n\n对外提供**HTTP**的`Get`和`Post`请求来获取对应资源的页面源码，提供给Scanner解析\n\n### Stream\n\n流的处理，可以将文件内容读取，提供给Scanner解析\n\n### Plan\n\n自定义计划，可以让Seeker根据Plan的内容进行计划爬取\n\n#### SeekerTrack\n\n自定义爬虫轨迹，其数据结构如下\n\n```\nprivate String uri;\n\n    /*\n     * 请求类型\n     */\n    private HttpMethod httpMethod;\n\n    /*\n     * 请求参数\n     */\n    private Map\u003cString, Object\u003e params;\n\n    /*\n     * 具体轨迹信息\n     */\n    private List\u003cTrackBean\u003e trackBeans;\n\n    /*\n     * 搜索器 \n     */\n    private String searcher;\n\n    /*\n     * 收获\n     */\n    private List\u003cHarvestBean\u003e harvestCollect = new ArrayList\u003cHarvestBean\u003e();\n\n```\n\n其中有两个对象`TrackBean`和`HarvestBean`\n\n**TrackBean**负责定制每一条路线具体的**行程**，数据结构如下\n\n```\n/*\n     * 前缀\n     */\n    private String prefix;\n\n    /*\n     * 属性内容\n     */\n    private String paramValue;\n\n    /*\n     * 属性内容 \n     */\n    private String paramName;\n\n    /*\n     * 是否收获\n     */\n    private Boolean recycle;\n\n    /*\n     * 是否初始化\n     */\n    private Boolean reset;\n\n    /*\n     * 子流程\n     */\n    private List\u003cTrackBean\u003e trackBeans;\n\n    /*\n     * 记录\n     */\n    private String record;\n\n```\n\n**HarvestBean**负责存储需要**收获**的轨迹所收获的数据\n\n这两者在**SeekerTrack**之中共存，相互合作实现`定制计划`=\u003e`分析`=\u003e`轨迹搜索`=\u003e`存储`的流程\n\n### Start\n\n负责以上整个流程的启动\n\n### StartHelper\n\n可以将流程定制为Json格式的数据快速启动，Json格式如下\n\n```\n{    \n'header':{\n            'method':'get',\n            'seacher':'com.nico.seeker.searcher.impl.NicoSearcher',\n            'uri':'http://tieba.baidu.com/f',\n            'params':[{\"key\":\"kw\",\"value\":\"java\"},{\"key\":\"ie\",\"value\":\"utf-8\"},{\"key\":\"pn\",\"value\":\"50\"}]\n        },\n'tracks':[\n            {'prefix':'div','paramName':'class','paramValue':'threadlist_title pull_left j_th_tit '},\n            {'prefix':'a','recycle':'true','record':'帖子链接'}\n        ]\n}\n\n```\n\nStartHelper会解析Json数据自动装配SeekerTrack来供Start启动\n\n# 使用\n## 1、超文本解析\nSeeker对文本的解析分两步：扫描 \u0026 搜索\n\n在解析一个超文本内容之前我们先拟定获取的文本内容为dom，接下来我们首先初始化扫描器：\n```\nSeekerScanner scanner = new NicoScanner(dom);\n```\n接下来获取搜索器\n```\nSeekerSearcher searcher = new NicoSearcher(scanner);\n```\nSeekerSearcher对外提供一下方法：\n\n```\npublic interface SeekerSearcher {\n\t\n\t/**\n\t * 查找DomBean集合\n\t * @param prefix\t\t标签前缀（div、span、ul） \n\t * @param paramName\t\t属性名\n\t * @param paramValue\t属性内容\n\t * @return\n\t */\n\tpublic SeekerSearcher searching(String prefix, String paramName, String paramValue);\n\t\n\t/**\n\t * 查找DomBean集合\n\t * @param prefix\t\t标签前缀（div、span、ul） \n\t * @param paramName\t\t属性名\n\t * @param paramValue\t属性内容\n\t * @param tmpDomBeans\t\t要搜索的集合\n\t * @return\n\t */\n\tpublic SeekerSearcher searching(String prefix, String paramName, String paramValue, DomBean[] tmpDomBeans);\n\t\n\t/**\n\t * 查找DomBean集合\n\t * @param prefix\t\t标签前缀（div） \n\t * @return\n\t */\n\tpublic SeekerSearcher searching(String prefix);\n\t\n\t/**\n\t * 查找DomBean集合\n\t * @param prefix\t\t标签前缀（div） \n\t * @param tmpDomBeans\t\t要搜索的集合\n\t * @return\n\t */\n\tpublic SeekerSearcher searching(String prefix, DomBean[] tmpDomBeans);\n\t\n\t/**\n\t * 查找DomBean集合\n\t * @param prefix\t\t标签前缀（div） \n\t * @param ret\t\t\ttrue：不跟进\n\t * @return\n\t */\n\tpublic SeekerSearcher searching(String prefix, boolean ret);\n\t\n\t/**\n\t * 查找DomBean集合\n\t * @param prefix\t\t标签前缀（div） \n\t * @param ret\t\t\ttrue：不跟进\n\t * @param tmpDomBeans\t\t要搜索的集合\n\t * @return\n\t */\n\tpublic SeekerSearcher searching(String prefix, boolean ret, DomBean[] tmpDomBeans);\n\t\n\t/**\n\t * 查找DomBean集合\n\t * @param prefix\t\t标签前缀（div、span、ul） \n\t * @param paramName\t\t属性名\n\t * @param paramValue\t属性内容\n\t * @param ret\t\t\ttrue：不跟进\n\t * @return\n\t */\n\tpublic SeekerSearcher searching(String prefix, String paramName, String paramValue, boolean ret);\n\t\n\t/**\n\t * 查找DomBean集合\n\t * @param prefix\t\t标签前缀（div、span、ul） \n\t * @param paramName\t\t属性名\n\t * @param paramValue\t属性内容\n\t * @param ret\t\t\ttrue：不跟进\n\t * @param tmpDomBeans\t\t要搜索的集合\n\t * @return\n\t */\n\tpublic SeekerSearcher searching(String prefix, String paramName, String paramValue, boolean ret, DomBean[] tmpDomBeans);\n\t\n\t/**\n\t * 获取DomBean集合\n\t * @return\n\t */\n\tpublic List\u003cDomBean\u003e getResults();\n\t\n\t/**\n\t * 获取单个DomBean\n\t * @return\n\t */\n\tpublic DomBean getSingleResult();\n\t\n\t/**\n\t * 设置DomBeans搜索对象\n\t * @param domBeans\n\t */\n\tpublic SeekerSearcher setDomBeans(DomBean[]  domBeans);\n\t\n\t/**\n\t * 设置DomBeans搜索对象\n\t * @param domBeans\n\t */\n\tpublic SeekerSearcher setDomBeans(List\u003cDomBean\u003e domBeans);\n\t\n\t/**\n\t * 是否开启全局搜索（默认开启）\n\t * @param globalCheck\n\t */\n\tpublic SeekerSearcher setGlobalCheck(boolean globalCheck);\n\t\n\t/**\n\t * 回到根节点\n\t */\n\tpublic SeekerSearcher reset();\n}\n```\n### 使用示例：\n以一下文本为例：\n```\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cbooks\u003e\n    \u003cbook id=\"dataSource\" class=\"com.nico.db.datasource.DataSource\"\u003e\n        \u003cparam key=\"uri\" value=\"jdbc:mysql://localhost:3306/nodb?useUnicode=true\u0026amp;characterEncoding=utf8\" /\u003e\n        \u003cparam key=\"username\" value=\"root\" /\u003e\n        \u003cparam key=\"password\" value=\"root\" /\u003e\n        \u003cparam key=\"driver\" value=\"com.mysql.jdbc.Driver\" /\u003e\n    \u003c/book\u003e\n    \n    \u003cbook id=\"session\" class=\"com.nico.db.session.branch.MysqlSession\"\u003e\n        \u003clabel name=\"dataSource\" ref=\"dataSource\"/\u003e\n    \u003c/book\u003e\n    \n    \u003cbook id=\"helper\" class=\"com.nico.db.helper.impl.MysqlDBHelper\"\u003e\n        \u003clabel name=\"session\" ref=\"session\"/\u003e\n    \u003c/book\u003e\n    \n    \u003cbook id=\"CustomerDaoImpl\" class=\"com.nico.example.dao.impl.CustomerDaoImpl\"\u003e\n    \t \u003cparam key=\"name\" value=\"nico\" /\u003e\n    \t \u003clabel name=\"dataSource\" ref=\"dataSource\"/\u003e\n    \u003c/book\u003e\n\u003c/books\u003e\n```\n#### 获取所有的book标签\n```\nseacher.searching(\"book\").getResults()\n```\n获取结果\n```\n[DomBean [param={id=dataSource, class=com.nico.db.datasource.DataSource}, prefix=book, selfSealing=false, paramStr=id=\"dataSource\" class=\"com.nico.db.datasource.DataSource\", body=\n        \u003cparam key=\"uri\" value=\"jdbc:mysql://localhost:3306/nodb?useUnicode=true\u0026amp;characterEncoding=utf8\" /\u003e\n        \u003cparam key=\"username\" value=\"root\" /\u003e\n        \u003cparam key=\"password\" value=\"root\" /\u003e\n        \u003cparam key=\"driver\" value=\"com.mysql.jdbc.Driver\" /\u003e\n    ], DomBean [param={id=session, class=com.nico.db.session.branch.MysqlSession}, prefix=book, selfSealing=false, paramStr=id=\"session\" class=\"com.nico.db.session.branch.MysqlSession\", body=\n        \u003clabel name=\"dataSource\" ref=\"dataSource\"/\u003e\n    ], DomBean [param={id=helper, class=com.nico.db.helper.impl.MysqlDBHelper}, prefix=book, selfSealing=false, paramStr=id=\"helper\" class=\"com.nico.db.helper.impl.MysqlDBHelper\", body=\n        \u003clabel name=\"session\" ref=\"session\"/\u003e\n    ], DomBean [param={id=CustomerDaoImpl, class=com.nico.example.dao.impl.CustomerDaoImpl}, prefix=book, selfSealing=false, paramStr=id=\"CustomerDaoImpl\" class=\"com.nico.example.dao.impl.CustomerDaoImpl\", body=\n    \t \u003cparam key=\"name\" value=\"nico\" /\u003e\n    \t \u003clabel name=\"dataSource\" ref=\"dataSource\"/\u003e\n    ]]\n```\n#### 获取book标签下的param标签\n```\nseacher.searching(\"book\").searching(\"param\").getResults()\n```\n获取结果\n```\n[DomBean [param={value=jdbc:mysql://localhost:3306/nodb?useUnicode=true\u0026amp;characterEncoding=utf8, key=uri}, prefix=param, selfSealing=true, paramStr=key=\"uri\" value=\"jdbc:mysql://localhost:3306/nodb?useUnicode=true\u0026amp;characterEncoding=utf8\" , body=null], DomBean [param={value=root, key=username}, prefix=param, selfSealing=true, paramStr=key=\"username\" value=\"root\" , body=null], DomBean [param={value=root, key=password}, prefix=param, selfSealing=true, paramStr=key=\"password\" value=\"root\" , body=null], DomBean [param={value=com.mysql.jdbc.Driver, key=driver}, prefix=param, selfSealing=true, paramStr=key=\"driver\" value=\"com.mysql.jdbc.Driver\" , body=null], DomBean [param={value=nico, key=name}, prefix=param, selfSealing=true, paramStr=key=\"name\" value=\"nico\" , body=null]]\n```\n#### 获取第一个book标签下的param标签\n```\nDomBean domBean = seacher.searching(\"book\").getSingleResult();\n\t\tList\u003cDomBean\u003e results = seacher.searching(\"param\", new DomBean[]{domBean}).getResults();\n\t\tSystem.out.println(results);\n```\n获取结果\n```\n[DomBean [param={value=jdbc:mysql://localhost:3306/nodb?useUnicode=true\u0026amp;characterEncoding=utf8, key=uri}, prefix=param, selfSealing=true, paramStr=key=\"uri\" value=\"jdbc:mysql://localhost:3306/nodb?useUnicode=true\u0026amp;characterEncoding=utf8\" , body=null], DomBean [param={value=root, key=username}, prefix=param, selfSealing=true, paramStr=key=\"username\" value=\"root\" , body=null], DomBean [param={value=root, key=password}, prefix=param, selfSealing=true, paramStr=key=\"password\" value=\"root\" , body=null], DomBean [param={value=com.mysql.jdbc.Driver, key=driver}, prefix=param, selfSealing=true, paramStr=key=\"driver\" value=\"com.mysql.jdbc.Driver\" , body=null]]\n```\n#### 获取key=password的标签\n```\nDomBean domBean = seacher.searching(\"param\", \"key\", \"password\").getSingleResult();\n\t\tSystem.out.println(domBean);\n```\n获取结果\n```\nDomBean [param={value=root, key=password}, prefix=param, selfSealing=true, paramStr=key=\"password\" value=\"root\" , body=null]\n```\n#### reset()的使用\n```\nDomBean domBean = seacher.searching(\"param\", \"key\", \"password\").getSingleResult();\n\t\tSystem.out.println(domBean);\n\t\tdomBean = seacher.searching(\"param\", \"key\", \"username\").getSingleResult();\n\t\tSystem.out.println(domBean);\n\t\tdomBean = seacher.reset().searching(\"param\", \"key\", \"username\").getSingleResult();\n\t\tSystem.out.println(domBean);\n```\n获取结果，第二次获取不到对应的标签，reset一下就可以获取到\n```\nDomBean [param={value=root, key=password}, prefix=param, selfSealing=true, paramStr=key=\"password\" value=\"root\" , body=null]\nnull\nDomBean [param={value=root, key=username}, prefix=param, selfSealing=true, paramStr=key=\"username\" value=\"root\" , body=null]\n```\n#### 带回归的搜索\n```\nDomBean domBean = seacher.searching(\"param\", \"key\", \"password\", true).getSingleResult();\n\t\tSystem.out.println(domBean);\n\t\tdomBean = seacher.searching(\"param\", \"key\", \"username\").getSingleResult();\n\t\tSystem.out.println(domBean);\n```\n获取结果，第二次获取可以获取到值\n```\nDomBean [param={value=root, key=password}, prefix=param, selfSealing=true, paramStr=key=\"password\" value=\"root\" , body=null]\nDomBean [param={value=root, key=username}, prefix=param, selfSealing=true, paramStr=key=\"username\" value=\"root\" , body=null]\n```\n#### 设置搜索模板，获取第一个book标签的第一个param标签\n```\nList\u003cDomBean\u003e books = seacher.searching(\"param\").getResults();\n\t\tDomBean domBean = seacher.searching(\"param\", new DomBean[]{books.get(0)}).getSingleResult();\n\t\tSystem.out.println(domBean);\n```\n获取结果\n```\nDomBean [param={value=jdbc:mysql://localhost:3306/nodb?useUnicode=true\u0026amp;characterEncoding=utf8, key=uri}, prefix=param, selfSealing=true, paramStr=key=\"uri\" value=\"jdbc:mysql://localhost:3306/nodb?useUnicode=true\u0026amp;characterEncoding=utf8\" , body=null]\n```\n\n当然以上示例中的参数混搭效果更佳，使用前最好看一下接口提供的方法介绍及参数介绍。\n\n## 2、网络爬虫\n\n### Seacher级使用\n\n我们拿CSDN专家组用户页做测试\n\n首先我们需要获取一个Scanner对象\n\n```\nMap\u003cString, Object\u003e params = new HashMap\u003cString, Object\u003e();\n        params.put(\"channelid\", 0);\n        params.put(\"page\", 1);\n        SeekerScanner scan = new NicoScanner(\"http://blog.csdn.net/peoplelist.html\", HttpMethod.GET, params);\n\n```\n\n然后我们获取Seacher对象\n\n```\nSeekerSearcher ns = new NicoSearcher(scan);\n\n```\n\n接下来使用Seacher来进行搜索，比如获取页码\n\n```\nList\u003cDomBean\u003e domBeans = ns.searching(\"div\", \"class\", \"page_nav\").searching(\"span\").getResults();\n        DomBean domBean = domBeans.get(0);\n        System.out.println(domBean);\n\n```\n\n运行之后控制台打印结果\n\n```\nDomBean [param={}, prefix=span, selfSealing=false, paramStr=, body= 1515条  共127页]\n\n```\n\n我们再看一下对应的页面源码\n\n```\n\u003cdiv class=\"page_nav\"\u003e\n    \u003cspan\u003e 1515条  共127页\u003c/span\u003e\n    \u003cstrong\u003e1\u003c/strong\u003e \n    \u003ca href=\"/peoplelist.html?\u0026page=2\"\u003e2\u003c/a\u003e\n    \u003ca href=\"/peoplelist.html?\u0026page=3\"\u003e3\u003c/a\u003e \n    \u003ca href=\"/peoplelist.html?\u0026page=4\"\u003e4\u003c/a\u003e\n    \u003ca href=\"/peoplelist.html?\u0026page=5\"\u003e5\u003c/a\u003e \n    \u003ca href=\"/peoplelist.html?\u0026page=6\"\u003e...\u003c/a\u003e \n    \u003ca href=\"/peoplelist.html?\u0026page=2\"\u003e下一页\u003c/a\u003e\n    \u003ca href=\"/peoplelist.html?\u0026page=127\"\u003e尾页\u003c/a\u003e \n\u003c/div\u003e\n\n```\n\n在这里要说一下`searching`这个方法的规则\n\n#### 搜索规则一\n\n```\nScanner扫描后的数据集是包含整个文本所有标签的，没searching一次，则当前数据集就会变成searching之后的数据集，也就是说，没搜索一次，我们的数据集都在更新，下次搜索之后再前一次搜索的基础上进行检索！！\n\n```\n\n那么我们如果想恢复到最初的数据集呢？当然有办法~\n\n`Seacher`提供给一个方法`public void reset()`可以恢复\n\n#### 搜索规则二\n\n```\nSeacher默认搜索是对全局文档 进行搜索的，但是我们的文档可是分层的，如果我们只想搜索当前层的文档怎么办呢？\n\n```\n\n`Seacher`提供`public void setGlobalCheck(boolean globalCheck)`来改变当前搜索范围\n\n`boolean globalCheck`设定\n\n| 值 | 影响 |\n| --- | --- |\n| true | 全局搜索 |\n| false | 只针对本层搜索 |\n\n### Start级使用\n\n使用Start级我们的流程制定比起Seacher级就规范了很多，同样上述操作获取页码，我们的代码就可以变成如下\n\n```\n//设置请求参数\n        Map\u003cString, Object\u003e params = new HashMap\u003cString, Object\u003e();\n        params.put(\"channelid\", 0);\n        params.put(\"page\", 1);\n        //设置轨迹\n        List\u003cTrackBean\u003e ts = new ArrayList\u003cTrackBean\u003e();\n        //轨迹一\n        ts.add(new TrackBean(\"div\", \"class\", \"page_nav\"));\n        //轨迹二（带回收和记录）\n        TrackBean t = new TrackBean(\"span\",true);\n        t.setRecord(\"页码爬取\");\n        ts.add(t);\n        //装入SeekerTrack\n        SeekerTrack st = new SeekerTrack();\n        st.setUri(\"http://blog.csdn.net/peoplelist.html\");\n        st.setParams(params);\n        st.setSearcher(\"com.nico.seeker.searcher.impl.NicoSearcher\");\n        st.setHttpMethod(HttpMethod.GET);\n        st.setTrackBeans(ts);\n        //装入SeekerStart\n        SeekerStart ss = new SeekerStart(st);\n        //启动SeekerStart\n        ss.run();\n        //查看结果集\n        for(HarvestBean hb: ss.getHarvestCollect()){\n            System.out.println(hb);\n        }\n\n```\n\n控制台打印结果\n\n```\nHarvestBean [time=2017-09-07 03:03:46, domBeans=[DomBean [param={}, prefix=span, selfSealing=false, paramStr=, body= 1515条 共127页]], record=页码爬取]\n\n```\n\n但是这样是不是也很繁琐呢？能不能更简单呢？请往下看~\n\n### StartHelper启动方式\n\n首先准备我们的Json文件\n\n```\n{\n{    \n'header':{\n            'method':'get',\n            'seacher':'com.nico.seeker.searcher.impl.NicoSearcher',\n            'uri':'http://blog.csdn.net/peoplelist.html',\n            'params':[{'key':'channelid','value':'0'},{'key':'page','value':'1'}]\n        },\n'tracks':[\n            {'prefix':'div','paramName':'class','paramValue':'page_nav'},\n            {'prefix':'span','recycle':'true','record':'页码爬取'}\n        ]\n}\n\n```\n\n接下来是Java代码\n\n```\nString tracks = \"\";\n        try {\n            tracks = NioUtils.readFileToString(new File(\"E://test/csdn.json\"));\n        } catch (IOException e) {\n            e.printStackTrace();\n        }\n        SeekerStarterHelper ssh = new SeekerStarterHelper(new SeekerStart());\n        ssh.runHelper(tracks);\n        List\u003cHarvestBean\u003e hbs = ssh.getHarvestCollect();\n        for(HarvestBean h: hbs){\n            System.out.println(h);\n        }\n\n```\n\n控制台打印结果\n\n```\nHarvestBean [time=2017-09-07 03:33:11, domBeans=[DomBean [param={}, prefix=span, selfSealing=false, paramStr=, body= 1515条 共127页]], record=页码爬取]\n\n```\n\n很少的代码就可以爬取内容\n\n# 后记\n\n`Seeker`已托管于`GitHub [https://github.com/ainilili/seeker](https://github.com/ainilili/seeker)`","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fainilili%2Fseeker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fainilili%2Fseeker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fainilili%2Fseeker/lists"}