{"id":16558718,"url":"https://github.com/pengmaster/spidertool","last_synced_at":"2025-03-21T11:31:28.058Z","repository":{"id":143248665,"uuid":"149429449","full_name":"pengMaster/SpiderTool","owner":"pengMaster","description":":collision:最简单的万能爬虫器","archived":false,"fork":false,"pushed_at":"2018-09-21T11:28:21.000Z","size":697,"stargazers_count":41,"open_issues_count":0,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-18T00:46:06.263Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pengMaster.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-19T09:55:04.000Z","updated_at":"2023-05-06T02:44:49.000Z","dependencies_parsed_at":"2023-06-29T19:15:52.328Z","dependency_job_id":null,"html_url":"https://github.com/pengMaster/SpiderTool","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pengMaster%2FSpiderTool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pengMaster%2FSpiderTool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pengMaster%2FSpiderTool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pengMaster%2FSpiderTool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pengMaster","download_url":"https://codeload.github.com/pengMaster/SpiderTool/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244790742,"owners_count":20510800,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T20:23:32.906Z","updated_at":"2025-03-21T11:31:28.038Z","avatar_url":"https://github.com/pengMaster.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 最简单的万能爬虫器\n\n### 项目介绍\n- 爬取指定网站url所有图片以及描述信息\n- 爬取指定网站中任何标签下的任意属性\n\n### 效果展示\n\u003cdiv\u003e\n    \u003cimg src=\"https://github.com/pengMaster/picApplyGit/blob/master/spider/spider_top_bg.jpg\"   alt=\"\"/\u003e\n\u003c/div\u003e\n\n##### 1.获取所有图片以及描述信息\n```java\n    ImageResult(name='142847_16a48_236.jpg', url='http://i.meizitu.net/thumbs/2018/07/142847_16a48_236.jpg', desc='艺妓的诱惑 日系美女水嫩性感柔媚多情', extendOne='')\n    ImageResult(name='146379_12c46_236.jpg', url='http://i.meizitu.net/thumbs/2018/08/146379_12c46_236.jpg', desc='美到喷血! 尤果女神韩安琪美艳性感尽显成熟女人味', extendOne='')\n    ImageResult(name='144268_27c28_236.jpg', url='http://i.meizitu.net/thumbs/2018/07/144268_27c28_236.jpg', desc='妩媚小女人张馨彤为诱惑而生 水湿白衬衫酥胸诱人', extendOne='')\n    ImageResult(name='137510_05b45_236.jpg', url='http://i.meizitu.net/thumbs/2018/06/137510_05b45_236.jpg', desc='尤果女神李莉莉高清写真 雪白酥胸诱惑满满', extendOne='')\n    ImageResult(name='147468_20f29_236.jpg', url='http://i.meizitu.net/thumbs/2018/08/147468_20f29_236.jpg', desc='气质满分！美女尤物美人骆雪淇白嫩美腿妩媚动人', extendOne='')\n    ImageResult(name='150001_07b63_236.jpg', url='http://i.meizitu.net/thumbs/2018/09/150001_07b63_236.jpg', desc='微胖小姐姐白晓白 轻熟的气质让你欲罢不能', extendOne='')\n    ImageResult(name='140752_29b05_236.jpg', url='http://i.meizitu.net/thumbs/2018/07/140752_29b05_236.jpg', desc='尤果美女轮番上阵诱惑升级 清凉陪你度过甜蜜盛夏', extendOne='')\n    ImageResult(name='147534_20e07_236.jpg', url='http://i.meizitu.net/thumbs/2018/08/147534_20e07_236.jpg', desc='还是熟妇最有味道 成熟端庄万种风情', extendOne='')\n\n```\n##### 2.获取指定标签（img）下面指定属性（\"data-original\", \"alt\"）\n```java\n    [http://i.meizitu.net/thumbs/2018/08/148211_26b29_236.jpg, 尤果女神方子萱奶大腿长屁股翘，动作挑逗引人遐想]\n    [http://i.meizitu.net/thumbs/2018/06/140460_28c43_236.jpg, 妩媚轻熟女极品美臀大胆露出 惹火身材完美勾魂]\n    [http://i.meizitu.net/thumbs/2018/06/138976_16y03_236.jpg, 气质小姐姐清爽优雅魅力难挡 性感妩媚尽显轻熟女范]\n    [http://i.meizitu.net/thumbs/2018/06/140116_25c19_236.jpg, 御姐气息十足，尤果模特莫妮卡尽显精致小女人气质]\n    [http://i.meizitu.net/thumbs/2018/08/145201_02a09_236.jpg, 妖艳御姐萌汉药萌汉药baby撩衣露乳激情销魂]\n```\n\n### 最简单使用方式\n\n##### 1. Add it in your root build.gradle at the end of repositories:\n\n```java\nallprojects {\n\t\trepositories {\n\t\t\t...\n\t\t\tmaven { url 'https://jitpack.io' }\n\t\t}\n\t}\n```\n##### 2. Add the dependency\n\n```java\ndependencies {\n\t        implementation 'com.github.pengMaster:SpiderTool:1.0.0'\n\t}\n```\n##### 3. Add use to activity\n\n```java\n\n         案例： 获取妹子图网站所有图片\n           //妹子图网站Url\n           val url = \"http://www.mzitu.com/tag/ugirls/\"\n           //kotlin 获取该网站所有图片以及描述信息\n           SpiderUtil.getImageByUrl(url, object : ImageResultListener {\n                override fun result(imageList: HashSet\u003cImageResult\u003e) {\n                    mTvContent.text = imageList.toString()\n                    imageList.forEach {\n                        Log.e(\"getImageByUrl\", it.toString())\n                    }\n\n                }\n            })\n```\n```java\n           //java 版\n            SpiderUtil.Companion.getImageByUrl(url, new ImageResultListener() {\n                @Override\n                public void result(@NotNull HashSet\u003cImageResult\u003e imageList) {\n\n                }\n            });\n```\n```java\n          案例： 获取妹子图网站所有img标签下\"data-original\", \"alt\"\n            //标签\n            val tag = \"img\"\n            //要获取属性集合\n            val attributeList = listOf\u003cString\u003e(\"data-original\", \"alt\")\n            SpiderUtil.getAttributeByTag(url, tag, attributeList, object : ResultListener {\n                override fun result(list: HashSet\u003cArrayList\u003cString\u003e\u003e) {\n                    mTvContent.text = list.toString()\n                    list.forEach {\n                        Log.e(\"getAttributeByTag\", it.toString())\n                    }\n                }\n            })\n\n```\n\n##### 4. add permission to your menifest\n\n```java\n    \u003c!--联网--\u003e\n    \u003cuses-permission android:name=\"android.permission.INTERNET\" /\u003e\n```\n#### 注意\n    1.比如妹子图网站获取的图片url如果进行展示或者下载，需要进行防盗链处理操作\n    2.获取指定标签下的指定属性\n    例如：\n```java\n\u003cimg width='236' height='354' class='lazy' alt='清纯中带着性感 尤果网女神美替极品身材一览无遗' src='http://i.meizitu.net/pfiles/img/lazy.png' data-original='http://i.meizitu.net/thumbs/2018/09/150833_14c39_236.jpg' /\u003e\n```\n    获取img标签下  alt  src 属性\n    tag = img\n    list = (\"alt\",  \"src\")\n\n\n#### 参与贡献\n\n1. Fork 本项目\n2. 新建 Feat_xxx 分支\n3. 提交代码\n4. 新建 Pull Request\n\n\n#### github地址\n\n - 项目地址：https://github.com/pengMaster/SpiderTool\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpengmaster%2Fspidertool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpengmaster%2Fspidertool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpengmaster%2Fspidertool/lists"}