{"id":15485975,"url":"https://github.com/licoy/java-crawler","last_synced_at":"2025-04-10T16:50:27.907Z","repository":{"id":99594742,"uuid":"100690433","full_name":"Licoy/Java-Crawler","owner":"Licoy","description":"通过java使用jsoup爬虫框架爬取数据 ","archived":false,"fork":false,"pushed_at":"2017-08-18T08:43:04.000Z","size":62,"stargazers_count":4,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-19T07:03:40.239Z","etag":null,"topics":["crawler","java","jsoup"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Licoy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-08-18T08:24:48.000Z","updated_at":"2021-12-03T10:00:58.000Z","dependencies_parsed_at":"2023-06-28T06:53:21.703Z","dependency_job_id":null,"html_url":"https://github.com/Licoy/Java-Crawler","commit_stats":{"total_commits":6,"total_committers":1,"mean_commits":6.0,"dds":0.0,"last_synced_commit":"cd37eb0fe601438e54a45a208b7cceb22ebabdd7"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Licoy%2FJava-Crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Licoy%2FJava-Crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Licoy%2FJava-Crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Licoy%2FJava-Crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Licoy","download_url":"https://codeload.github.com/Licoy/Java-Crawler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248255723,"owners_count":21073370,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","java","jsoup"],"created_at":"2024-10-02T06:05:21.603Z","updated_at":"2025-04-10T16:50:27.898Z","avatar_url":"https://github.com/Licoy.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# JAVA爬虫 - Java-Crawler\n使用java爬虫框架(jsoup)爬取数据\n# 例：爬开源中国最新项目\n## 调用\n```\npublic class GitOsChinaTest {\n    @Test\n    public void run() throws Exception {\n        new GitOsChina(1,2).run();\n    }\n}\n```\n## 结果：\n```\n16:35:31.063 [pool-1-thread-1] INFO cn.licoy.thread.GitOsChina - -\u003e当前执行 : https://git.oschina.net/explore/recommend?page=1\n16:35:31.070 [pool-1-thread-2] INFO cn.licoy.thread.GitOsChina - -\u003e当前执行 : https://git.oschina.net/explore/recommend?page=2\n16:35:32.771 [main] INFO cn.licoy.thread.GitOsChina - -\u003e线程池任务已执行完成\nProject{name='smart-web2', author='狂晕', lang='Java', watch='139', star='288', fork='125', url='https://git.oschina.net/bcworld/smart-web2'}\nProject{name='UPMS', author='lyg945', lang='Java', watch='38', star='56', fork='20', url='https://git.oschina.net/lyg945/UPMS'}\nProject{name='kind', author='郑州java', lang='Java', watch='75', star='136', fork='52', url='https://git.oschina.net/zhengzhoujava/kind'}\nProject{name='xtp通用权限管理系统', author='贤二智能', lang='Java', watch='34', star='56', fork='32', url='https://git.oschina.net/shenghaijiang/xtp'}\nProject{name='SVN资源权限管理系统', author='微笑风采', lang='Java', watch='85', star='159', fork='87', url='https://git.oschina.net/hpboys/svnadmin'}\nProject{name='React-VR', author='紫苏吃个蕉', lang='JavaScript', watch='8', star='27', fork='6', url='https://git.oschina.net/zisuzz/react-vr'}\nProject{name='xxpay', author='jmdhappy', lang='Java', watch='194', star='468', fork='168', url='https://git.oschina.net/jmdhappy/xxpay-master'}\nProject{name='easy-jdbc', author='yydf', lang='Java', watch='10', star='27', fork='7', url='https://git.oschina.net/yydf/easy-jdbc'}\nProject{name='Pigeon', author='monsterLin', lang='Android', watch='18', star='34', fork='5', url='https://git.oschina.net/monsterLin/Pigeon'}\nProject{name='asyncio', author='james', lang='C++', watch='2', star='3', fork='2', url='https://git.oschina.net/zhanglix/asyncio'}\nProject{name='OpenTracker', author='小樱', lang='C', watch='4', star='5', fork='0', url='https://git.oschina.net/cc12655/OpenTracker'}\nProject{name='JApiDocs', author='叶大侠', lang='Java', watch='52', star='118', fork='25', url='https://git.oschina.net/yeguozhong/JApiDocs'}\nProject{name='Hero', author='点融科技', lang='JavaScript', watch='47', star='57', fork='16', url='https://git.oschina.net/dianrong/Hero'}\nProject{name='vnpy', author='若水', lang='C++', watch='18', star='42', fork='9', url='https://git.oschina.net/wruoshuiy/vnpy'}\nProject{name='YurunEvent', author='宇润', lang='PHP', watch='6', star='11', fork='0', url='https://git.oschina.net/yurunsoft/yurunevent'}\nProject{name='jTool', author='拭目以待', lang='JavaScript', watch='4', star='5', fork='3', url='https://git.oschina.net/baukh/jTool'}\nProject{name='SpringBootUnity', author='小莫', lang='Java', watch='108', star='214', fork='70', url='https://git.oschina.net/hupeng/SpringBootUnity'}\nProject{name='ad_tools', author='jiangzeyin', lang='Java', watch='15', star='24', fork='12', url='https://git.oschina.net/jiangzeyin/ad_tools'}\nProject{name='dbtracer', author='ghsea', lang='Java', watch='18', star='37', fork='10', url='https://git.oschina.net/ghsea/dbtracer'}\nProject{name='sc', author='Vincent', lang='Java', watch='33', star='78', fork='28', url='https://git.oschina.net/wangxinforme/sc'}\nProject{name='mylinks-m0m1-open-sdk', author='浙江劢领智能科技有限公司', lang='C', watch='6', star='9', fork='5', url='https://git.oschina.net/mqlinks/mylinks-m0m1-open-sdk'}\nProject{name='phpboot', author='cayman', lang='PHP', watch='11', star='25', fork='3', url='https://git.oschina.net/caoyangmin/phpboot'}\nProject{name='SmartSql', author='Ahoo', lang='C#', watch='8', star='13', fork='4', url='https://git.oschina.net/AhooWang/SmartSql'}\nProject{name='壹凯巴cmsV2.0', author='yihank', lang='PHP', watch='7', star='13', fork='7', url='https://git.oschina.net/yihank/YiKaiBacmsV2.0'}\nProject{name='JFBlog-maven', author='Realfighter', lang='Java', watch='8', star='20', fork='15', url='https://git.oschina.net/realfighter/JFBlog-maven'}\nProject{name='arthur', author='ArthurFamily', lang='Java', watch='10', star='14', fork='6', url='https://git.oschina.net/ArthurFamily/arthur'}\nProject{name='Rexjs', author='china-liji', lang='JavaScript', watch='3', star='1', fork='0', url='https://git.oschina.net/jQun/Rexjs'}\nProject{name='TensorFlow-Bitcoin-Robot', author='feiwang', lang='Python', watch='6', star='9', fork='2', url='https://git.oschina.net/fendouai/TensorFlow-Bitcoin-Robot'}\nProject{name='Brouhaha', author='惊奇漫画', lang='Objective-C', watch='3', star='4', fork='0', url='https://git.oschina.net/JingQiManHua/Brouhaha'}\nProject{name='tangyuan2', author='xson_org', lang='Java', watch='23', star='19', fork='12', url='https://git.oschina.net/xsonorg/tangyuan2'}\nProject{name='weixin-dubbo-springboot', author='blueriver', lang='Java', watch='34', star='62', fork='27', url='https://git.oschina.net/blueriver/weixin-dubbo-springboot'}\nProject{name='网页宠物插件', author='lt1726', lang='JavaScript', watch='10', star='16', fork='5', url='https://git.oschina.net/lutao1726/WeiChunCaiChaJian'}\nProject{name='flyray-base', author='boleixiongdi', lang='Java', watch='117', star='210', fork='102', url='https://git.oschina.net/boleixiongdi/flyray'}\nProject{name='NextMQTT', author='陈永佳', lang='Java', watch='9', star='16', fork='3', url='https://git.oschina.net/yoojia/NextMQTT'}\nProject{name='live-chat在线聊天室', author='furioussoul', lang='NodeJS', watch='12', star='18', fork='3', url='https://git.oschina.net/65465498/live-chat'}\nProject{name='BMS', author='haibing871802', lang='Java', watch='72', star='149', fork='59', url='https://git.oschina.net/imsroot/BMS'}\nProject{name='NextInput-Android', author='陈永佳', lang='Android', watch='4', star='22', fork='2', url='https://git.oschina.net/yoojia/NextInput-Android'}\nProject{name='mqttclient', author='JesusSlim', lang='PHP', watch='2', star='3', fork='1', url='https://git.oschina.net/JesusSlim/mqttclient'}\nProject{name='jquery-webos-win10', author='菩提树下杨过', lang='JavaScript', watch='13', star='28', fork='9', url='https://git.oschina.net/bodhiyg/jquery-webos-win10'}\nProject{name='dp-LTE', author='小林攻城狮', lang='Java', watch='90', star='197', fork='57', url='https://git.oschina.net/zhocuhenglin/dp-security'}\nProject{name='2048_cli', author='(._.)码农BTS', lang='C', watch='7', star='14', fork='2', url='https://git.oschina.net/coder-bts/2048_cli'}\nProject{name='OPEN_CTP_X', author='量化交易', lang='Android', watch='9', star='4', fork='1', url='https://git.oschina.net/openctp/open_ctp_x'}\nProject{name='typeofit', author='前端巨浪', lang='JavaScript', watch='6', star='6', fork='1', url='https://git.oschina.net/yaohaixiao/typeofit'}\nProject{name='goreporter', author='wgliang', lang='Go', watch='5', star='5', fork='6', url='https://git.oschina.net/wgliang/goreporter'}\nProject{name='athena-support', author='这里的名字只能十个字', lang='Java', watch='27', star='34', fork='11', url='https://git.oschina.net/opdar/athena-support'}\nProject{name='eweapp', author='tumobi', lang='JavaScript', watch='16', star='36', fork='8', url='https://git.oschina.net/tumobi/eweapp'}\nProject{name='swoole-worker', author='花花世界欢乐多', lang='PHP', watch='10', star='13', fork='1', url='https://git.oschina.net/FEIGE/swoole-worker'}\nProject{name='wakew-news', author='憧憬Licoy', lang='Java', watch='16', star='43', fork='12', url='https://git.oschina.net/licoy/wakew-news'}\nProject{name='ILog CMS', author='duzhi', lang='Java', watch='29', star='57', fork='37', url='https://git.oschina.net/duzhime/DUZHI_BLOG'}\nProject{name='EasyReport', author='hacken', lang='Java', watch='47', star='100', fork='31', url='https://git.oschina.net/yunzhi/EasyReport'}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flicoy%2Fjava-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flicoy%2Fjava-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flicoy%2Fjava-crawler/lists"}