{"id":13671075,"url":"https://github.com/xingag/spider_python","last_synced_at":"2025-04-27T14:33:05.674Z","repository":{"id":34338563,"uuid":"148454366","full_name":"xingag/spider_python","owner":"xingag","description":"python爬虫","archived":false,"fork":false,"pushed_at":"2023-12-31T04:49:40.000Z","size":3795,"stargazers_count":979,"open_issues_count":15,"forks_count":447,"subscribers_count":33,"default_branch":"master","last_synced_at":"2024-11-11T08:43:50.459Z","etag":null,"topics":["bs4","python","python3","requests","scrapy","urllib","xpath"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xingag.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-12T09:20:54.000Z","updated_at":"2024-11-11T06:52:22.000Z","dependencies_parsed_at":"2024-11-11T08:33:22.485Z","dependency_job_id":"b56a597e-a02c-48fb-90bf-7a2b275e4df2","html_url":"https://github.com/xingag/spider_python","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xingag%2Fspider_python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xingag%2Fspider_python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xingag%2Fspider_python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xingag%2Fspider_python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xingag","download_url":"https://codeload.github.com/xingag/spider_python/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251154279,"owners_count":21544471,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bs4","python","python3","requests","scrapy","urllib","xpath"],"created_at":"2024-08-02T09:00:58.136Z","updated_at":"2025-04-27T14:33:03.346Z","avatar_url":"https://github.com/xingag.png","language":"Python","readme":"# spider_python\n\n## 前言\n\n如果想查看详细的教程，请关注微信公众号：**AirPython**\n\n![](./raw/qr.jpeg)\n\n\n\n## 普通的爬虫\n\n* [爬取电影天堂最新的电影数据 - xpath](./spiders/spider_dytt.py)\n\n* [爬取腾讯招聘的职位数据 - xpath](./spiders/spider_tencent_recruit.py)\n\n* [爬取中国天气网全国天气并生成饼状图 - bs4](./spiders/spider_china_weather.py)\n\n* [爬取古诗词网的数据 - re](./spiders/spider_gushiwen.py)\n\n* [爬取糗事百科上的段子数据 - re](./spiders/spider_qiu_shi_bai_ke.py)\n\n\n\n## 多线程爬虫\n\n* [多线程爬取斗图吧的表情图并下载到本地 - xpath + threading](./spiders/spider_dou_tu_la.py)\n* [使用 itchat 发送表情到指定的人和微信群](./spiders/发表情/)\n* [多线程爬取百思不得姐的文字和图片信息并写入到csv中](./spiders/spider_bai_si_bu_de_jie.py)\n\n\n\n## Selenium 自动化爬虫\n\n* [爬取拉勾网的职位信息 - selenium + requests + lxml ](./spiders/spider_lagou.py)\n\n* [爬取 Boss 直聘网的职位信息 - selenium + lxml](./spiders/spider_boss.py)\n\n\n\n## Scrapy 框架爬虫\n* [爬取糗事百科的段子保存到 JSON 文件中](./scrapy/qsbk/readme.MD)\n* [爬取微信小程序论坛的数据](./scrapy/weixin_community/readme.MD)\n* [登录豆瓣网并修改个性签名](./scrapy/douban_login/readme.MD)\n* [下载汽车之家的高清图片到本地](./scrapy/qczj/readme.MD)\n* [爬取简书网所有文章数据](./scrapy/jianshu_spider/)\n* [爬取房天下所有房的数据，包含新房、二手房](./scrapy/sfw_spider)\n\n\n\n\n\n## feapder\n\n* [feapder AirSpider实例](./feapder/tophub_demo)\n\n\n\n## Node.js 爬虫\n\n* [使用 puppeteer 爬取简书文章并保存到本地](./js/jian_shu.js)\n\n  \n\n## 其他\n\n* [使用 Python 定位到女朋友的位置](./获取女友的位置)\n* [女朋友背着我，用 Python 偷偷隐藏了她的行踪](./ModifyLocation)\n* [微信群聊记录](./微信聊天记录)\n* [Python 调用 JAR](./Python调用JAR)\n\n","funding_links":[],"categories":["WebGL"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxingag%2Fspider_python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxingag%2Fspider_python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxingag%2Fspider_python/lists"}