{"id":20840856,"url":"https://github.com/kangvcar/awsomespider","last_synced_at":"2025-05-08T22:03:51.693Z","repository":{"id":40677198,"uuid":"103138073","full_name":"kangvcar/AwsomeSpider","owner":"kangvcar","description":"Python爬虫小项目汇总（招聘信息/电影信息/股票信息/天气信息/贴吧信息/图片信息/视频信息..）","archived":false,"fork":false,"pushed_at":"2023-02-15T21:35:09.000Z","size":17399,"stargazers_count":63,"open_issues_count":7,"forks_count":26,"subscribers_count":2,"default_branch":"master","last_synced_at":"2023-03-04T22:37:20.570Z","etag":null,"topics":["beautifulsoup","lxml","pymysql","pyspider","python","scrapy","selenium","spider","urllib2"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kangvcar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-09-11T13:17:01.000Z","updated_at":"2022-12-11T07:02:03.000Z","dependencies_parsed_at":"2023-01-24T09:30:17.126Z","dependency_job_id":null,"html_url":"https://github.com/kangvcar/AwsomeSpider","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kangvcar%2FAwsomeSpider","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kangvcar%2FAwsomeSpider/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kangvcar%2FAwsomeSpider/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kangvcar%2FAwsomeSpider/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kangvcar","download_url":"https://codeload.github.com/kangvcar/AwsomeSpider/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225110416,"owners_count":17422420,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beautifulsoup","lxml","pymysql","pyspider","python","scrapy","selenium","spider","urllib2"],"created_at":"2024-11-18T01:18:05.216Z","updated_at":"2024-11-18T01:18:05.912Z","avatar_url":"https://github.com/kangvcar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pyproject\n\n\u003e 2017 python爬虫学习\n\n\n\n## Usage_pdb.txt\nPdb: The Python Debugger \u003c/br\u003e\nPython调试器 使用方法快速入门\n\n\n\n## NumGame Folders\n### -NumGame1.py\n简单的猜数字的大小\n\n### -NumGame2.py\n多玩家猜数字游戏\n\n### -NumGame5.py\n游戏规则如下：\u003cbr\u003e\n主持人确定猜数字的范围和pk次数\u003cbr\u003e\n每位参赛者按顺序输入自己的名字\u003cbr\u003e\n主持人确定每次可以猜的次数（默认4次）\u003cbr\u003e\n开始比赛-每位参赛者按顺序猜一次，如果猜对加一分\u003cbr\u003e\n如果在规定次数大家都没有猜出来，打印出答案\u003cbr\u003e\n比赛结束后打印排行榜\u003cbr\u003e\n\n### -object.py\n面向对象编程练习\n\n### -Youdao-reptile.py\n使用 urllib2 爬取有道翻译的源代码并写入Youdao.txt文件\n\n\n\n## project Folders\n\u003e Scrapy 爬虫框架目录结构\n```\nproject/\n    scrapy.cfg \t\t\t#项目的配置文件\n    project/ \t\t\t#该项目的python模块,之后您将在此加入代码\n        __init__.py \t\n        items.py \t\t#项目中的item文件\n        pipelines.py \t#项目中的pipelines文件\n        settings.py \t#项目的设置文件\n        spiders/ \t\t#放置spider代码的目录\n            __init__.py\n            ...\n```\n### -project/spiders/suzhouSpider.py\n爬取www.suzhou.tianqi.com 页面的六天天气,并写入result.txt文件\n\n\n\n## Spiders Folders\n\u003e python 爬虫项目\n### - Spiders/Spider_jisilu.py\n使用 selenium 爬取www.jisilu.com 页面的部分数据并写入文件\n### - Spiders/Spider_jisilu_2.py\n基于Spiders/Spider_jisilu.py, 把各部分改成def函数\n### - Spiders/Spider_jisilu_3.py\n基于Spiders/Spider_jisilu.py, 增加了标题的输出\n### - Spiders/Spider_jisilu_4.py\n基于Spiders/Spider_jisilu.py, 使用面向对象的方法重构代码\n### - Spiders/Spider_qiushibaike.py\n爬取 www.qiushibaike.com 页面的段子，并实现回车查看段子，按Q退出\n### - Spiders/Spider_SZtianqi.py\n使用 BeautifulSoup 爬取 www.suzhou.tianqi.com 页面的数据，并用BeautifulSoup过滤\n### - Spiders/Spider_tieba.py\n爬取 tieba.baidu.com 页面的帖子并写入文件，使用面向对象编程\n\n\n\n## training Folders\n\u003e 各爬虫工具模块的快速入门学习, 实例验证\n### - training/Usage_BeautifulSoup.py\nBeautifulSoup 快速入门，实例学习\n### - training/Usage_BeautifulSoup_CSS_Select.py\nCSS_Select =\u003e CSS选择器 快速入门，实例学习\n### - training/Usage_BeautifulSoup_find_.py\nBeautifulSoup 的 find_all 等函数 快速入门，实例学习\n### - training/Usage_PyQuery.py\nPyQuery 快速入门，实例学习\n### - training/Usage_Requests.py\nRequests 快速入门，实例学习\n### - training/Usage_Selenium.py\nSelenium 快速入门，实例学习\n### - training/Usage_Urllib2.py\nUrllib2 快速入门，实例学习\n### - training/Usage_XPath.py\nXPath 快速入门，实例学习\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkangvcar%2Fawsomespider","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkangvcar%2Fawsomespider","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkangvcar%2Fawsomespider/lists"}