{"id":22848301,"url":"https://github.com/hellogithub-team/hydra","last_synced_at":"2025-09-07T03:35:00.805Z","repository":{"id":41087320,"uuid":"196114921","full_name":"HelloGitHub-Team/Hydra","owner":"HelloGitHub-Team","description":"简单但绝不简陋的 Python3 爬虫项目，集合 HelloGitHub 所有平台的账号信息。会基础的 Python 语法就能看懂！","archived":false,"fork":false,"pushed_at":"2023-03-21T22:49:45.000Z","size":735,"stargazers_count":203,"open_issues_count":1,"forks_count":40,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-13T10:50:25.109Z","etag":null,"topics":["python","spider"],"latest_commit_sha":null,"homepage":"https://hellogithub.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HelloGitHub-Team.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-10T02:20:51.000Z","updated_at":"2025-04-07T17:57:27.000Z","dependencies_parsed_at":"2025-01-10T12:41:52.621Z","dependency_job_id":"b1534284-ce08-44c1-b2d9-6b5238dadc58","html_url":"https://github.com/HelloGitHub-Team/Hydra","commit_stats":{"total_commits":36,"total_committers":3,"mean_commits":12.0,"dds":0.08333333333333337,"last_synced_commit":"15fb6ff20ed1ec3fa41dfb85d7a70499ea696d0c"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HelloGitHub-Team/Hydra","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelloGitHub-Team%2FHydra","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelloGitHub-Team%2FHydra/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelloGitHub-Team%2FHydra/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelloGitHub-Team%2FHydra/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HelloGitHub-Team","download_url":"https://codeload.github.com/HelloGitHub-Team/Hydra/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HelloGitHub-Team%2FHydra/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273992863,"owners_count":25203790,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","spider"],"created_at":"2024-12-13T04:11:17.253Z","updated_at":"2025-09-07T03:35:00.757Z","avatar_url":"https://github.com/HelloGitHub-Team.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hydra（九头蛇）\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"75%\" src=\"https://cdn.jsdelivr.net/gh/HelloGitHub-Team/Hydra@main/doc/cover.png\"/\u003e\n  \u003cbr\u003e\u003cimg alt=\"GitHub Workflow Status\" src=\"https://img.shields.io/github/workflow/status/HelloGitHub-Team/Hydra/hydra?style=flat-square\"\u003e\n  \u003cimg alt=\"Codacy grade\" src=\"https://img.shields.io/codacy/grade/e11ce1e341554699ad523dbfe75be9c6?style=flat-square\"\u003e\n  \u003cimg alt=\"Codacy coverage\" src=\"https://img.shields.io/codacy/coverage/e11ce1e341554699ad523dbfe75be9c6?style=flat-square\"\u003e\n  \u003cbr\u003e\u003cstrong\u003e简单但绝不简陋的 Python3 爬虫项目。\u003c/strong\u003e\n  \u003cbr\u003e参考\u003ca href=\"https://mp.weixin.qq.com/s/K4RGr5NqMFAUKtB0KFPV5g\"\u003e「建立完美的 Python 项目」\u003c/a\u003e 创建 \n\u003c/p\u003e\n\nHydra 力求用最简单的代码实现聚合 HG 多平台的数据。\n\n从本项目中你可以看到：**熟悉的 [Python 基础语法](https://github.com/521xueweihan/python)**、**如何编写爬虫**、**操作数据库**、**常用第三库**、**分析网页**、**解析接口**、**编写单元测试**、**mock 请求**、**异常监控和管理**、**保证代码质量的自动化**、**GitHub Action** 等实战应用。\n\n此项目是汇集「HelloGitHub」在每个平台的账号和内容数据，方便我们的作者们看到自己作品的数据（[投稿吗？](https://www.yuque.com/docs/share/bf781d29-cc94-44dd-b226-9d70fa38fa1c)）。支持平台：[博客园](https://www.cnblogs.com/xueweihan/)、[头条](https://www.toutiao.com/c/user/token/MS4wLjABAAAAigrrKo-3rjLpxaH4Go3BrZRqHTIhLW3e30QjfFXgzNQ/)、[知乎](https://www.zhihu.com/people/xueweihan)、[掘金](https://juejin.cn/user/1574156384091320)、[即刻](https://web.okjike.com/u/ff31a838-6eb9-440d-9970-dabc5b2c0309) 等。\n\n你要[加入](https://mp.weixin.qq.com/s/9FUQ2i0HbemwfIj9sa1p0A)我们吗？\n\n## 一、运行\n\n\u003e 基于 Python 3.9.1 实现，理论上支持 3.7.5+\n\n首先，下载项目：`git clone` or [点击下载 zip 包](https://github.com/HelloGitHub-Team/Hydra/archive/main.zip)\n\n然后，在项目根目录创建配置文件，[.local_env.yaml](/doc/local_env.yaml)。\n\n最后，把玩起来吧！\n\n1. 安装 poetry：`pip install poetry`\n\n2. 安装依赖：在项目根目录执行 `poetry install --no-root`\n\n3. 运行单个爬虫：`poetry run python main.py wechat|cnblogs|toutiao|csdn|zhihu|juejin|jike`\n\n运行遇到问题和更多说明[点这里](/doc/install.md)，贡献代码[看这里](/doc/contribute.md)\n\n## 二、展示\n比如：查看某一日发布的原创文章数据\n\n```\nSELECT\n\tsummary ,\n\tclicks_count ,\n\tplatform ,\n\tpublish_date\nFROM\n\thydra_content\nWHERE\n\tcontent_type = \"article\"\nAND publish_date = \"2021-03-01\"\nAND(\n\tis_original = 1\n\tOR is_original IS NULL\n);\n```\n\n```\n+-----------------------------------------+----------------+------------+----------------+\n| summary                                 |   clicks_count | platform   | publish_date   |\n|-----------------------------------------+----------------+------------+----------------|\n| 更新啦！第 59 期《HelloGitHub》开源月刊 |             77 | csdn       | 2021-03-01     |\n| 更新啦！第 59 期《HelloGitHub》月刊     |           5133 | wechat     | 2021-03-01     |\n| 更新啦！第 59 期《HelloGitHub》开源月刊 |           1022 | cnblogs    | 2021-03-01     |\n| 更新啦！第 59 期《HelloGitHub》开源月刊 |           1053 | toutiao    | 2021-03-01     |\n| 更新啦！第 59 期《HelloGitHub》开源月刊 |           1879 | zhihu      | 2021-03-01     |\n| 更新啦！第 59 期《HelloGitHub》开源月刊 |            931 | juejin     | 2021-03-01     |\n+-----------------------------------------+----------------+------------+----------------+\n6 rows in set\nTime: 0.050s\n```\n\n## 三、声明\n\u003ca rel=\"license\" href=\"https://creativecommons.org/licenses/by-nc-nd/4.0/deed.zh\"\u003e\u003cimg alt=\"知识共享许可协议\" style=\"border-width: 0\" src=\"https://licensebuttons.net/l/by-nc-nd/4.0/88x31.png\"\u003e\u003c/a\u003e\u003cbr\u003e本作品采用 \u003ca rel=\"license\" href=\"https://creativecommons.org/licenses/by-nc-nd/4.0/deed.zh\"\u003e署名-非商业性使用-禁止演绎 4.0 国际\u003c/a\u003e 进行许可。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhellogithub-team%2Fhydra","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhellogithub-team%2Fhydra","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhellogithub-team%2Fhydra/lists"}