{"id":20560281,"url":"https://github.com/programming-with-love/skyeyesystem","last_synced_at":"2025-09-25T21:31:33.115Z","repository":{"id":175489938,"uuid":"653970283","full_name":"Programming-With-Love/SkyEyeSystem","owner":"Programming-With-Love","description":"天眼系统，每隔十分钟爬取各个平台的热搜数据并入库。包括原始热搜数据存入mysql。词频统计存入Redis。","archived":false,"fork":false,"pushed_at":"2023-06-21T01:54:25.000Z","size":173,"stargazers_count":1,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-11-16T03:54:01.253Z","etag":null,"topics":["crawler","mysql","redis","skyeye","skyeyewall","springboot"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Programming-With-Love.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-15T06:07:36.000Z","updated_at":"2023-11-29T08:41:08.000Z","dependencies_parsed_at":null,"dependency_job_id":"af56a172-cba8-4cda-9396-594af0b9c119","html_url":"https://github.com/Programming-With-Love/SkyEyeSystem","commit_stats":null,"previous_names":["shaoxiongdu/skyeyesystem","programming-with-love/skyeyesystem"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Programming-With-Love%2FSkyEyeSystem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Programming-With-Love%2FSkyEyeSystem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Programming-With-Love%2FSkyEyeSystem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Programming-With-Love%2FSkyEyeSystem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Programming-With-Love","download_url":"https://codeload.github.com/Programming-With-Love/SkyEyeSystem/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234255602,"owners_count":18803691,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","mysql","redis","skyeye","skyeyewall","springboot"],"created_at":"2024-11-16T03:54:06.661Z","updated_at":"2025-09-25T21:31:27.804Z","avatar_url":"https://github.com/Programming-With-Love.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch3 align=\"center\"\u003eSkyEyeSystem\u003c/h3\u003e\n\n  \u003cp align=\"center\"\u003e\n     一个基于SpringBoot的全网热点爬虫项目\n    \u003cbr /\u003e\n    \u003ca href=\"./README.md\"\u003e中文\u003c/a\u003e\n    ·\n    \u003ca href=\"./README_en.md\"\u003eEnglish\u003c/a\u003e\n  \u003c/p\u003e\n\u003cdetails open=\"open\"\u003e\n  \u003csummary\u003e目录\u003c/summary\u003e\n  \u003col\u003e\n    \u003cli\u003e\n      \u003ca href=\"#关于项目\"\u003e关于项目\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\n      \u003ca href=\"#快速启动\"\u003e快速启动\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#先决条件\"\u003e先决条件\u003c/a\u003e\u003c/li\u003e\n        \u003cli\u003e\u003ca href=\"#安装\"\u003e安装\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#使用\"\u003e使用\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#贡献\"\u003e贡献\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#许可证\"\u003e许可证\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#联系\"\u003e联系\u003c/a\u003e\u003c/li\u003e\n  \u003c/ol\u003e\n\u003c/details\u003e\n\n## 关于项目\n\n每天下午三点定时爬取全网热搜数据。包括\n\n- 微博热搜\n- B站热搜\n- CSDN热搜\n- 知乎热搜\n- 今日头条\n- 百度热搜\n\n爬取数据之后\n\n1. 会将原始数据存入MySQL。\n2. 进行词频统计 存入Redis。\n\n## 快速启动\n\n此处说明了如何快速的使用本项目\n\n### 先决条件\n\n确保您的安装器是Maven\n\n### 安装\n\n1. maven sync\n2. 执行SQL脚本 [SQL脚本](src/main/resources/db/ddl.sql)\n3. application中配置您的数据库地址\n4. db/config中配置redis地址。\n5. 启动即可。\n\n## 使用\n\n#### 1. 手动执行爬虫操作\n\n执行[HotSpotCrawlerTest.java](src/test/java/cn/shoxiongdu/SkyEyeSystem/task/hotspot/crawl/BiliBiliCrawlerTest.java)\n\n#### 2. 配置爬虫的执行时间\n\n修改[爬虫任务](src/main/java/cn/shoxiongdu/SkyEyeSystem/task/hotspot/crawl/CrawlerTask.java)中的注解值即可。\n接收标准的CRON参数。 可使用[Cron在线表达式生成器](http://cron.ciding.cc/) 在线生成\n\n```java\n@Scheduled(cron = \"0 0 15 * * ?\") // 每天下午3点执行\npublic void crawl(){\n        ...\n        }\n```\n\n## 贡献\n\n贡献使开源社区成为一个学习、激励和创造的绝佳场所。非常感谢您所做的任何贡献。\n\n1.fork项目\n\n2.创建功能分支（`git checkout-b Feature/AamazingFeature`）\n\n3.提交更改（`git-Commit-m'添加一些AmazingFeature`）\n\n4.推送至分支（`git Push origin feature/AamazingFeature`）\n\n5.打开拉取请求\n\n## 许可证\n\n基于MIT的许可证分发，传输请遵循相关开源协议: [MIT许可证](LICENSE )\n\n## 联系\n\n- 杜少雄 email@shaoxiongdu.cn\n- 微信: 15603430511\n- 个人博客: https://shaoxiongdu.cn\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprogramming-with-love%2Fskyeyesystem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprogramming-with-love%2Fskyeyesystem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprogramming-with-love%2Fskyeyesystem/lists"}