{"id":19151886,"url":"https://github.com/azcodingaccount/ischool","last_synced_at":"2025-05-07T05:42:12.334Z","repository":{"id":233762189,"uuid":"784773842","full_name":"AZCodingAccount/iSchool","owner":"AZCodingAccount","description":"智搜通—大数据量下的综合性教务公告检索平台","archived":false,"fork":false,"pushed_at":"2024-10-31T07:22:52.000Z","size":13966,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-19T19:05:06.561Z","etag":null,"topics":["elasticsearch","flask","rabbitmq","redis","spider","springai","springcloud","springcloudalibaba","xxl-job"],"latest_commit_sha":null,"homepage":"http://search.bugdesigner.cn","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AZCodingAccount.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-04-10T14:27:41.000Z","updated_at":"2024-10-31T07:22:56.000Z","dependencies_parsed_at":"2024-04-17T10:02:17.853Z","dependency_job_id":"1bde3dcb-3adf-48e5-bd52-177485b40bef","html_url":"https://github.com/AZCodingAccount/iSchool","commit_stats":null,"previous_names":["azcodingaccount/ischool"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AZCodingAccount%2FiSchool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AZCodingAccount%2FiSchool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AZCodingAccount%2FiSchool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AZCodingAccount%2FiSchool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AZCodingAccount","download_url":"https://codeload.github.com/AZCodingAccount/iSchool/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252823666,"owners_count":21809707,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","flask","rabbitmq","redis","spider","springai","springcloud","springcloudalibaba","xxl-job"],"created_at":"2024-11-09T08:15:54.759Z","updated_at":"2025-05-07T05:42:12.295Z","avatar_url":"https://github.com/AZCodingAccount.png","language":"Java","readme":"# 大数据量下的教务公告聚合搜索平台\n\n## 需求分析❓\n\n​\t\t大学🎓系统**性能差**、**可扩展性低**、**易用性差**已是共识。网站应该回归到为用户服务的本质，在此背景下，教务公告检索系统应运而生。本系统解决师生对于教务公告检索的需求，实现**高效检索🔍**、**AI问答🤖**、**用户点评💭**等一站式功能。下面给出功能模块图如下\n\n\u003cimg src=\"https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20240522153811319.png\" alt=\"image-20240522153811319\" style=\"zoom:50%;\" /\u003e\n\n项目后端所用技术架构概览👀\n\n![image-20240613104609407](https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20240613104609407.png)\n\n## 技术选型🛠️\n\n**前端：**\n\nVue3框架、Element-Plus组件库\n\n**后端：**\n\n开发框架：Spring Cloud2023、Spring Cloud Alibaba2023、SpringBoot3.2.4\n\n数据库：Redis、ElasticSearch8.11、MySQL\n\n中间件：RabbitMQ消息队列、OpenFeign远程调用、分布式任务调度框架 xxl-job\n\n其他：SpringAI框架调用OpenAI生成式AI接口、Python爬虫技术🕷️\n\n## 项目亮点✨\n\n1. SpringCloud Gateway网关实现`用户鉴权`、`跨域解决`、`接口文档聚合`\n\n2. Elastic Search+Redis实现高效检索，10w级数据量检索，从传统数据库查询**大于30s**优化到—\u003e命中缓存情况下**小于100ms**，不命中缓存**小于1s**。\n\n3. xxl-job定时任务结合python爬虫实现`教务网站数据库数据`、`MySQL`、`Redis`、`Elastic Search`四者的增量和全量同步\n\n4. RabbitMQ进行AI生成建议的消息调用，实现**异步解耦**。使用SpringAI框架封装的开箱即用的OpenAI接口，实现**进一步聊聊**功能。\n\n## 文档📚\n\n- 查看[开发文档](https://github.com/AZCodingAccount/iSchool/blob/main/开发文档.md)了解项目开发与优化过程\n- 查看[接口文档](https://github.com/AZCodingAccount/iSchool/tree/main/接口文档)了解项目接口设计\n\n## 贡献🤝\n\n项目欢迎任何形式的贡献\n\n- 提出[issue](https://github.com/AZCodingAccount/iSchool/issues)报告 bug 或要求接入新系统\n- 提交[PR](https://github.com/AZCodingAccount/iSchool/pulls)帮助完善项目\n\n## 查询接口性能测试\n\n工具：jmeter、apipost\n\n​\t\t使用redis缓存在用户同时搜索一个关键词时，测试报告如下：\n\n硬件配置：4h8g服务器，es堆内存配置1G\n\n​\t\t本地测试结果表明，在3000并发时，前99.99%的用户响应时间可以控制在1.2s以内。但在线上环境时，只在500并发时，时间在可以容忍的范围内(200并发时时间小于1s)，分析原因是因为**服务器硬件配置不足**、**服务器网络上行不足**、**网络传输延迟**等原因。\n\n解决方案可以**加机器**、**加配置**。目前项目瓶颈在服务器而非数据库这边\n\n1. 本地测试，时间稳定在1s以下，在3000并发时，错误率明显上升。\n\n\u003cimg src=\"https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20241020221204804.png\" alt=\"image-20241020221204804\" style=\"width:67%;\" /\u003e\n\n不使用缓存，3000并发下，本地QPS和响应时间下降4倍左右，由此可见，虽然es也有缓存，但是redis缓存也是有一定必要的\n\n\u003cimg src=\"https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20241020222542994.png\" alt=\"image-20241020222542994\" style=\"width:67%;\" /\u003e\n\n1. 在无可用线程时，连接会直接拒绝，解决方案：\n   1. 增加最大线程数（默认是100）\n   2. 加机器\n   3. 改变数据库连接池的大小（默认几十），优化逻辑，每个请求执行快一点（这里已经到顶了）\n2. 4h8g服务器，给的es堆内存只有1G。\n\n100并发下，QPS达到了80，平均响应时间在100ms左右\n\n\u003cimg src=\"https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20241020214822618.png\" alt=\"image-20241020214822618\" width=66% /\u003e\n\n​    500并发下，QPS在40左右徘徊，等待最后几个请求时出现明显卡顿，平均响应时间增加到了4s\n\n\u003cimg src=\"https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20241020215109256.png\" alt=\"image-20241020215109256\" style=\"width:66%;\" /\u003e\n\n​    1000并发下，与500个用户类似，起初请求QPS在40左右，处理后续请求有明显卡顿，平均响应时间增加到了11s\n\n\u003cimg src=\"https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20241020215539764.png\" alt=\"image-20241020215539764\" style=\"width:66%;\" /\u003e\n\n2000并发下，QPS15，平均响应时间20s，**第一次出现error**\n\n\u003cimg src=\"https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20241020220259655.png\" alt=\"image-20241020220259655\" style=\"width:67%;\" /\u003e\n\n增加到4000以后，错误率明显上升达到30%，错误信息为连接超时\n\n\u003cimg src=\"https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20241020220606635.png\" alt=\"image-20241020220606635\" style=\"width:67%;\" /\u003e\n\n## 快速启动与项目概览 🚀\n\n由于后端涉及到中间件较多，请观看B站视频启动项目与预览项目：[iSchool项目介绍](https://space.bilibili.com/501122856)\n\n项目预览如下：\n\n搜索页\n\n![image-20240719203443727](https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20240719203443727.png)\n\nAI问答页\n\n![image-20240719203531956](https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20240719203531956.png)\n\n社区模块页\n\n![image-20240719203907039](https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20240719203907039.png)\n\n评论页\n\n![image-20240719203941258](https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20240719203941258.png)\n\n个人中心页\n\n![image-20240719203811664](https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/image-20240719203811664.png)\n\n欢迎页\n\n![img](https://my-picture-bed1-1321100201.cos.ap-beijing.myqcloud.com/mypictures/clip_image002.jpg)\n\n\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazcodingaccount%2Fischool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fazcodingaccount%2Fischool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazcodingaccount%2Fischool/lists"}