{"id":19794621,"url":"https://github.com/threadnew/encycproject","last_synced_at":"2025-10-13T13:38:10.158Z","repository":{"id":43585356,"uuid":"192483402","full_name":"ThreadNew/EncycProject","owner":"ThreadNew","description":"一个爬取宠物百科内容的后台demo。","archived":false,"fork":false,"pushed_at":"2022-12-10T20:23:50.000Z","size":9759,"stargazers_count":0,"open_issues_count":15,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-11T04:44:16.395Z","etag":null,"topics":["solr","springboot-mybatis","springboot2","vue-router","vuejs2","webmagic"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ThreadNew.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-18T06:55:30.000Z","updated_at":"2019-06-27T12:40:11.000Z","dependencies_parsed_at":"2023-01-26T09:00:31.984Z","dependency_job_id":null,"html_url":"https://github.com/ThreadNew/EncycProject","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreadNew%2FEncycProject","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreadNew%2FEncycProject/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreadNew%2FEncycProject/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreadNew%2FEncycProject/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ThreadNew","download_url":"https://codeload.github.com/ThreadNew/EncycProject/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241136851,"owners_count":19916001,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["solr","springboot-mybatis","springboot2","vue-router","vuejs2","webmagic"],"created_at":"2024-11-12T07:13:44.446Z","updated_at":"2025-10-13T13:38:05.124Z","avatar_url":"https://github.com/ThreadNew.png","language":"JavaScript","readme":"\n\u003cimg src=\"resource/img/plogo.png\" height=\"120px\"\u003e\u003c/img\u003e\n# EncycProject\n\u003e EncycProject项目使用现在流行的前后端分离技术；此项目创作目的是实践。利用此项目快速搭建环境和快速入门。该项目的主线是建立宠物社区网站。目前该项目完成第一个模块：宠物百科；剩余模块陆续开发中。（此项目可以变更，第一模块耦合度低）\n---\n### 项目模块\n- **宠物百科**：利用爬虫爬取相关数据，建立中心库，搜索引擎使用的solr7  \n    -  **模块的项目演示**\n    \u003e如下图 [图示链接](resource/img/project.gif)\n    \u003cimg src=\"resource/img/project.gif\"/\u003e\n\n- 敬请期待。。。\n---\n## 1 项目的体系结构\n### 1.1 后端\n\u003e如下图所示     \n```\nD:.\n\n│  pom.xml\n│\n│          \n├─src\n│  ├─main\n│  │  ├─java\n│  │  │  └─com\n│  │  │      └─threadnew\n│  │  │          │  EncycprojectApplication.java\n│  │  │          │  \n│  │  │          ├─controller\n│  │  │          │      EncycController.java ：宠物百科的控制器\n│  │  │          │      index.java\n│  │  │          │      \n│  │  │          ├─dao\n│  │  │          │      EncycDao.java\n│  │  │          │      \n│  │  │          ├─pojo\n│  │  │          │      Encyc.java\n│  │  │          │      EncycResult.java\n│  │  │          │      EncycSolr.java\n│  │  │          │      Page.java\n│  │  │          │      \n│  │  │          ├─service\n│  │  │          │      EncycService.java\n│  │  │          │      \n│  │  │          ├─solr  ：sorl的实现类\n│  │  │          │      SolrUtil.java\n│  │  │          │      \n│  │  │          └─webmagic : 有关爬虫的实现类\n│  │  │                  EncycPageProcessor.java\n│  │  │                  EncycPipeline.java\n│  │  │                  SpikeFileCacheQueueScheduler.java\n│  │  │                  \n│  │  └─resources\n│  │      │  application.yml\n│  │      │  \n│  │      ├─static\n│  │      └─templates\n│  └─test\n│      └─java\n│          └─com\n│              └─threadnew\n│                      EncycprojectApplicationTests.java\n│                      \n└\n```\n\u003e\u003e其中 **webmagic**目录下的文件是关于爬虫的实现；**solr** 目录下的文件是关于solr的实现。   \n ### 1.2 前端    \n \u003e如下图所示    \n\n\u003cimg src=\"resource/img/front.jpg\"/\u003e  \n\n\u003e 其中上图红色标记此处要在开发环境中配置代理，否则跨域无法访问。\n## 2 技术选型  \n### 2.1 后台\n- springboot2\n- Mybatis\n- solr\n- webMagic（Java的爬虫框架）\n- druid\n### 2.2 前台（待续）\n- vue2.js\n- webPack\n- vue-router\n- axios\n- vant （有赞的前端的ui框架）\n---\n## 3 项目搭建\n### 3.1 框架整合\n#### 3.1.1 pom.xml\n```\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cproject xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n         xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd\"\u003e\n    \u003cmodelVersion\u003e4.0.0\u003c/modelVersion\u003e\n    \u003cparent\u003e\n        \u003cgroupId\u003eorg.springframework.boot\u003c/groupId\u003e\n        \u003cartifactId\u003espring-boot-starter-parent\u003c/artifactId\u003e\n        \u003cversion\u003e2.1.5.RELEASE\u003c/version\u003e\n        \u003crelativePath/\u003e \u003c!-- lookup parent from repository --\u003e\n    \u003c/parent\u003e\n    \u003cgroupId\u003ecom.threadnew\u003c/groupId\u003e\n    \u003cartifactId\u003eencycproject\u003c/artifactId\u003e\n    \u003cversion\u003e0.0.1-SNAPSHOT\u003c/version\u003e\n    \u003cname\u003eencycproject\u003c/name\u003e\n    \u003cdescription\u003eDemo project for Spring Boot\u003c/description\u003e\n\n    \u003cproperties\u003e\n        \u003cjava.version\u003e1.8\u003c/java.version\u003e\n    \u003c/properties\u003e\n\n    \u003cdependencies\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.springframework.boot\u003c/groupId\u003e\n            \u003cartifactId\u003espring-boot-starter-web\u003c/artifactId\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.mybatis.spring.boot\u003c/groupId\u003e\n            \u003cartifactId\u003emybatis-spring-boot-starter\u003c/artifactId\u003e\n            \u003cversion\u003e2.0.1\u003c/version\u003e\n        \u003c/dependency\u003e\n\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.springframework.boot\u003c/groupId\u003e\n            \u003cartifactId\u003espring-boot-devtools\u003c/artifactId\u003e\n            \u003cscope\u003eruntime\u003c/scope\u003e\n            \u003coptional\u003etrue\u003c/optional\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003emysql\u003c/groupId\u003e\n            \u003cartifactId\u003emysql-connector-java\u003c/artifactId\u003e\n            \u003cscope\u003eruntime\u003c/scope\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.springframework.boot\u003c/groupId\u003e\n            \u003cartifactId\u003espring-boot-configuration-processor\u003c/artifactId\u003e\n            \u003coptional\u003etrue\u003c/optional\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.projectlombok\u003c/groupId\u003e\n            \u003cartifactId\u003elombok\u003c/artifactId\u003e\n            \u003coptional\u003etrue\u003c/optional\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.springframework.boot\u003c/groupId\u003e\n            \u003cartifactId\u003espring-boot-starter-test\u003c/artifactId\u003e\n            \u003cscope\u003etest\u003c/scope\u003e\n        \u003c/dependency\u003e\n        \u003c!--导入webmagic爬虫的包--\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eus.codecraft\u003c/groupId\u003e\n            \u003cartifactId\u003ewebmagic-core\u003c/artifactId\u003e\n            \u003cversion\u003e0.7.3\u003c/version\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eus.codecraft\u003c/groupId\u003e\n            \u003cartifactId\u003ewebmagic-extension\u003c/artifactId\u003e\n            \u003cversion\u003e0.7.3\u003c/version\u003e\n        \u003c/dependency\u003e\n        \u003c!--配置druid--\u003e\n        \u003c!-- https://mvnrepository.com/artifact/com.alibaba/druid-spring-boot-starter --\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003ecom.alibaba\u003c/groupId\u003e\n            \u003cartifactId\u003edruid-spring-boot-starter\u003c/artifactId\u003e\n            \u003cversion\u003e1.1.17\u003c/version\u003e\n        \u003c/dependency\u003e\n        \u003c!--import solr--\u003e\n        \u003c!-- Compile --\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.springframework.boot\u003c/groupId\u003e\n            \u003cartifactId\u003espring-boot-starter-data-solr\u003c/artifactId\u003e\n        \u003c/dependency\u003e\n\n    \u003c/dependencies\u003e\n\n    \u003cbuild\u003e\n        \u003cplugins\u003e\n            \u003cplugin\u003e\n                \u003cgroupId\u003eorg.springframework.boot\u003c/groupId\u003e\n                \u003cartifactId\u003espring-boot-maven-plugin\u003c/artifactId\u003e\n            \u003c/plugin\u003e\n        \u003c/plugins\u003e\n    \u003c/build\u003e\n\n\u003c/project\u003e\n\n```\n#### 3.1.2 applicaton.yml\n```\nspring:\n  data:\n    solr:\n      host: http://localhost:8080/solr\n  datasource:\n    name: encycproject\n    type: com.alibaba.druid.pool.DruidDataSource\n    #druid相关配置\n    druid:\n      #监控统计拦截的filters\n      filters: stat\n      driver-class-name: com.mysql.cj.jdbc.Driver\n      #基本属性\n      url: jdbc:mysql://127.0.0.1:3306/animal?useUnicode=true\u0026characterEncoding=utf8\u0026useSSL=false\u0026serverTimezone=GMT%2B8\n      username: root\n      password: 1234\n      #配置初始化大小/最小/最大\n      initial-size: 1\n      min-idle: 1\n      max-active: 20\n      #获取连接等待超时时间\n      max-wait: 60000\n      #间隔多久进行一次检测，检测需要关闭的空闲连接\n      time-between-eviction-runs-millis: 60000\n      #一个连接在池中最小生存的时间\n      min-evictable-idle-time-millis: 300000\n      validation-query: SELECT 'x'\n      test-while-idle: true\n      test-on-borrow: false\n      test-on-return: false\n      #打开PSCache，并指定每个连接上PSCache的大小。oracle设为true，mysql设为false。分库分表较多推荐设置为false\n      pool-prepared-statements: false\n      max-pool-prepared-statement-per-connection-size: 20\nserver:\n  port: 8100\n\n```\n#### 3.1.3 前端的axios的代理\n\u003e在config文件夹下的index.js中的proxyTable下添加如下代码\n```\n '/apis':{\n        target: 'http://192.168.43.103:8100/',  // 后台api\n        changeOrigin: true,  //是否跨域\n        // secure: true,\n        pathRewrite: {\n          '^/apis': ''   //需要rewrite的,\n        }\n      }\n```\n\u003e 未完待续\n---\n## 4 项目效果\n#### 4.1 爬虫项目图：  \n**爬虫数据**\n\u003cimg src=\"resource/img/webmagic.jpg\"/\u003e  \n**数据库数据**\n\u003cimg src=\"resource/img/data.jpg\"/\u003e  \n**前端**\n\u003cimg src=\"resource/img/vuehome.jpg\"/\u003e \n\u003cimg src=\"resource/img/vuedetail.jpg\"/\u003e\n\u003e待续  \n## 5 说明  \n### 项目\n\u003e 此项目读者可以下载在此基础上重新编写，或者利用此项目的环境进行别的项目开发。\n\n\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreadnew%2Fencycproject","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthreadnew%2Fencycproject","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreadnew%2Fencycproject/lists"}