https://github.com/noseparte/depthsearch

A project through search engine and Hadoop, based on springboot.
https://github.com/noseparte/depthsearch

hadoop java lucence mongodb redis

Last synced: about 1 month ago
JSON representation

A project through search engine and Hadoop, based on springboot.

• Host: GitHub
• URL: https://github.com/noseparte/depthsearch
• Owner: noseparte
• Created: 2018-12-20T06:35:41.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2018-12-27T13:09:13.000Z (about 6 years ago)
• Last Synced: 2024-11-06T22:08:35.320Z (3 months ago)
• Topics: hadoop, java, lucence, mongodb, redis
• Language: Java
• Size: 77.1 KB
• Stars: 3
• Watchers: 4
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
## depth-search(深度搜索)

   ~~~

   A project through search engine and Hadoop, based on springboot.

   ~~~

   

   Update on 2018/12/20 at 21:02

   ~~~

   1.SpringBoot loads the devtools hot update(SpringBoot加载devtools热更新)

   2.@value injection properties cannot be static(@value注入属性不能是静态的)

   3.The mongo driver gets the data in the database table and walks through it(mongo驱动程序获取数据库表中的数据并遍历它)

   ~~~

   

   Update on 2018/12/25 at 15:33

   ~~~

   1. The @scheduled Lucene index database was updated once /5 minutes ago(@scheduled Lucene索引数据库5分钟更新一次)

   2. Mongodb is retrieved through keyword keyword search in combination with IK participle(Mongodb是通过关键词搜索结合IK分词检索的)

   ~~~

   

   Update on 2018/12/26 at 11:46

   ~~~

   1. Webmagic network spider(webmagic网络爬虫)

   ~~~

   

   Update on 2018/12/27 at 11:46

   ~~~

   1. Webmagic network spider(webmagic网络爬虫)

      NationalGeographyPageProcessor    Processor抓取页面

      NationalGeographyPipeline         抓取网页数据，入库mongodb

      NationalGeographyRepo             pojo

   ~~~ 

   

   Update on 2018/12/27 at 21:05

   ~~~

   1. Crawling national geographic data(爬取国家地理数据)

   2. code optimization Extract common part injection Service(抽取公共部分 Service注入)

   3. Integrated reids (集成redis)   

   ~~~