Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/aimer1124/JianshuSpider
Use Node.js,HighChart,BootStrap,Mongo,Cucumber with Gulp to scrapy information from Jianshu.
https://github.com/aimer1124/JianshuSpider
gulp mongodb nodejs spider
Last synced: about 2 months ago
JSON representation
Use Node.js,HighChart,BootStrap,Mongo,Cucumber with Gulp to scrapy information from Jianshu.
- Host: GitHub
- URL: https://github.com/aimer1124/JianshuSpider
- Owner: aimer1124
- Created: 2016-06-20T10:38:49.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2018-09-25T07:40:24.000Z (over 6 years ago)
- Last Synced: 2024-08-05T17:29:04.047Z (5 months ago)
- Topics: gulp, mongodb, nodejs, spider
- Language: JavaScript
- Homepage: http://www.jianshu.com/
- Size: 463 KB
- Stars: 10
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: history.md
Awesome Lists containing this project
- awesome-hacking-lists - aimer1124/JianshuSpider - Use Node.js,HighChart,BootStrap,Mongo,Cucumber with Gulp to scrapy information from Jianshu. (JavaScript)
README
# JianshuSpider
[中文文档](https://github.com/aimer1124/JianshuSpider/blob/master/README_ZH.md)
[](https://travis-ci.org/aimer1124/JianshuSpider) [](https://codeclimate.com/github/aimer1124/JianshuSpider) [](https://codeclimate.com/github/aimer1124/JianshuSpider/coverage) [](https://codeclimate.com/github/aimer1124/JianshuSpider)
> The function in this project is all `my self mind`, because i want to improve my `Node.js`.So it's `open source`, welcome `PR` `Fork` or `Star`.
## Project
- Source
- [https://github.com/aimer1124/JianshuSpider](https://github.com/aimer1124/JianshuSpider)
- Demand
- [Teambition-JianshuSpider](https://www.teambition.com/project/57a1802f767c4b360c918e49/tasks/scrum/57a1802f767c4b360c918e4c)
- UI
- [Home](https://www.processon.com/view/link/57a1c693e4b0de6d056db518)
- [Article](https://www.processon.com/view/link/57a2d0f1e4b0358f8ad7f03b)
- [Author](https://www.processon.com/diagraming/5819751de4b03a76e94bea6f)
- [Collections](https://www.processon.com/diagraming/5819746ae4b06e7dcfc9b338)
- [Search](https://www.processon.com/apps/58197ab8e4b03400d95e4c62)
- ShowCase
- Home
- Articles
- Authors
- Collections
- Search
## Code
- Start
- DB: `sudo mongod`
- Project: `gulp`
- Test: `gulp test`
- Version
- Node: `8.11.3`
- Mongo: `4.0.2`
- Dependencies
- Backend
- express: [http://expressjs.com/](http://expressjs.com/),node web framework
- body-parser: Parse HTTP request body.
- cookie-parser: Parse cookie header and populate req.cookies. See also cookies and keygrip.
- debug: tiny node.js debugging utility modelled after node core's debugging technique.
- morgan: HTTP request logger.
- serve-favicon: Serve a favicon.
- async: [https://github.com/caolan/async](https://github.com/caolan/async),provides straight-forward, powerful functions for working with asynchronous JavaScript
- cheerio: [https://github.com/cheeriojs/cheerio](https://github.com/cheeriojs/cheerio),Fast, flexible, and lean implementation of core jQuery designed specifically for the server.
- moment: [http://momentjs.com](http://momentjs.com),Parse, validate, manipulate, and display dates in JavaScript.
- mongoose: [http://mongoosejs.com/](http://mongoosejs.com/),elegant mongodb object modeling for node.js
- node-schedule: [https://github.com/node-schedule/node-schedule](https://github.com/node-schedule/node-schedule),a flexible cron-like and not-cron-like job scheduler for Node.js.
- SuperAgent: [http://visionmedia.github.io/superagent/](http://visionmedia.github.io/superagent/),light-weight progressive ajax API crafted for flexibility, readability, and a low learning curve after being frustrated with many of the existing request APIs.
- FrontEnd
- HighCharts: [http://www.highcharts.com](http://www.highcharts.com),set up interactive charts in their web pages
- jade: [http://jade-lang.com/](http://jade-lang.com/), robust, elegant, feature rich template engine for Node.js
- bootstrap: [http://getbootstrap.com/](http://getbootstrap.com/),developing responsive, mobile first projects on the web.
- bootstrap-table: An extended Bootstrap table with radio, checkbox, sort, pagination, and other added features.
- bootstrap-select: utilizes Bootstrap's dropdown.js to style and bring additional functionality to standard select elements.
- Develop
- browser-sync: [http://browsersync.io](http://browsersync.io), Time-saving synchronised browser testing.
- gulp: [http://gulpjs.com](http://gulpjs.com),Automate and enhance your workflow
- gulp-mocha: [https://www.npmjs.com/package/gulp-mocha/](https://www.npmjs.com/package/gulp-mocha/),a thin wrapper around Mocha and your issue is most likely with Mocha.
- gulp-nodemon: [https://www.npmjs.com/package/gulp-nodemon/](https://www.npmjs.com/package/gulp-nodemon/), gulp + nodemon + convenience
- Test
- cucumber: [https://cucumber.io/](https://cucumber.io/),Simple, human collaboration
- selenium-webdriver: [http://www.seleniumhq.org/projects/webdriver/](http://www.seleniumhq.org/projects/webdriver/),Driving a browser natively as a user would either locally or on a remote machine using the Selenium Server it marks a leap forward in terms of browser automation.
- chromedriver: [https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/chrome/ChromeDriver.html](https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/chrome/ChromeDriver.html),A WebDriver implementation that controls a Chrome browser running on the local machine.
- should: [https://github.com/shouldjs/should.js](https://github.com/shouldjs/should.js), BDD style assertions for node.js
- SuperTest: [https://github.com/visionmedia/supertest](https://github.com/visionmedia/supertest), Super-agent driven library for testing node.js HTTP servers using a fluent API
- Change
- [History](https://github.com/aimer1124/JianshuSpider/blob/master/history.md)