https://github.com/xanke/node-crawler-server

一个轻量级nodejs的远程采集服务器
https://github.com/xanke/node-crawler-server

crawler nodejs server

Last synced: 27 days ago
JSON representation

一个轻量级nodejs的远程采集服务器

• Host: GitHub
• URL: https://github.com/xanke/node-crawler-server
• Owner: xanke
• Created: 2017-11-04T14:12:09.000Z (about 7 years ago)
• Default Branch: master
• Last Pushed: 2023-01-23T18:21:14.000Z (almost 2 years ago)
• Last Synced: 2023-02-26T04:41:57.690Z (almost 2 years ago)
• Topics: crawler, nodejs, server
• Language: JavaScript
• Homepage:
• Size: 74.2 KB
• Stars: 6
• Watchers: 2
• Forks: 0
• Open Issues: 5
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# node-crawler

一个基于nodejs的网站采集服务器

可部署在任何服务器上远程调用

#### DEMO

发送请求

run npm dev

POST: /scan

```json

{

    "url": "http://www.charlotterusse.com/shoes/shop-all-shoes?sz=30&start=0",

    "model": [{"find":".product-tile","child":[{"name":"name","type":"","method":[{"find":".name-link","attr":""}]},{"name":"price","type":"","method":[{"find":".price-sales","attr":""}]},{"name":"image","type":"","method":[{"find":"img","attr":"src"}]},{"name":"url","type":"","method":[{"find":"a","attr":"href"}]}]}],

    "oid": "7e11c4cb-0456-e9a9-65d5-b64c86d12b94"

}

```

#### 部署

推荐使用 forever https://github.com/foreverjs/forever

##### 启动服务器

forever start server.js