https://github.com/akarshsatija/mr-scrappy

To crawl a page on a ecom website and save its data in Mongo
https://github.com/akarshsatija/mr-scrappy

Last synced: 1 day ago
JSON representation

To crawl a page on a ecom website and save its data in Mongo

• Host: GitHub
• URL: https://github.com/akarshsatija/mr-scrappy
• Owner: AkarshSatija
• Created: 2015-02-22T12:08:07.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2015-02-22T15:07:44.000Z (over 9 years ago)
• Last Synced: 2023-08-07T02:54:20.173Z (over 1 year ago)
• Language: JavaScript
• Size: 145 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Crawl-Engine

Descriptipn: To crawl a page on a ecom website and save its data in Mongo

Sample Link: https://www.kultureshop.in/papercut-mens-art-tees.html/

Prerequisite

- NodeJS should be installed

- MongoDb Server should be installed and running in the machine

- NPM should be installed

Then,

Clone this repo

To Install Dependencies Run-

```js

npm install

```

Config file is /config.js to add mongoDB credentials

To start the app Run- 

```js

npm start

```

The url is currently fetched from an inline variable on index.js and currently server is also being created in same file

I'll update this and write it in better and modular way.