https://github.com/tetreum/price-crawler

Article price crawler
https://github.com/tetreum/price-crawler

crawler nodejs

Last synced: 24 days ago
JSON representation

Article price crawler

• Host: GitHub
• URL: https://github.com/tetreum/price-crawler
• Owner: tetreum
• Created: 2018-11-11T21:25:31.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2023-01-04T17:58:18.000Z (about 2 years ago)
• Last Synced: 2023-03-23T22:15:10.522Z (almost 2 years ago)
• Topics: crawler, nodejs
• Language: JavaScript
• Size: 885 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 10
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
[![Build Status](https://travis-ci.org/tetreum/price-crawler.svg?branch=master)](https://travis-ci.org/tetreum/price-crawler)

It will try to find the price of the received url by looking for metadata tags or using a custom crawler if available.

# Requirements

- node >= v8

# Setup

- `npm install`

# Usage

- `node index.js`

- Make the request against `http://localhost:18935/price?url=ARTICLE_URL_YOU_WANT_ITS_PRICE`

On success reply will be:

```js

{

  "amount": 499,

  "currency": "EUR",

}

```

On fail:

```js

{

}

```

# Add a custom crawler

If the website you wanna crawl doesn't have SEO metadata (ex: amazon). You can create a custom crawler in `/crawlers/` folder. Before using the generic methods, price-crawler will check if the given url has it's own crawler there.