https://github.com/eklem/vinmonopolet-crawler

Crawling Vinmonopolet-data and indexing it to a norch search index
https://github.com/eklem/vinmonopolet-crawler

crawler dataset javascript norch search-engine

Last synced: about 1 month ago
JSON representation

Crawling Vinmonopolet-data and indexing it to a norch search index

• Host: GitHub
• URL: https://github.com/eklem/vinmonopolet-crawler
• Owner: eklem
• License: mit
• Created: 2017-02-27T14:28:56.000Z (almost 8 years ago)
• Default Branch: master
• Last Pushed: 2018-12-08T19:41:44.000Z (about 6 years ago)
• Last Synced: 2024-10-15T10:42:46.155Z (3 months ago)
• Topics: crawler, dataset, javascript, norch, search-engine
• Language: JavaScript
• Homepage:
• Size: 4.41 MB
• Stars: 0
• Watchers: 1
• Forks: 1
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# vinmonopolet-crawler

Crawling Vinmonopolet-data and indexing it to a `norch` search index, using [`vinmonopolet`](https://github.com/rexxars/vinmonopolet) and [`search-index-indexer`](https://github.com/eklem/search-index-indexer) library.

Has these features:

* Removing unwanted key/value pars by it's keys

* Changing values to arrays for defined keys. Preparing for being used as filters.

* Copying value of defined key to id-value. Makes it easy to update index.

* Creating image URL based on id/code.

* Pushing filter.category to product.productType when different for better filter functionality.