https://github.com/eduardo-matos/simple-crawler

Simple crawler app that searches for product prices, names and urls for each product at epocacosmeticos.com.br
https://github.com/eduardo-matos/simple-crawler

Last synced: about 1 month ago
JSON representation

Simple crawler app that searches for product prices, names and urls for each product at epocacosmeticos.com.br

• Host: GitHub
• URL: https://github.com/eduardo-matos/simple-crawler
• Owner: eduardo-matos
• License: unlicense
• Created: 2014-09-06T01:48:29.000Z (about 10 years ago)
• Default Branch: master
• Last Pushed: 2014-09-10T22:40:30.000Z (about 10 years ago)
• Last Synced: 2023-03-17T18:31:16.564Z (over 1 year ago)
• Language: Python
• Size: 160 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Simple Crawler

[![Build Status](https://travis-ci.org/eduardo-matos/simple-crawler.svg?branch=master)](https://travis-ci.org/eduardo-matos/simple-crawler)

Simple crawler app that searches for product prices, names and urls for each product at epocacosmeticos.com.br

## Installation

run `pip install -r requirements.txt`.

## Exporting as CSV

run `scrapy crawl crawler -o products.csv -t csv`.

## Testing

run `python runtests.py`.

If you want to run only the unit tests, run `python runtests.py SpiderTest`.