https://github.com/raspi/scrapy-supermicro

Web crawler for Supermicro (supermicro.com) website
https://github.com/raspi/scrapy-supermicro

motherboard scrapy specifications spider webcrawler

Last synced: 8 months ago
JSON representation

Web crawler for Supermicro (supermicro.com) website

• Host: GitHub
• URL: https://github.com/raspi/scrapy-supermicro
• Owner: raspi
• License: apache-2.0
• Created: 2020-07-18T02:06:14.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2020-07-18T04:06:43.000Z (almost 6 years ago)
• Last Synced: 2025-02-25T22:46:35.639Z (over 1 year ago)
• Topics: motherboard, scrapy, specifications, spider, webcrawler
• Language: Python
• Homepage:
• Size: 16.6 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# scrapy-supermicro

Web crawler for Supermicro ([supermicro.com](https://supermicro.com/)) web site.

## Requirements

* Python

* [Scrapy](https://scrapy.org/)

## Notes

* 30 day cache is used in `settings.py`

## Spiders

Everything is downloaded to `items` directory.

### Motherboards

Download all motherboard specifications as JSON files

    scrapy crawl all

### Single motherboard

Download single motherboard's specifications as JSON file

    scrapy crawl mb -a mb="X11SPH-nCTPF"