Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/rasulkireev/ketohub_raw_spider

Last synced: 3 months ago
JSON representation

Host: GitHub
URL: https://github.com/rasulkireev/ketohub_raw_spider
Owner: rasulkireev
Created: 2017-08-13T23:05:11.000Z (over 7 years ago)
Default Branch: master
Last Pushed: 2024-05-14T23:00:38.000Z (8 months ago)
Last Synced: 2024-10-15T07:08:50.630Z (3 months ago)
Language: Python
Size: 108 KB
Stars: 3
Watchers: 2
Forks: 2
Open Issues: 2
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

        # KetoHub Raw Spider

[![Build Status](https://travis-ci.org/mtlynch/ketohub_raw_spider.svg?branch=master)](https://travis-ci.org/mtlynch/ketohub_raw_spider)

[![Coverage Status](https://coveralls.io/repos/github/mtlynch/ketohub_raw_spider/badge.svg?branch=master)](https://coveralls.io/github/mtlynch/ketohub_raw_spider?branch=master)

To run the spiders:

```bash

TIMESTAMP=$(date --iso-8601=seconds | sed -r 's/://g')

OUTPUT_DIR="${HOME}/data/raw/${TIMESTAMP}/"

scrapy crawl diet-doctor -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl greek-goes-keto -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl hey-keto-mama -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl keto-diet-app -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl keto-size-me -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl ketoconnect -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl ketogasm -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl ketovale -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl ketovangelist-kitchen -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl low-carb-yum -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl queen-bs -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl ruled-me -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl skinny-taste -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl sugar-free-mom -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl wholesome-yum -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

scrapy crawl your-friends-j -s "DOWNLOAD_ROOT=${OUTPUT_DIR}"

```