https://github.com/q-m/rest-scrapyd
Ruby client for the Scrapyd REST API
https://github.com/q-m/rest-scrapyd
rest ruby scraping scrapy
Last synced: about 1 year ago
JSON representation
Ruby client for the Scrapyd REST API
- Host: GitHub
- URL: https://github.com/q-m/rest-scrapyd
- Owner: q-m
- License: mit
- Created: 2015-02-21T20:29:18.000Z (over 11 years ago)
- Default Branch: master
- Last Pushed: 2015-08-04T08:02:39.000Z (almost 11 years ago)
- Last Synced: 2024-09-17T22:52:53.662Z (almost 2 years ago)
- Topics: rest, ruby, scraping, scrapy
- Language: Ruby
- Size: 152 KB
- Stars: 1
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
rest-scrapyd
============
[](http://badge.fury.io/rb/rest-scrapyd)
[](http://rubydoc.info/github/q-m/rest-scrapyd)
Ruby client for the [Scrapyd](http://scrapyd.readthedocs.org/) REST API
built on top of [rest-core](https://github.com/godfat/rest-core).
Installation
------------
```sh
gem install rest-scrapyd
```
or, when using a Gemfile
```ruby
gem 'rest-scrapyd'
```
and run `bundle install`.
Usage
-----
```ruby
require 'rest-scrapyd'
r = RestScrapyd.new site: "http://example.com:6800/"
r.listprojects
# => ["project1", "project2"]
r.listspiders(project: "project1")
# => ["spider1", "spider2"]
# connect to default site at http://localhost:6800/
r = RestScrapyd.new
# and set a default project
r.project = "project1"
r.listspiders
# => ["spider1", "spider2"]
r.listversions
# => ["123456-master"]
# you can also specify a default project on construction
r = RestScrapyd.new project: "project1"
r.schedule("spider1", "123456-master")
# => "1234567890abcdef1234567890abcdef"
# http basic is also possible, when running scrapyd behind a reverse proxy
r = RestScrapyd.new site: "https://example.com:6843/", username: "deploy", password: "s3cret"
```
For more information, see the [RestScrapyd](http://rubydoc.info/github/q-m/rest-scrapyd/RestScrapyd.html)
and [Scrapyd API](http://scrapyd.readthedocs.org/en/latest/api.html) documentation.
Copyright
---------
Copyright © 2015 wvengen, released under the [MIT license](LICENSE).