https://github.com/scrapy-plugins/scrapy-crawlera

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
https://github.com/scrapy-plugins/scrapy-crawlera

crawler crawler-detection plugin proxy scraping scrapy

Last synced: 2 months ago
JSON representation

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy

• Host: GitHub
• URL: https://github.com/scrapy-plugins/scrapy-crawlera
• Owner: scrapy-plugins
• License: bsd-3-clause
• Created: 2015-04-24T21:21:32.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2024-02-22T20:51:23.000Z (9 months ago)
• Last Synced: 2024-04-14T16:34:48.602Z (7 months ago)
• Topics: crawler, crawler-detection, plugin, proxy, scraping, scrapy
• Language: Python
• Homepage:
• Size: 270 KB
• Stars: 348
• Watchers: 57
• Forks: 88
• Open Issues: 6
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE.txt

Awesome Lists containing this project

• awesome-scrapy - scrapy-crawlera
• awesome - scrapy-crawlera - Crawlera middleware for Scrapy (Scrapy Middleware)

README

        
======================

scrapy-zyte-smartproxy

======================

.. image:: https://img.shields.io/pypi/v/scrapy-zyte-smartproxy.svg

   :target: https://pypi.python.org/pypi/scrapy-zyte-smartproxy

   :alt: PyPI Version

.. image:: https://travis-ci.org/scrapy-plugins/scrapy-zyte-smartproxy.svg?branch=master

   :target: http://travis-ci.org/scrapy-plugins/scrapy-zyte-smartproxy

   :alt: Build Status

.. image:: http://codecov.io/github/scrapy-plugins/scrapy-zyte-smartproxy/coverage.svg?branch=master

   :target: http://codecov.io/github/scrapy-plugins/scrapy-zyte-smartproxy?branch=master

   :alt: Code Coverage

scrapy-zyte-smartproxy is a `Scrapy downloader middleware`_ to use one of

Zyte’s proxy services: either the `proxy mode`_ of `Zyte API`_ or `Zyte Smart

Proxy Manager`_ (formerly Crawlera).

.. _Scrapy downloader middleware: https://doc.scrapy.org/en/latest/topics/downloader-middleware.html

.. _proxy mode: https://docs.zyte.com/zyte-api/usage/proxy-mode.html

.. _Zyte API: https://docs.zyte.com/zyte-api/get-started.html

.. _Zyte Smart Proxy Manager: https://www.zyte.com/smart-proxy-manager/

Requirements

============

* Python 2.7 or Python 3.4+

* Scrapy 1.4+

Installation

============

You can install scrapy-zyte-smartproxy using pip::

    pip install scrapy-zyte-smartproxy

Documentation

=============

Documentation is available online at

https://scrapy-zyte-smartproxy.readthedocs.io/ and in the ``docs`` directory.