https://github.com/scrapinghub/spidermon

Scrapy Extension for monitoring spiders execution.
https://github.com/scrapinghub/spidermon

crawling hacktoberfest monitoring monitoring-tool scraping scrapinghub spiders testing

Last synced: about 2 months ago
JSON representation

Scrapy Extension for monitoring spiders execution.

• Host: GitHub
• URL: https://github.com/scrapinghub/spidermon
• Owner: scrapinghub
• License: bsd-3-clause
• Created: 2015-11-17T12:27:38.000Z (almost 9 years ago)
• Default Branch: master
• Last Pushed: 2024-07-15T14:08:48.000Z (2 months ago)
• Last Synced: 2024-07-16T16:01:50.247Z (2 months ago)
• Topics: crawling, hacktoberfest, monitoring, monitoring-tool, scraping, scrapinghub, spiders, testing
• Language: Python
• Homepage: https://spidermon.readthedocs.io
• Size: 1.36 MB
• Stars: 524
• Watchers: 76
• Forks: 94
• Open Issues: 43
• Metadata Files:
  • Readme: README.rst
  • Changelog: CHANGES.rst
  • Contributing: CONTRIBUTING.rst
  • License: LICENSE

Awesome Lists containing this project

• awesome-scrapy - spidermon

README

        
=========

Spidermon

=========

.. image:: https://github.com/scrapinghub/spidermon/workflows/CI/badge.svg?branch=master

    :target: https://github.com/scrapinghub/spidermon/actions?workflow=CI

    :alt: CI Status

.. image:: https://img.shields.io/codecov/c/github/scrapinghub/spidermon.svg

    :target: http://codecov.io/github/scrapinghub/spidermon?branch=master

    :alt: Coverage report

.. image:: https://img.shields.io/pypi/v/spidermon.svg

    :target: https://pypi.python.org/pypi/spidermon

    :alt: pypi version

.. image:: https://img.shields.io/pypi/l/spidermon.svg

    :target: https://github.com/scrapinghub/spidermon/blob/master/LICENSE

    :alt: licence

.. image:: https://img.shields.io/pypi/pyversions/spidermon.svg

    :target: https://pypi.python.org/pypi/spidermon

    :alt: python versions

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg

    :target: https://github.com/ambv/black

    :alt: Code style: black

Overview

========

Spidermon is an extension for Scrapy spiders. The package provides useful tools for data validation, stats monitoring, and notification messages. This way you leave the monitoring task to Spidermon and just check the reports/notifications.

Requirements

============

* Python Version: 3.8, 3.9, 3.10, 3.11 or 3.12

Install

=======

The quick way::

    pip install spidermon

For more details see the install section in the documentation:

https://spidermon.readthedocs.io/en/latest/installation.html

Documentation

=============

Documentation is available online at https://spidermon.readthedocs.io/ and in the ``docs`` directory.