https://github.com/rangertaha/scrapy-prometheus-exporter

Scrapy stats exporter for prometheus
https://github.com/rangertaha/scrapy-prometheus-exporter

grafana prometheus prometheus-exporter

Last synced: about 1 month ago
JSON representation

Scrapy stats exporter for prometheus

• Host: GitHub
• URL: https://github.com/rangertaha/scrapy-prometheus-exporter
• Owner: rangertaha
• License: mit
• Created: 2017-09-07T04:45:14.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2024-10-28T04:32:51.000Z (2 months ago)
• Last Synced: 2024-11-20T07:25:30.778Z (about 1 month ago)
• Topics: grafana, prometheus, prometheus-exporter
• Language: Python
• Homepage:
• Size: 147 KB
• Stars: 17
• Watchers: 4
• Forks: 11
• Open Issues: 2
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

README

        
==========================

scrapy-prometheus-exporter

==========================

scrapy-prometheus-exporter is an extension to export Scrapy stats to be used

with Prometheus

Grafana Dashboard

=================

.. image:: /grafana/grafana.png

   :height: 100px

   :width: 200 px

   :scale: 50 %

   :alt: Grafana dashboard of the exported data

   :align: center

Installation

============

Install scrapy-prometheus-exporter using ``pip``::

    $ pip install scrapy-prometheus-exporter

Configuration

=============

First, you need to include the extension to your ``EXTENSIONS`` dict in

``settings.py``, for example::

    EXTENSIONS = {

        'scrapy_prometheus_exporter.prometheus.WebService': 500,

    }

By default the extension is enabled. To disable the extension you need to

set `PROMETHEUS_ENABLED`_ to ``False``.

The web server will listen on a port specified in `PROMETHEUS_PORT`_

(by default, it will try to listen on port 9410)

The endpoint for accessing exported metrics is::

    http://0.0.0.0:9410/metrics

Settings

========

These are the settings that control the metrics exporter:

PROMETHEUS_ENABLED

------------------

Default: ``True``

A boolean which specifies if the exporter will be enabled (provided its

extension is also enabled).

PROMETHEUS_PORT

---------------

Default: ``[6080]``

The port to use for the web service. If set to ``None`` or ``0``, a

dynamically assigned port is used.

PROMETHEUS_HOST

---------------

Default: ``'0.0.0.0'``

The interface the web service should listen on.

PROMETHEUS_PATH

---------------

Default: ``'metrics'``

The url path to access exported metrics Example::

    http://0.0.0.0:9410/metrics

PROMETHEUS_UPDATE_INTERVAL

--------------------------

Default: ``30``

This extensions periodically collects stats for exporting. The interval in

seconds between metrics updates can be controlled with this setting.