{"id":19900015,"url":"https://github.com/scrapy-plugins/scrapy-statsd","last_synced_at":"2025-05-02T22:32:08.129Z","repository":{"id":41975542,"uuid":"185615931","full_name":"scrapy-plugins/scrapy-statsd","owner":"scrapy-plugins","description":null,"archived":false,"fork":false,"pushed_at":"2022-10-19T19:42:23.000Z","size":43,"stargazers_count":6,"open_issues_count":4,"forks_count":4,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-07T08:02:04.457Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scrapy-plugins.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-08T13:53:43.000Z","updated_at":"2023-04-26T12:06:36.000Z","dependencies_parsed_at":"2023-01-20T01:48:44.166Z","dependency_job_id":null,"html_url":"https://github.com/scrapy-plugins/scrapy-statsd","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapy-plugins%2Fscrapy-statsd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapy-plugins%2Fscrapy-statsd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapy-plugins%2Fscrapy-statsd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapy-plugins%2Fscrapy-statsd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scrapy-plugins","download_url":"https://codeload.github.com/scrapy-plugins/scrapy-statsd/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252116462,"owners_count":21697382,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T20:10:50.649Z","updated_at":"2025-05-02T22:32:07.824Z","avatar_url":"https://github.com/scrapy-plugins.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"scrapy-statsd-extension\n=======================\n\nThe scrapy-statsd-extension is a scrapy extension to send scrapy stats\nto statsd host.\n\nNaming\n------\n\nScrapy stats names will be mapped to names supported by statsd. Some\nexamples:\n\n::\n\n    downloader/request_bytes -\u003e downloader.request_bytes\n    downloader/request_method_count/GET -\u003e downloader.request_method_count.GET\n    finish_reason -\u003e finish_reason\n    robotstxt/exception_count/\u003cclass 'PermissionError'\u003e -\u003e robotstxt.exception_count.class_PermissionError\n\nInstallation\n------------\n\nInstall scrapy-statsd-extension using ``pip``:\n\n::\n\n    $ pip install scrapy-statsd-extension\n\nConfiguration\n-------------\n\nFirst, you need to include the extension to your ``EXTENSIONS`` dict in\n``settings.py``, like so:\n\n::\n\n    STATSD_ENABLED = True\n\n    EXTENSIONS = {\n        ...\n        'scrapy_statsd_extension.StatsdExtension': 123,\n        ...\n    }\n\nSettings\n--------\n\nTo configure host settings, you can adjust the following:\n\n::\n\n    STATSD_HOST = 'localhost'\n    STATSD_PORT = 8125\n\nThe default endpoint for sending stats will be:\n\n::\n\n    http://localhost:8125\n\nTo enable the extension you must set:\n\n::\n\n    STATSD_ENABLED = False\n\nPeriodic logging is enabled by default but you can disable it, in which\ncase the metrics will be logged only once, when a spider is closed:\n\n::\n\n    STATSD_LOG_PERIODIC = True\n\nBy default, stats are logged every 5 seconds, you can adjust that using\n``STATSD_LOG_EVERY`` which is the number of seconds between logging\noperations:\n\n::\n\n    STATSD_LOG_EVERY = 5\n\nSet the desired prefix:\n\n::\n\n    STATSD_PREFIX = 'scrapy'\n\nYou can specify which prefixes you want logged if you don't want all\nscrapy stats. The default is an empty list which indicates that all\nstats should be logged. You can, for example, log only downloader and\nrobotstxt exception stats by setting ``STATSD_LOG_ONLY`` to\n``['downloader', 'robotstxt.exception_count']``.\n\n::\n\n    STATSD_LOG_ONLY = []\n\nYou can also specify prefixes to ignore the same way using\n``STATSD_IGNORE``:\n\n::\n\n    STATSD_IGNORE = []\n\nTags\n----\n\nCertain platforms such as datadog and influxdb offer tagging options.\n\nTo enable tagging set ``STATSD_TAGGING`` to ``True``, it is disabled by\ndefault:\n\n::\n\n    STATSD_TAGGING = False\n\nThen, you can set custom tags using ``STATSD_TAGS``. Currently, only\n``spider_name_tag`` is supported and setting it to True will add the spider's\nas a tag on all metrics:\n\n::\n\n    STATSD_TAGS = {\n        'spider_name_tag': True\n    }\n\nYou can also set custom tags by setting ``statsd_tags`` attribute on each \nspider. This must be a dictionary containing tag names as keys and tag values as\ndictionary values.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapy-plugins%2Fscrapy-statsd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscrapy-plugins%2Fscrapy-statsd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapy-plugins%2Fscrapy-statsd/lists"}