{"id":18797785,"url":"https://github.com/networktocode/ntc-netbox-plugin-metrics-ext","last_synced_at":"2025-06-16T13:10:35.974Z","repository":{"id":41243091,"uuid":"281795478","full_name":"networktocode/ntc-netbox-plugin-metrics-ext","owner":"networktocode","description":"NetBox Plugin to improve the instrumentation of NetBox and expose additional metrics (Application Metrics, RQ Worker).","archived":false,"fork":false,"pushed_at":"2023-09-12T08:38:22.000Z","size":323,"stargazers_count":39,"open_issues_count":5,"forks_count":12,"subscribers_count":39,"default_branch":"master","last_synced_at":"2025-06-03T02:02:00.774Z","etag":null,"topics":["netbox","netbox-plugin"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/networktocode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-07-22T22:27:08.000Z","updated_at":"2025-02-10T15:26:02.000Z","dependencies_parsed_at":"2025-04-15T09:01:06.206Z","dependency_job_id":null,"html_url":"https://github.com/networktocode/ntc-netbox-plugin-metrics-ext","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/networktocode/ntc-netbox-plugin-metrics-ext","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networktocode%2Fntc-netbox-plugin-metrics-ext","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networktocode%2Fntc-netbox-plugin-metrics-ext/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networktocode%2Fntc-netbox-plugin-metrics-ext/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networktocode%2Fntc-netbox-plugin-metrics-ext/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/networktocode","download_url":"https://codeload.github.com/networktocode/ntc-netbox-plugin-metrics-ext/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networktocode%2Fntc-netbox-plugin-metrics-ext/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260166308,"owners_count":22968635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["netbox","netbox-plugin"],"created_at":"2024-11-07T22:09:33.367Z","updated_at":"2025-06-16T13:10:35.935Z","avatar_url":"https://github.com/networktocode.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ntc-netbox-plugin-metrics-ext\n\nA plugin for [NetBox](https://github.com/netbox-community/netbox) to expose additional metrics information.\n\nThe plugin is composed of multiple features that can be used independantly:\n- Application Metrics Endpoint: prometheus endpoint at `/api/plugins/metrics-ext/app-metrics`\n- RQ Worker Metrics Command: Add prometheus endpoint on each RQ worker\n\n# Application Metrics Endpoint\n\nNetBox already exposes some information via a Prometheus endpoint but the information currently available are mostly at the system level and not at the application level.\n- **SYSTEM Metrics** are very useful to instrument code, track ephemeral information and get a better visibility into what is happening. (Example of metrics: nbr of requests, requests per second, nbr of exceptions, response time, etc ...) The idea is that when multiple instances of NetBox are running behind a load balancer each one will produce a different set of metrics and the monitoring system needs to collect these metrics from all running instances and aggregate them in a dashboard. NetBox exposes some system metrics at `localhost/metrics` [NetBox DOC](https://netbox.readthedocs.io/en/stable/additional-features/prometheus-metrics/).\n- **APPLICATION Metrics** are at a higher level and represent information that is the same across all instances of an application running behind a load balancer. If I have 3 instances of NetBox running, there is no point to ask each of them how many Device objects I have in the database, since they will always return the same information. In this case, the goal is to expose only 1 endpoint that can be served by any running instance.\n\nSystem metrics and application level metrics are complementary with each other\n\nCurrently the plugin exposes these simple metrics by default:\n- RQ Queues stats\n- Reports stats\n- Models count (configurable via configuration.py)\n\n## Add your own metrics\n\nThis plugin supports some options to generate and publish your own application metrics behind the same endpoint.\n\n### Option 1 - Register function(s) via configuration.py.\n\nIt's possible to create your own function to generate some metrics and register it to the plugin in the configuration.py.\nHere is an example where the custom function are centralized in a `metrics.py` file, located next to the main `configuration.py`.\n\n```python\n# metrics.py\nfrom prometheus_client.core import GaugeMetricFamily\n\ndef metric_prefix_utilization():\n    \"\"\"Report prefix utilization as a metric per container.\"\"\"\n    from ipam.models import Prefix  # pylint: disable=import-outside-toplevel\n\n    containers = Prefix.objects.filter(status=\"container\").all()\n    g = GaugeMetricFamily(\n        \"netbox_prefix_utilization\", \"percentage of utilization per container prefix\", labels=[\"prefix\", \"role\", \"site\"]\n    )\n\n    for container in containers:\n\n        site = \"none\"\n        role = \"none\"\n        if container.role:\n            role = container.role.slug\n\n        if container.site:\n            site = container.site.slug\n\n        g.add_metric(\n            [str(container.prefix), site, role], container.get_utilization(),\n        )\n\n    yield g\n```\nThe new function can be imported in the `configuration.py` file and registered with the plugin.\n```python\n# configuration.py\nfrom netbox.metrics import metric_prefix_utilization\nPLUGINS_CONFIG = {\n    \"netbox_metrics_ext\": {\n      \"app_metrics\": {\n        \"extras\": [\n          metric_prefix_utilization\n        ]\n      }\n    }\n},\n```\n\n### Option 2 - Registry for third party plugins\n\nAny plugin can include its own metrics to improve the visibility and/or the troubleshooting of the plugin itself.\nThird party plugins can register their own function(s) using the `ready()` function as part of their PluginConfig class.\n\n```python\n# my_plugin/__init__.py\nfrom netbox_metrics_ext import register_metric_func\nfrom netbox.metrics import metric_circuit_bandwidth\n\nclass MyPluginConfig(PluginConfig):\n    name = \"netbox_myplugin\"\n    verbose_name = \"Demo Plugin \"\n    # [ ... ]\n    def ready(self):\n        super().ready()\n        register_metric_func(metric_circuit_bandwidth)\n```\n\n### Option 3 - NOT AVAILABLE YET - Metrics directory\n\nIn the future it will be possible to add metrics by adding them in a predefined directory, similar to reports and scripts.\n\n## Parameters\n\nThe behavior of the app_metrics feature can be controlled with the following list of settings (under `netbox_metrics_ext \u003e app_metrics`):\n- `reports` boolean (default True), publish stats about the reports (success, warning, info, failure)\n- `queues` boolean (default True), publish stats about RQ Worker (nbr of worker, nbr and type of job in the different queues)\n- `models` nested dict, publish the count for a given object (Nbr Device, Nbr IP etc.. ). The first level must be the name of the module in lowercase (dcim, ipam etc..), the second level must be the name of the object (usually starting with a uppercase)\n    ```python\n    {\n      \"dcim\": {\"Site\": True, \"Rack\": True, \"Device\": True,},\n      \"ipam\": {\"IPAddress\": True, \"Prefix\": True}\n    }\n    ```\n## Usage\n\nConfigure your Prometheus server to collect the application metrics at `/api/plugins/metrics-ext/app-metrics/`\n\n```yaml\n# Sample prometheus configuration\nscrape_configs:\n  - job_name: 'netbox_app'\n    scrape_interval: 60s\n    metrics_path: /api/plugins/metrics-ext/app-metrics\n    static_configs:\n      - targets: ['netbox']\n```\n\n# RQ Worker Metrics Endpoint\n\nThis plugin add a new django management command `rqworker_metrics` that is behaving identically to the default `rqworker` command except that this command also exposes a prometheus endpoint (default port 8001).\n\nWith this endpoint it become possible to instrument the tasks running asyncronously in the worker.\n\n## Usage\n\nThe new command needs to be executed on the worker as a replacement for the default `rqworker`\n```\npython manage.py rqworker_metrics\n```\n\nThe port used to expose the prometheus endpoint can be configured for each worker in CLI.\n```\npython manage.py rqworker_metrics --prom-port 8002\n```\n\nSince the rq-worker is based on a fork model, for this feature to work it''s required to use prometheus in multi processes mode.\nTo enable this mode the environment variable `prometheus_multiproc_dir` must be define and point at a valid directory.\n\n# Installation\n\nThe plugin is available as a Python package in pypi and can be installed with pip\n```shell\npip install ntc-netbox-plugin-metrics-ext\n```\n\n\u003e The plugin is compatible with NetBox 2.8.1 and higher\n\nTo ensure Application Metrics Plugin is automatically re-installed during future upgrades, create a file named `local_requirements.txt` (if not already existing) in the NetBox root directory (alongside `requirements.txt`) and list the `ntc-netbox-plugin-metrics-ext` package:\n\n```no-highlight\n# echo ntc-netbox-plugin-metrics-ext \u003e\u003e local_requirements.txt\n```\n\nOnce installed, the plugin needs to be enabled in your `configuration.py`\n```python\n# In your configuration.py\nPLUGINS = [\"netbox_metrics_ext\"]\n\n# PLUGINS_CONFIG = {\n#   \"netbox_metrics_ext\": {\n#     \"app_metrics\": {\n#       \"models\": {\n#         \"dcim\": {\"Site\": True, \"Rack\": True, \"Device\": True,},\n#          \"ipam\": {\"IPAddress\": True, \"Prefix\": True},\n#        },\n#        \"reports\": True,\n#        \"queues\": True,\n#       }\n#     }\n#   }\n# }\n```\n\n## Included Grafana Dashboard\n\nIncluded within this plugin is a Grafana dashboard which will work with the example configuration above. To install this dashboard import the JSON from [Grafana Dashboard](netbox_grafana_dashboard.json) into Grafana.\n\n![Netbox Grafana Dashboard](netbox_grafana_dashboard.png)\n\n# Contributing\n\nPull requests are welcomed and automatically built and tested against multiple version of Python and multiple version of NetBox through TravisCI.\n\nThe project is packaged with a light development environment based on `docker-compose` to help with the local development of the project and to run the tests within TravisCI.\n\nThe project is following Network to Code software development guideline and is leveraging:\n- Black, Pylint, Bandit and pydocstyle for Python linting and formatting.\n- Django unit test to ensure the plugin is working properly.\n\n### CLI Helper Commands\n\nThe project is coming with a CLI helper based on [invoke](http://www.pyinvoke.org/) to help setup the development environment. The commands are listed below in 3 categories `dev environment`, `utility` and `testing`.\n\nEach command can be executed with `invoke \u003ccommand\u003e`. All commands support the arguments `--netbox-ver` and `--python-ver` if you want to manually define the version of Python and NetBox to use. Each command also has its own help `invoke \u003ccommand\u003e --help`\n\n#### Local dev environment\n```\n  build            Build all docker images.\n  debug            Start NetBox and its dependencies in debug mode.\n  destroy          Destroy all containers and volumes.\n  start            Start NetBox and its dependencies in detached mode.\n  stop             Stop NetBox and its dependencies.\n```\n\n#### Utility\n```\n  cli              Launch a bash shell inside the running NetBox container.\n  create-user      Create a new user in django (default: admin), will prompt for password.\n  makemigrations   Run Make Migration in Django.\n  nbshell          Launch a nbshell session.\n```\n#### Testing\n\n```\n  tests            Run all tests for this plugin.\n  pylint           Run pylint code analysis.\n  pydocstyle       Run pydocstyle to validate docstring formatting adheres to NTC defined standards.\n  bandit           Run bandit to validate basic static code security analysis.\n  black            Run black to check that Python files adhere to its style standards.\n  unittest         Run Django unit tests for the plugin.\n```\n\n## Questions\n\nFor any questions or comments, please check the [FAQ](FAQ.md) first and feel free to swing by the [Network to Code slack channel](https://networktocode.slack.com/) (channel #networktocode).\nSign up [here](http://slack.networktocode.com/)\n\n## Default Metrics for the application metrics endpoint\n\nBy Default the plugin will generate the following metrics\n```\n# HELP netbox_queue_stats Per RQ queue and job status statistics\n# TYPE netbox_queue_stats gauge\nnetbox_queue_stats{name=\"check_releases\",status=\"finished\"} 0.0\nnetbox_queue_stats{name=\"check_releases\",status=\"started\"} 0.0\nnetbox_queue_stats{name=\"check_releases\",status=\"deferred\"} 0.0\nnetbox_queue_stats{name=\"check_releases\",status=\"failed\"} 0.0\nnetbox_queue_stats{name=\"check_releases\",status=\"scheduled\"} 0.0\nnetbox_queue_stats{name=\"default\",status=\"finished\"} 0.0\nnetbox_queue_stats{name=\"default\",status=\"started\"} 0.0\nnetbox_queue_stats{name=\"default\",status=\"deferred\"} 0.0\nnetbox_queue_stats{name=\"default\",status=\"failed\"} 0.0\nnetbox_queue_stats{name=\"default\",status=\"scheduled\"} 0.0\n# HELP netbox_report_stats Per report statistics\n# TYPE netbox_report_stats gauge\nnetbox_report_stats{name=\"test_hostname\",status=\"success\"} 13.0\nnetbox_report_stats{name=\"test_hostname\",status=\"warning\"} 0.0\nnetbox_report_stats{name=\"test_hostname\",status=\"failure\"} 0.0\nnetbox_report_stats{name=\"test_hostname\",status=\"info\"} 0.0\n# HELP netbox_model_count Per NetBox Model count\n# TYPE netbox_model_count gauge\nnetbox_model_count{app=\"dcim\",name=\"Site\"} 24.0\nnetbox_model_count{app=\"dcim\",name=\"Rack\"} 24.0\nnetbox_model_count{app=\"dcim\",name=\"Device\"} 46.0\nnetbox_model_count{app=\"ipam\",name=\"IPAddress\"} 58.0\nnetbox_model_count{app=\"ipam\",name=\"Prefix\"} 18.0\n# HELP netbox_app_metrics_processing_ms Time in ms to generate the app metrics endpoint\n# TYPE netbox_app_metrics_processing_ms gauge\nnetbox_app_metrics_processing_ms 19.90485\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetworktocode%2Fntc-netbox-plugin-metrics-ext","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetworktocode%2Fntc-netbox-plugin-metrics-ext","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetworktocode%2Fntc-netbox-plugin-metrics-ext/lists"}