https://github.com/newrelic/newrelic-airflow-plugin

Send airflow metrics to New Relic!
https://github.com/newrelic/newrelic-airflow-plugin

airflow-plugin monitoring newrelic python telemetry

Last synced: 6 months ago
JSON representation

Send airflow metrics to New Relic!

• Host: GitHub
• URL: https://github.com/newrelic/newrelic-airflow-plugin
• Owner: newrelic
• License: apache-2.0
• Created: 2019-11-06T19:34:02.000Z (over 6 years ago)
• Default Branch: main
• Last Pushed: 2024-08-26T23:35:00.000Z (almost 2 years ago)
• Last Synced: 2025-09-24T23:44:26.297Z (9 months ago)
• Topics: airflow-plugin, monitoring, newrelic, python, telemetry
• Language: Python
• Homepage:
• Size: 43.9 KB
• Stars: 25
• Watchers: 23
• Forks: 19
• Open Issues: 18
• Metadata Files:
  • Readme: README.rst
  • Contributing: CONTRIBUTING.rst
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Codeowners: .github/CODEOWNERS

Awesome Lists containing this project

README

          
|header|

.. |header| image:: https://github.com/newrelic/opensource-website/raw/master/src/images/categories/Community_Plus.png

    :target: https://opensource.newrelic.com/oss-category/#community-plus

New Relic Airflow Metric Exporter

=================================

|ci| |black|

.. |ci| image:: https://img.shields.io/azure-devops/build/NRAzurePipelines/Python/19.svg

    :target: https://dev.azure.com/NRAzurePipelines/Python/_build/latest?definitionId=19&branchName=master

.. |black| image:: https://img.shields.io/badge/code%20style-black-000000.svg

    :target: https://github.com/psf/black

A plugin for `Apache Airflow `_ to send

`metrics `_ to

`New Relic `_.

Requirements

------------

Airflow versions >= 1.10 are preferred for ease of use; however, versions >= 1.8 should work.

Using the plugin

----------------

To start, the ``newrelic-airflow-plugin`` package must be installed. To install

via pip:

.. code-block:: bash

    $ pip install newrelic-airflow-plugin

Configuration

+++++++++++++

Set the ``NEW_RELIC_INSERT_KEY`` environment variable to a valid

`New Relic insert key `_

The ``NEW_RELIC_SERVICE_NAME`` environment variable can be set to customize the

``service.name`` attribute on all generated metrics. The default value is

``Airflow``.

``NEW_RELIC_HOST`` environment variable can be used to set datacenter host.

For example, to send metrics to EU data center set this variable to ``metric-api.eu.newrelic.com``

By default metrics will be send to US data center.

Airflow Versions >= 2.0

++++++++++++++++++++++++

Disable Airflow's lazy plugin loading. This is required for the plugin to properly patch Airflow's Stats engine.

This can be done via environment variable:

``AIRFLOW__CORE__LAZY_LOAD_PLUGINS=False``

Or can be set in your config file (``airflow.cfg``):

.. code-block::

    [core]

    lazy_load_plugins = False

Airflow Versions >= 1.10,<2.0

++++++++++++++++++++++++++++++

✨ That's it! ✨

Airflow Versions < 1.10

+++++++++++++++++++++++

The `newrelic_plugin.py `_

file must be copied into the configured ``plugins_folder`` directory. This

defaults to ``{AIRFLOW_HOME}/plugins``.