https://github.com/sectorlabs/celery-cloudwatch
Uploads results of Celery tasks to AWS CloudWatch.
https://github.com/sectorlabs/celery-cloudwatch
Last synced: 11 months ago
JSON representation
Uploads results of Celery tasks to AWS CloudWatch.
- Host: GitHub
- URL: https://github.com/sectorlabs/celery-cloudwatch
- Owner: SectorLabs
- License: mit
- Created: 2016-11-22T15:41:51.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2022-01-06T22:23:19.000Z (over 4 years ago)
- Last Synced: 2025-06-06T00:49:15.279Z (about 1 year ago)
- Language: Python
- Size: 23.4 KB
- Stars: 42
- Watchers: 8
- Forks: 9
- Open Issues: 7
-
Metadata Files:
- Readme: README.rst
- License: LICENSE.md
Awesome Lists containing this project
README
.. image:: https://scrutinizer-ci.com/g/SectorLabs/celery-cloudwatch/badges/build.png?b=master&s=dbee8972701eec8cb3f256360b6c8f2880a8eae7
:target: https://scrutinizer-ci.com/g/SectorLabs/celery-cloudwatch/
.. image:: https://scrutinizer-ci.com/g/SectorLabs/celery-cloudwatch/badges/quality-score.png?b=master&s=b715993dcceb3ec81785e324f3cb36b17c40fa0f
:target: https://scrutinizer-ci.com/g/SectorLabs/celery-cloudwatch/
.. image:: https://badge.fury.io/py/celery-cloudwatch-logs.svg
:target: https://pypi.python.org/pypi/celery-cloudwatch-logs
.. image:: https://img.shields.io/badge/license-MIT-blue.svg
This is a simple monitoring tool that intercepts the results of Celery tasks and uploads the result to AWS CloudWatch.
Installation
------------
Install from PyPi:
.. code-block:: bash
pip install celery-cloudwatch-logs
Configuration
-------------
Configuration is done through environment variables. The following environment variables must be set in order for ``celery-cloudwatch`` to work:
* ``AWS_CLOUDWATCH_ACCESS_KEY``
* ``AWS_CLOUDWATCH_SECRET_KEY``
* ``AWS_CLOUDWATCH_GROUP_NAME``
``AWS_CLOUDWATCH_ACCESS_KEY`` and ``AWS_SECRET_KEY`` are optional. If not specified, Boto3 will handle the configuration for these keys, as described here:
http://boto3.readthedocs.io/en/latest/guide/configuration.html
Optionally, the following environment variables may be set:
* ``REDIS_URL="redis://"``
* ``AWS_CLOUDWATCH_REGION="eu-west-1"``
The CloudWatch log group does not have to exist. If it doesn't exists, it will be created.
Running
-------
.. code-block:: bash
$ celery cloudwatch
This requires you to have ran ``setup.py`` or installed this package through pip. Alternatively, run:
.. code-block:: bash
$ python -m celery_cloudwatch
How it works
------------
Celery CloudWatch connects to your broker and monitors tasks in real time. When a task succeeded, failed was rejected or revoked, it uploads all available information about that task into a log stream on AWS CloudWatch Logs.
Based on the specified log group name in the ``AWS_CLOUDWATCH_GROUP_NAME``, a log group will be created. For each possible result, a separate log stream is created. For each task result, an entry is added to the log stream associated with the result.
Known issues
------------
* ``--broker`` on ``celery cloudwatch`` is ignored.
* No descriptive way to specify other brokers than Redis.
All brokers supported by Celery will work, simply specify the broker URL through the ``REDIS_URL`` environment variable.