https://github.com/agentultra/dmon

A stream-processing service for monitoring distributed clusters
https://github.com/agentultra/dmon

Last synced: 3 months ago
JSON representation

A stream-processing service for monitoring distributed clusters

• Host: GitHub
• URL: https://github.com/agentultra/dmon
• Owner: agentultra
• Created: 2013-04-24T13:28:30.000Z (about 11 years ago)
• Default Branch: master
• Last Pushed: 2015-02-06T21:00:49.000Z (over 9 years ago)
• Last Synced: 2024-01-18T11:01:58.201Z (5 months ago)
• Language: Python
• Size: 268 KB
• Stars: 15
• Watchers: 5
• Forks: 3
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Lists

• alex-mikhalev-awesome-stars - dmon - A stream-processing service for monitoring distributed clusters (Python)

README

        
dmon

====

A stream-processing service for monitoring distributed clusters.

Inspired by Riemann, dmon will use Python coroutines as its

stream-processing language.  An example configuration will look

something like:

    from dmon.streams import *

    stream = when(lambda event: event['state'] == "warning",

                  email("[email protected]"))

If you have Hy installed you can also write your configuration in Hy:

    (import [dmon.streams [*]])

    

    (setv stream

      (average 2

        (when (fn [ev] (> (.get ev "metric") 1.85))

              (email "[email protected]"))))

Why Python For Configuration?

-----------------------------

### Flexibility ###

The user defines the stream functions they need. Dmon doesn't need to

anticipate every aggregate or rating function and reinvent the wheel.

No need to learn another configuration language: use the one you're

already familiar with.

### Developer Tools ###

Streams are defined in a plain, old Python module. All of the tools

you use to test, lint, and maintain Python code work on your stream

modules. All of the editor support you rely on just works because it's

just normal Python code.

### Contributor Friendliness ###

No need to maintain a parser.

Getting Started

---------------

To start a dmon server, you'll first need to write a Python module

with your streams in it.  A stream is just a co-routine that receives

events ("is yielded into"), does some processing, and sends the events

to one or more streams.  Bind your streams at the module level to

names containing the word, "stream".  Then tell dmon about where it

can import your module and it'll handle the rest:

    $ export PYTHONPATH="/path/to/my/module:$PYTHONPATH"

    $ export DMON_STREAM_MODULE="mystream"

    $ cd /to/my/dmon/env && source bin/activate

    (dmon)$ cd dmon && pip install -e .

    ...

    (dmon)$ python -m dmon.dmon

    Listening for event datagrams on:  127.0.0.1 7689

That should get the UDP listener up and ready to receive events.

For development purposes I've implemented the most rudimentary

protocol possible: JSON.  I plan to eventually add Protocol Buffers

(and maybe Thrift) support in the future.

An event message has the following keys:

- host

- service

- state

- description

- time

- ttl

- metric

And all fields are required.  This will change in the future as I

intend to mirror Riemann's event protocol which allows fields to be

optional.

Testing

-------

To run the unit tests:

    $ python setup.py test

You can also interact with the dmon server using a convenient shell:

    $ python -m dmon.shell -s 

Where shell can be one of *python*, *ipython*, or *bpython*. It is

optional and defaults to *python*. The alternative options require the

respective shells be installed on your system.

# TODO #

- protocol

  - protobuf

  - thrift ?

  - jsonschema ?

- event storage / querying (in progress)

- web socket support