{"id":18440005,"url":"https://github.com/sni/lmd","last_synced_at":"2025-04-07T11:10:58.017Z","repository":{"id":10760923,"uuid":"66875262","full_name":"sni/lmd","owner":"sni","description":"Livestatus Multitool Daemon - Create livestatus federation from multiple sources","archived":false,"fork":false,"pushed_at":"2025-03-14T22:58:58.000Z","size":4123,"stargazers_count":43,"open_issues_count":4,"forks_count":31,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-03-31T10:06:50.998Z","etag":null,"topics":["go","icinga","livestatus","livestatus-api","monitoring","naemon","nagios","thruk"],"latest_commit_sha":null,"homepage":"https://labs.consol.de/omd/packages/lmd/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sni.png","metadata":{"files":{"readme":"README.md","changelog":"Changes","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-08-29T19:46:38.000Z","updated_at":"2025-03-14T22:59:01.000Z","dependencies_parsed_at":"2024-01-26T13:11:44.783Z","dependency_job_id":"6919096b-a815-4d68-86eb-b6ae01547d80","html_url":"https://github.com/sni/lmd","commit_stats":{"total_commits":1464,"total_committers":26,"mean_commits":56.30769230769231,"dds":"0.16530054644808745","last_synced_commit":"8236956bbb798c6a8f96939d48c55955ba9fbbe9"},"previous_names":[],"tags_count":68,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sni%2Flmd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sni%2Flmd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sni%2Flmd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sni%2Flmd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sni","download_url":"https://codeload.github.com/sni/lmd/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247640465,"owners_count":20971557,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","icinga","livestatus","livestatus-api","monitoring","naemon","nagios","thruk"],"created_at":"2024-11-06T06:27:58.077Z","updated_at":"2025-04-07T11:10:57.999Z","avatar_url":"https://github.com/sni.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LMD - Livestatus Multitool Daemon\n\n[![Build Status](https://github.com/sni/lmd/workflows/citest/badge.svg)](https://github.com/sni/lmd/actions?query=workflow:citest)\n[![Go Report Card](https://goreportcard.com/badge/github.com/sni/lmd)](https://goreportcard.com/report/github.com/sni/lmd)\n[![License: GPL v3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](http://www.gnu.org/licenses/gpl-3.0)\n\nLMD fetches Livestatus data from one or multiple sources and provides:\n\n- A combined Livestatus API for those sources.\n- A cache which makes Livestatus queries a lot faster than requesting them directly from the remote sources.\n- A enhanced Livestatus API with more useful output formats and sorting. See list below.\n- Aggregated Livestatus results for multiple backends.\n- A Prometheus exporter for Livestatus based metrics for Nagios, Icinga, Shinken and Naemon.\n\nSo basically this is a \"Livestatus In / Livestatus Out\" daemon. Its main purpose is to\nprovide the backend handling of the [Thruk Monitoring Gui](http://www.thruk.org) in a native\ncompiled fast daemon, but it should work for everything which requires Livestatus.\n\n\u003cimg src=\"docs/Architecture.png\" alt=\"Architecture\" style=\"width: 600px;\"/\u003e\n\nLog table requests and commands are just passed through to the actual backends.\n\n## Requirements\n\nYou will need\n\n- Go \u003e= 1.23\n\nto compile lmd.\n\n## How does it work\n\nAfter starting LMD, it fetches all tables via Livestatus API from all\nconfigured remote backends. It then polls periodically all dynamic parts of the\nobjects, like host status, plugin output or downtime status.\n\nWhen there are no incoming connections to LMD, it switches into the idle mode\nwith a slower poll interval. As soon as the first client requests some data,\nLMD will do a spin up and run a synchronous update (with small timeout) and\nchange back to the normal poll interval.\n\n## Usage\n\nIf you want to use LMD with Thruk within OMD, see the [omd/lmd\npage](https://labs.consol.de/omd/packages/lmd/) for a quick start. The\n[OMD-Labs Edition](https://labs.consol.de/omd/) is already prepared to use LMD.\n\n## Installation\n\n    %\u003e go install github.com/sni/lmd/v2/lmd@latest\n\nor\n\n    %\u003e git clone https://github.com/sni/lmd\n    %\u003e cd lmd\n    %\u003e make\n\nQuick start with command line parameters:\n\n    lmd -o listen=:3333 -o connection=test,remote_host:6557\n\nOr copy lmd.ini.example to lmd.ini and change to your needs. Then run lmd.\nYou can specify the path to your config file with `--config`.\n\n    lmd --config=/etc/lmd/lmd.ini\n\n## Configuration\n\nThe configuration is explained in the `lmd.ini.example` in detail.\nThere are several different connection types.\n\n### TCP Livestatus\n\nRemote Livestatus connections via tcp can be defined as:\n\n    [[Connections]]\n    name   = \"Monitoring Site A\"\n    id     = \"id1\"\n    source = [\"192.168.33.10:6557\"]\n\nIf the source is a cluster, you can specify multiple addresses like\n\n    source = [\"192.168.33.10:6557\", \"192.168.33.20:6557\"]\n\n### Unix Socket Livestatus\n\nLocal unix sockets Livestatus connections can be defined as:\n\n    [[Connections]]\n    name   = \"Monitoring Site A\"\n    id     = \"id1\"\n    source = [\"/var/tmp/nagios/live.sock\"]\n\n### Cluster Mode\n\nIt is possible to operate LMD in a cluster mode which means multiple LMDs connect to a network and share the resources.\nAll backend connections will be split up and divided upon all cluster nodes. Incoming requests will be forwarded and merged.\n\nIn order to setup cluster operations, you need to add a http(s) listener and a list of nodes.\nAll nodes should share the same configuration file.\n\n    Listen  = [\"/var/tmp/lmd.sock\", \"http://*:8080\"]\n    Nodes   = [\"http://10.0.0.1:8080\", \"http://10.0.0.2:8080\"]\n\n## What is different in LMD\n\nThere are some new/changed Livestatus query headers:\n\n### Output Format\n\nThe default OutputFormat is `wrapped_json` but `json` is also supported.\n\nThe `wrapped_json` format will put the normal `json` result in a hash with\nsome more extra meta data:\n\n- columns: the column names (if requested)\n- data: the original result.\n- total_count: the number of matches in the result set _before_ the limit and offset applied.\n- rows_scanned: the number of data rows scanned to produce the result set.\n- failed: a hash of backends which have errored for some reason.\n\n### Response Header\n\nThe only ResponseHeader supported right now is `fixed16`.\n\n### Backends Header\n\nThere is a new Backends header which may set a space separated list of\nbackends. If none specific, all are returned.\n\nex.:\n\n    Backends: id1 id2\n\n### Offset Header\n\nThe offset header can be used to only retrieve a subset of the complete result\nset. Best used together with the sort header.\n\n    Offset: 100\n    Limit: 10\n\nThis will return entries 100-109 from the overall result set.\n\n### Sort Header\n\nThe sort header can be used to sort the results by one or more columns.\nMultiple sort header can be used.\n\n    Sort: \u003ccolumn name\u003e \u003casc/desc\u003e\n\nSorting by custom variables is possible like this:\n\n    Sort: custom_variables \u003cname\u003e \u003casc/desc\u003e\n\nex.:\n\n    GET hosts\n    Sort: state asc\n    Sort: name desc\n    Sort: custom_variables WORKER asc\n\n### Additional Columns\n\n- peer_key: id of the backend where this object belongs too (all tables)\n- peer_name: name of the backend where this object belongs too (all tables)\n- has_long_plugin_output: flag if there is long_plugin_output or not (hosts/services table)\n\n### Additional Tables\n\n- sites: list of connected backends\n\n## Resource Usage\n\nThe improved performance comes at a price of course. The following numbers\nshould give you a rough idea on what to expect: An example installation with\n200.000 services at a 3 second update interval uses around 1.5gB of memory and\n200kByte/s of bandwidth.  This makes an average of 7kB memory and 1Byte/s of\nbandwidth usage per service.\n\nHowever your milage may vary, these number heavily depend on the size of the\nplugin output and the check interval of your services.  Use the Prometheus\nexporter to create nice graphs to see how your environment differs.\n\nBtw, changing the update interval to 30 seconds does not reduce the used\nbandwidth, you just have to update many services every 30 seconds than small\npackages every 3 seconds.\n\n## Debugging\n\nLMD can be started with a golang tcp debug profiler:\n\n    lmd -config lmd.ini -debug-profiler localhost:6060\n\nYou can then fetch memory heap profile with:\n\n    curl http://localhost:6060/debug/pprof/heap --output heap.tar.gz\n\nor directly run the profiler from go:\n\n    go tool pprof -web http://localhost:6060/debug/pprof/heap\n\nAccordingly cpu profile can be created by:\n\n    curl http://localhost:6060/debug/pprof/profile --output cpu.tar.gz\n\nThose profiles can then be further processed by gos pprof tool:\n\n    go tool pprof cpu.tar.gz\n\n## Ideas\n\nSome ideas may or may not be implemented in the future\n\n- Add transparent and half-transparent mode which just handles the map/reduce without cache.\n  This is implemented for log table and commands anyway already. Just requires an additional header.\n- Cache last 24h of logfiles to speed up most logfile requests\n- Fix updating comments (takes too long after sending commands)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsni%2Flmd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsni%2Flmd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsni%2Flmd/lists"}