{"id":24552419,"url":"https://github.com/peschuster/logmonitor","last_synced_at":"2025-10-04T03:30:47.113Z","repository":{"id":2929604,"uuid":"3940872","full_name":"peschuster/LogMonitor","owner":"peschuster","description":"Monitoring log files on windows systems.","archived":false,"fork":false,"pushed_at":"2016-10-18T17:49:45.000Z","size":4820,"stargazers_count":23,"open_issues_count":0,"forks_count":18,"subscribers_count":4,"default_branch":"master","last_synced_at":"2023-03-20T15:16:17.009Z","etag":null,"topics":["daemon","graphite","logs","metrics","monitoring","service","statsd"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/peschuster.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-04-05T15:02:42.000Z","updated_at":"2021-01-21T14:53:33.000Z","dependencies_parsed_at":"2022-08-25T17:00:25.072Z","dependency_job_id":null,"html_url":"https://github.com/peschuster/LogMonitor","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peschuster%2FLogMonitor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peschuster%2FLogMonitor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peschuster%2FLogMonitor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peschuster%2FLogMonitor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/peschuster","download_url":"https://codeload.github.com/peschuster/LogMonitor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235213143,"owners_count":18953728,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["daemon","graphite","logs","metrics","monitoring","service","statsd"],"created_at":"2025-01-23T01:31:31.684Z","updated_at":"2025-10-04T03:30:39.756Z","avatar_url":"https://github.com/peschuster.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LogMonitor\n\n*LogMonitor* is a combination of *logtail* and [*logster*](https://github.com/etsy/logster), but specifically designed for windows systems. It watches directories or files for changes, extracts metrics from the added content and submits these metrics to a configured backend (*graphite*, *statsd* or *console*).\n\nExtracting metrics from added content is done by *PowerShell* scripts, which can be created or customized at will.\n\n## Documentation\n\n![Architecture of LogMonitor](https://raw.github.com/peschuster/LogMonitor/master/docs/LogMonitorArchitecture.png)\n\n*LogMonitor* consists of three components:\n\n 1. Watching files or directories for changes.\n 2. Parsing added content.\n 3. Submitting metrics to backends.\n\nEvery component can be configured seperately and plugged together at will. Therefore you could e.g. watch two directories for changes, process all changes through the same parsers, but submit changes of the directories to seperat backends. *Mapping* between the components is always performed by regular expressions on the file names of the read changes.\n\n## Running *LogMonitor*\n\n*LogMonitor* can run in a console window (simply start `LogMonitor.exe`) or as a windows service.\n\nTo install *LogMonitor* as a Windows Service execute the following command:\n\n    C:\\Windows\\Microsoft.NET\\Framework\\v4.0.30319\\installutil.exe LogMonitor.exe\n\nAnd start the *LogMonitor* service with the command\n\n    net start LogMonitor\n\n## Configuration\n\nConfiguration is done completely through the `App.config` (i.e. `LogMonitor.config`) file. An example is available at [source/LogMonitor/App.config](https://github.com/peschuster/LogMonitor/blob/master/source/LogMonitor/App.config)\n\nAdd the configuration section to your `app.config` file:\n\n    \u003cconfigSections\u003e\n      \u003csection name=\"logMonitor\" type=\"LogMonitor.Configuration.LogMonitorConfiguration, LogMonitor\"/\u003e\n    \u003c/configSections\u003e\n\n### Watched directories\n\n    \u003cwatch\u003e\n      \u003cadd path=\"C:\\inetpub\\logs\\LogFiles\\W3SVC1\" type=\"w3c\" filter=\"*.log\" maxDaysInactive=\"2\" /\u003e\n    \u003c/watch\u003e\n\n**Options:**\n\n * `path` - Path to a directory or file.\n * `filter` - *[optional]*, default: \"*\". Filter for watched files in directory.\n * `type` - *[optional]*, if `w3c` all lines are split in respective fields. Powershell scripts are provided with an `W3CChange` object.\n * `maxDaysInactive` - *[optional]*, default: -1. If set, only files with activity during the last `x` days (configured value) are watched for changes.\n * `bufferTime` - *[optional]*, default: 500. Time (in ms) for which events by the file system are buffered (i.e. aggregated) before processing.\n * `intervalTime` - *[optional]*, default: 5000. Interval (in ms) of manual checks for changed files (in case file system raised no events).\n\n### Parser for metrics\n\n    \u003cparser\u003e\n      \u003cadd scriptPath=\".\\Scripts\\CallCountProcessor.ps1\" pattern=\"\\.log$\" /\u003e\n      \u003cadd scriptPath=\".\\Scripts\\TimeTakenProcessor.ps1\" pattern=\"\\.log$\" /\u003e\n      \u003cadd scriptPath=\".\\Scripts\\HttpStatusProcessor.ps1\" pattern=\"\\.log$\" /\u003e\n    \u003c/parser\u003e\n\n**Options:**\n\n * `scriptPath` - Path to *PowerShell* script.\n * `pattern` - RegEx pattern applied to log file names, to map parsers to specific files, file types or locations.\n\n### Output backends\n\n    \u003coutput\u003e\n      \u003cadd pathPattern=\"w3SVC7\" type=\".\" target=\"statsd\" metricsPrefix=\"mysite1\" /\u003e\n      \u003cadd pathPattern=\"w3SVC6\" type=\".\" target=\"statsd\" metricsPrefix=\"mysite2\" /\u003e\n      \u003cadd pathPattern=\"w3SVC5\" type=\".\" target=\"statsd\" metricsPrefix=\"mysite3\" /\u003e\n    \u003c/output\u003e\n\n**Options:**\n\n * `pathPattern` - RegEx pattern applied to log file names, to map output backends to specific files, file types or locations.\n * `type` - RegEx pattern to filter metrics by type.\n * `target` - Name of the backend (`graphite`, `statsd` or `console`).\n * `metricsPrefix` - Prefix applied to all metrics, before sending them to the backend.\n\n### Additional configuration\n\nThe *graphite* and *statsd* backends need extra configuration:\n\n    \u003cconfigSections\u003e\n      \u003csection name=\"graphite\" type=\"Graphite.Configuration.GraphiteConfiguration, Graphite\" /\u003e\n    \u003c/configSections\u003e\n    \u003cgraphite xmlns=\"http://github.com/peschuster/Graphite/Configuration\"\u003e\n      \u003cgraphite address=\"127.0.0.1\" port=\"2003\" transport=\"Tcp\" /\u003e\n      \u003cstatsd address=\"127.0.0.1\" port=\"8125\" prefixKey=\"test\" /\u003e\n    \u003c/graphite\u003e\n\n*LogMonitor* uses the graphite base library for these backends: [Graphite](https://github.com/peschuster/graphite-client)\n\nTo see/store log messages you can simply add a *trace listener* in your configuration:\n\n    \u003csystem.diagnostics\u003e\n      \u003ctrace autoflush=\"true\" indentsize=\"4\"\u003e\n        \u003clisteners\u003e\n          \u003cadd name=\"configConsoleListener\" type=\"System.Diagnostics.ConsoleTraceListener\" /\u003e\n        \u003c/listeners\u003e\n      \u003c/trace\u003e\n    \u003c/system.diagnostics\u003e\n\n## Writing *PowerShell* script parsers\n\nA *PowerShell* script for extracting metrics from added lines must always contain a function with the following signature:\n\n    Function MetricProcessor ([LogMonitor.FileChange] $change)\n    {\n    }\n\nIt also must always return a list of `LogMonitor.Metric` objects. Objects of this type can be created by calling `[LogMonitor.Metric]::Create(..)`.\n\nHere is an example for simply returning the number of added lines:\n\n    $metrics = @()\n        \n    if ($change.GetType().FullName -eq \"LogMonitor.Processors.W3CChange\")\n    {\n\t\t$metrics += [LogMonitor.Metric]::Create('calls', $change.Values.Count, [LogMonitor.MetricType]::Counter)\n    }\n        \n    return $metrics\n\nThis example also checks for the type `LogMonitor.Processors.W3CChange` which is passed to the *PowerShell* script for log files of type `w3c`.\n\n## Planned features\n\n- Command line interface for reading historical data.\n\n## Build\n\nHow to build *LogMonitor*:\n \n1. Go to `\\build\\` directory\n2. Execute `go.bat`","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeschuster%2Flogmonitor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeschuster%2Flogmonitor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeschuster%2Flogmonitor/lists"}