{"id":28233495,"url":"https://github.com/logdna/winevt-tailer","last_synced_at":"2025-06-14T23:30:36.333Z","repository":{"id":63635355,"uuid":"543322919","full_name":"logdna/winevt-tailer","owner":"logdna","description":"Windows Event Log Tailer -  provides live tailing of Windows events to standard output or log file. Can run as console application or Windows service.  Distributed as signed standalone executable.","archived":false,"fork":false,"pushed_at":"2023-12-24T09:12:41.000Z","size":435,"stargazers_count":2,"open_issues_count":4,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-03-26T23:00:37.564Z","etag":null,"topics":["events","logdna-agent","logging","logs","mezmo","python","tail","tailer","windows"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/logdna.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-09-29T21:18:53.000Z","updated_at":"2023-11-09T15:02:35.000Z","dependencies_parsed_at":"2022-11-22T22:25:00.137Z","dependency_job_id":"9f9bfb25-3114-4c5d-a3e1-6181cbea4dd5","html_url":"https://github.com/logdna/winevt-tailer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logdna%2Fwinevt-tailer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logdna%2Fwinevt-tailer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logdna%2Fwinevt-tailer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logdna%2Fwinevt-tailer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/logdna","download_url":"https://codeload.github.com/logdna/winevt-tailer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254950715,"owners_count":22153379,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["events","logdna-agent","logging","logs","mezmo","python","tail","tailer","windows"],"created_at":"2025-05-18T21:10:10.868Z","updated_at":"2025-06-14T23:30:36.327Z","avatar_url":"https://github.com/logdna.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# winevt-tailer\nWindows Event Log Tailer provides live tailing of Windows events to standard output when running as console application or to log file when running as a service. It is written in Python, and is MIT licensed open source.\n\n\u003c!-- TOC --\u003e\n* [Features](#features)\n* [Installation](#installation)\n* [Getting Started](#getting-started)\n  * [Console mode](#console-mode)\n  * [Service mode](#service-mode)\n* [Advanced Usage](#advanced-usage)\n  * [CLI Options](#cli-options)\n  * [Configuration File](#configuration-file)\n  * [Event Channels and XPath queries](#event-channels-and-xpath-queries)\n  * [Environment vars](#environment-vars)\n  * [Event Transforms](#event-transforms)\n* [How to Build](#how-to-build)\n* [Integration with Mezmo Agent](#integration-with-mezmo-agent)\n\u003c!-- TOC --\u003e\n\n## Features\n\n- Live tail - following new events\n- Lookback\n- XPath queries\n- Event Transforms - applying user defined transformations to events\n- Keeping track of last tailed events - maintaining persistent state between runs\n- Windows service mode with self install and uninstall\n- Custom tail output using standard logging framework\n- Using fast native libxml2 (lxml) for XML parsing and XSLT transforms (JSON assembly)\n- Integration with Mezmo Agent\n\n## Installation\n\nTailer is distributed as signed standalone executable available for download from [Releases](https://github.com/logdna/winevt-tailer/releases).\n\n## Getting Started\n\n### Console mode\n\nTo tail last 100 events from Application and last 100 events from System event logs:\n\n```\nwinevt-tailer\n```\n\nTailer will output each event log message to stdout as utf-8 encoded single-line JSON. New line, '\\' and '\"' chars in JSON field values are escaped with '\\'.\n\nTo tail last 10 events and to follow new events:\n\n```\nwinevt-tailer -f -b 10\n```\n\n\n### Service mode\n\nTo install Tailer as Windows service:\n\n```winevt-tailer -i```\n\nor\n\n```winevt-tailer -i \u003cCLI options\u003e```\n\nService defaults:\n- service name: ```winevt-tailer_\u003ctailer_name\u003e```\n- tailer name: ```tail1```, see \"-n\" option\n\nFunctionally this service will be equivalent to CLI mode: ```winevt-tailer \u003cCLI options\u003e```. To change service CLI options - just run the same \"-i\" command again with new set of options. Persistent state \"-p\" and follow mode \"-f\" are enabled by default in service mode.\n\nTailer logs by default are stored in ```c:/ProgramData/logs```:\n\n```\nwindows_tail1.log           -- Windows events in one-line-JSON format, ready to be streamed by Mezmo Agent\nwinevt-tailer_tail1.log     -- service instance log\n```\n\nThe location can be changed in config file in ```winevt-tailer.logging``` section.\n\nTo uninstall the service:\n\n```winevt-tailer -u```\n\n\n## Advanced Usage\n\n### CLI Options\n\n```\n\u003e winevt-tailer.exe -h\n\nusage: winevt-tailer.exe [-v | -h | -l | -e | -i | -u | -r] [-f] [-p] [-c filepath] [-n NAME] [-b LOOKBACK] [--tailer_config TAILER_CONFIG] [--logging_config LOGGING_CONFIG]\n                         [-t TRANSFORMS_PATH] [-s]\n\nTail Windows Event logs using single-line JSON format\n\noptions:\n  -v, --version         Show program version info and exit.\n  -h, --help            Show this help message and exit.\n  -l, --list            List event channel names accessible to current user. Some channels may need Admin rights.\n  -e, --print_config    Print effective config and exit.\n  -i, --install_service\n                        Install windows service.\n  -u, --uninstall_service\n                        Uninstall windows service.\n  -r, --reset           Reset persistent state - delete event bookmarks.\n  -f, --follow          Follow and output new events as they arrive. True in service mode.\n  -p, --persistent      Remember last tailed event for each channel and tail only new events after restart. Default: off\n  -c filepath, --config filepath\n                        Config file path, file format: YAML\n  -n NAME, --name NAME  Tailer name. Also defines where to look for config: winevt-tailer/\u003cname\u003e in YAML file; TAILER_CONFIG_\u003cname\u003e and TAILER_LOGGING_\u003cname\u003e in env vars (as YAML string)\n  -b LOOKBACK, --lookback LOOKBACK\n                        Defines how many old events to tail. -1 means all available events. default is 100. Applied in non-persistent mode or when event channel persistent state was not\n                        stored.\n  --tailer_config TAILER_CONFIG\n                        Named tailer config section as YAML string\n  --logging_config LOGGING_CONFIG\n                        Logging config section as YAML string\n  -t TRANSFORMS_PATH, --transforms_path TRANSFORMS_PATH\n                        Path to custom transforms\n  -s, --startup_hello   Output Startup Hello line. Part of Mezmo Agent Tailer API. Default: off\n ```\n\n### Configuration File\n\nTailer accepts configuration file name in \"-c\" option. The file format is YAML. Use \"-e\" option to dump effective config to a file and then use it as configuration file:\n\n```\nwinevt-tailer -f -b 10 -e \u003e config.yaml\n```\n\nThen you can start Tailer with generated config file:\n\n```\nwinevt-tailer -c config.yaml\n```\n\nwhich will be equivalent to running \"winevt-tailer -f -b 10\".\n\nConfiguration file structure:\n\n```\nwinevt-tailer:\n    logging:\n      \u003cstandard python logging config\u003e\n    tail1:\n      \u003cnamed tailer config\u003e\n```\n\nNamed tailer config section corresponds to tailer name specified in \"-n\" option, default is \"tail1\". Configuration file can have multiple named tailer configs. \nWhen tailer service starts it prints effective config to service log file. Default service log file: ```c:\\ProgramData\\logs\\winevt-tailer_tail1.log```.\nIn service mode persistent state \"-p\" and follow mode \"-f\" are enabled by default.\n\n### Event Channels and XPath queries\n\nTailer supports up to 64 event channels with optional individual custom filters using the same XPath syntax used in Windows Event Viewer.\nDefault event channels: ```Application, System```. Channels are defined in named tailer config section. Output from \"winevt-tailer -e\":\n\n```\nwinevt-tailer:\n    tail1:\n        bookmarks_dir: .\n        channels:\n        -   name: Application           \u003c\u003c\u003c\u003c Channel name\n            query: '*'                  \u003c\u003c\u003c\u003c XPath query\n        -   name: System\n            query: '*'\n```\n\nThe same channel name can be used multiple times with different query filters. This may create duplicate events if channels queries produce overlapped content.\n\nWindows event Viewer can be used to create channel filter:\n\n![image](https://user-images.githubusercontent.com/7530150/204931587-6e2e045d-e7fe-402f-97b3-1fc44e26da0b.png)\n\nthen switch to XML view of the filter and extract XPath query (highlighted):\n\n![image](https://user-images.githubusercontent.com/7530150/204931845-68af1728-38fa-4549-9d65-30582e533681.png)\n\nUse extracted XPath string as query value in tailer config file:\n```\nwinevt-tailer:\n    tail1:\n        bookmarks_dir: .\n        channels:\n        -   name: Application\n            query: '*[System[(EventID=4098)]]'\n```\n\n\n### Environment vars\n\nNamed tailer config and logging config sections can be passed in environment vars (as minified one-line-yaml string):\n\n```\n  TAILER_CONFIG                     - content of named tailer config section\n  TAILER_CONFIG_\u003ctailer_name\u003e       - overrides TAILER_CONFIG\n\n  TAILER_LOGGING                    - content of logging section\n  TAILER_LOGGING_\u003ctailer_name\u003e      - overrides TAILER_LOGGING\n```\n\nEnvironment vars override config file. CLI options override environment vars and config file.\n\n\n### Event Transforms\n\nTailer event processing pipeline:\n\n```\n  Event Channel -\u003e Event XML Object -\u003e Transforms (channel level) -\u003e Transforms (top level) -\u003e XML Object to JSON -\u003e Output\n```\n\nTransforms can be configured at two levels:\n\n- channel      - applied first, channel specific \n- top level    - applied at the end, common transforms for all channels\n\nExample:\n\n```\nwinevt-tailer:\n    tail1:\n        transforms:                                       \u003c\u003c\u003c\u003c top level\n        - winevt_tailer.transforms.xml_remove_binary\n        - winevt_tailer.transforms.xml_render_message\n        - winevt_tailer.transforms.xml_to_json\n        channels:\n        - name: Application\n          query: '*'\n          transforms:                                     \u003c\u003c\u003c\u003c channel level \n          - my_transform.custom_channel_specific\n```\n\nTransform name is Python object importable full dotted path that used in import statements. Transforms are applied in the listed order. Tailer supports adding custom transforms defined in external py module file.\n\nHere's a working example of event *deduplication transform* or reduction filter that skips subsequent events that have the same text in Message tag:\n\n```\nfrom lxml import etree\nimport win32evtlog\n\n\ndef dedup_by_message(context: dict, event_h, event_obj: object) -\u003e object:\n    \"\"\"\n    This transform implements simple deduplication based on rendered Message field by transforms.xml_render_message\n    xform. It uses context object to store last event.\n    Args:\n        context: a dictionary that can be used to store data that preserved between calls\n        event_h: event handle returned from win32evtlog API\n        event_obj:  lxml.etree - parsed XMl object\n    Returns:\n        None - to skip event, otherwise unmodified original event obj\n    \"\"\"\n    my_context = context.get(\"dedup_by_message\")\n    if my_context is None:\n        context[\"dedup_by_message\"] = {}\n        return event_obj  # No skip, 1st event\n    try:\n        new_msg = etree.SubElement(event_obj, 'Message')\n        last_event = my_context.get(\"last_event\")\n        if last_event is not None:\n            last_msg = etree.SubElement(last_event, 'Message')\n            if last_msg.text == new_msg.text:\n                return None  # Skip, same message\n    except Exception:\n        # pywintypes.error: (2, 'EvtOpenPublisherMetadata', 'The system cannot find the file specified.')\n        pass\n    finally:\n        my_context[\"last_event\"] = event_obj\n    return event_obj  # No skip\n```\n\nSave it to ```my_transforms.py```. Create default config file and add new transform function dot path ```my_transforms.dedup_by_message``` after ```xml_render_message```:\n\n```\nwinevt-tailer:\n    tail1:\n      transforms:\n      - winevt_tailer.transforms.xml_remove_binary\n      - winevt_tailer.transforms.xml_render_message\n      - my_transforms.dedup_by_message                   \u003c\u003c\u003c\u003c\u003c\u003c\u003c\u003c\u003c the deduping transform\n      - winevt_tailer.transforms.xml_to_json\n```\n\nThen run Tailer:\n\n```\nwinevt-tailer -f -c config.yaml\n```\n\nFor other examples see built-in transforms in: [winevt_tailer/transforms.py](winevt_tailer/transforms.py)\n\nIn CLI mode Tailer is looking for transforms in current working directory, in service mode - in the location of winevt-tailer.exe. Transforms path can also be specified using \"-t\" option. \n\n\n## How to Build\n\nRecommended development setup:\n- install [miniconda](https://repo.anaconda.com/miniconda/Miniconda3-py38_4.12.0-Windows-x86_64.exe)\n- start conda shell\n- create env\n\n```\nconda create -n work -c conda-forge python=3.10 git make\nconda activate work\npip install poetry\n```\n\n- build exe\n\n```\nmake clean\nmake lint\nmake test\nmake build\n```\n\nbuild will create ```winevt-tailer.exe``` in folder ```build/dist```\n\n- test executable\n\n```\nbuild/dist/winevt-tailer.exe\n```\n\n\n## Integration with Mezmo Agent\n\nTailer can be used with [Mezmo Agent](https://github.com/logdna/logdna-agent-v2) to stream log files to [Mezmo.com](https://www.mezmo.com). Just install Tailer as service and then install [Mezmo Agent for Windows](https://community.chocolatey.org/packages/mezmo-agent). More tight integration with Mezmo Agent using Agent Tailer API (IPC) will be available in next Agent release.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flogdna%2Fwinevt-tailer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flogdna%2Fwinevt-tailer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flogdna%2Fwinevt-tailer/lists"}