{"id":20504076,"url":"https://github.com/icinga/logstash-grok-pattern","last_synced_at":"2026-03-08T19:34:54.942Z","repository":{"id":145047583,"uuid":"87292853","full_name":"Icinga/logstash-grok-pattern","owner":"Icinga","description":"Pattern for the Logstash filter Grok","archived":false,"fork":false,"pushed_at":"2017-06-14T07:20:31.000Z","size":4,"stargazers_count":7,"open_issues_count":0,"forks_count":3,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-01-16T07:37:33.980Z","etag":null,"topics":["grok","grok-filter","icinga","logstash","parse-logs"],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Icinga.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-05T09:43:16.000Z","updated_at":"2018-09-17T07:35:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"32c791f6-240c-4ef4-8fab-499737b974e5","html_url":"https://github.com/Icinga/logstash-grok-pattern","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Icinga%2Flogstash-grok-pattern","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Icinga%2Flogstash-grok-pattern/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Icinga%2Flogstash-grok-pattern/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Icinga%2Flogstash-grok-pattern/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Icinga","download_url":"https://codeload.github.com/Icinga/logstash-grok-pattern/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242100931,"owners_count":20071823,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grok","grok-filter","icinga","logstash","parse-logs"],"created_at":"2024-11-15T19:36:14.126Z","updated_at":"2026-03-08T19:34:54.914Z","avatar_url":"https://github.com/Icinga.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Logstash Grok Pattern for Icinga\n\n[Logstash](https://www.elastic.co/products/logstash) is a data processing\npipeline that processes data. It can receive, collect, parse, transform and\nforward log events. This repository includes various\npattern for the Logstash filter [grok](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html).\n\nThe grok filter is included in a default Logstash installation. It is used to\nparse log events and split messages into multiple fields. Instead of writing\nregular expressions, users use predefined patterns to parse logs. Besides the\nincluded patterns, custom patterns can be added to extend the functionality.\n\n1. [Installation](#installation)\n2. [Examples](#examples)\n    * [Icinga 2 Main Log](#icinga-2-main-log)\n    * [Icinga 2 Debug Log](#icinga-2-debug-log)\n    * [Icinga 2 Startup Log](#icinga-2-startup-log)\n\n## Installation\nCustom patterns need to be accessible by the Logstash daemon. It does not matter\nwhere you put the files on the file system, as long as they are readable to the\n`logstash` user.\n\n```shell\nmkdir /etc/logstash/patterns\ncd /etc/logstash/patterns\ngit clone https://github.com/Icinga/logstash-grok-pattern.git icinga\n```\n\nTo use custom patterns, include the directory with the `patterns_dir` paramter\nin your grok filter:\n\n```ruby\ngrok {\n  patterns_dir   =\u003e [\"/etc/logstash/patterns/icinga\"]\n  ...\n}\n```\n\n## Examples\nThe following examples demonstrate how the patterns can be used to parse Icinga\nlog files. These are just examples, you are free to use the patterns in which\nway you want.\n\n### Icinga 2 Main Log\nThe main log of Icinga 2 includes general information about the behaviour of the\nprocess, each component of it and all enabled features. Logs are split into\nthree fields: `severity`,`facility` and `message`:\n\nThis example is based on a Logstash file input plugin. Other inputs can be used\nas well. The date filter moves the timestamp of the log event to the field\n`@timestamp`, which is used by default in\n[Kibana](https://www.elastic.co/products/kibana) to sort events.\n\n```ruby\ninput {\n  file {\n    path =\u003e \"/var/log/icinga2/icinga2.log\"\n    type =\u003e \"icinga.main\"\n    codec =\u003e multiline {\n      pattern             =\u003e \"^\\[\"\n      negate              =\u003e true\n      what                =\u003e previous\n      auto_flush_interval =\u003e 2\n    }\n  }\n}\n\nfilter {\n  if [type] == \"icinga.main\" {\n    grok {\n      patterns_dir   =\u003e [\"/etc/logstash/patterns/icinga\"]\n      match          =\u003e [\"message\", \"%{ICINGA_MAIN}\"]\n      remove_field   =\u003e [\"message\"]\n      add_tag        =\u003e [\"filter.grok.icinga.main\"]\n      tag_on_failure =\u003e [\"_grokparsefailure\", \"filter.icinga.main.grok.failure\"]\n    }\n\n    date {\n      match          =\u003e [\"icinga.main.timestamp\", \"yyyy-MM-dd HH:mm:ss Z\"]\n      target         =\u003e \"@timestamp\"\n      remove_field   =\u003e [\"icinga.main.timestamp\"]\n      tag_on_failure =\u003e [\"_dateparsefailur\", \"filter.icinga.debug.date.failure\"]\n    }\n  }\n}\n\noutput {\n  stdout {\n    codec =\u003e \"rubydebug\"\n  }\n}\n```\n\n### Icinga 2 Debug Log\nThe main log of Icinga 2 includes very detailed information about the behaviour\nof the process, each component of it and all enabled features. Logs are split\ninto three fields: `severity`,`facility` and `message`:\n\nThis example is based on a Logstash file input plugin. Other inputs can be used\nas well. The date filter moves the timestamp of the log event to the field\n`@timestamp`, which is used by default in\n[Kibana](https://www.elastic.co/products/kibana) to sort events.\n\n```ruby\ninput {\n  file {\n    path =\u003e \"/var/log/icinga2/debug.log\"\n    type =\u003e \"icinga.debug\"\n    codec =\u003e multiline {\n      pattern             =\u003e \"^\\[\"\n      negate              =\u003e true\n      what                =\u003e previous\n      auto_flush_interval =\u003e 2\n    }\n  }\n}\n\nfilter {\n  if [type] == \"icinga.debug\" {\n    grok {\n      patterns_dir   =\u003e [\"/etc/logstash/patterns/icinga\"]\n      match          =\u003e [\"message\", \"%{ICINGA_DEBUG}\"]\n      remove_field   =\u003e [\"message\"]\n      add_tag        =\u003e [\"filter.grok.icinga.debug\"]\n      tag_on_failure =\u003e [\"_grokparsefailure\", \"filter.icinga.debug.grok.failure\"]\n    }\n\n    date {\n      match          =\u003e [\"icinga.debug.timestamp\", \"yyyy-MM-dd HH:mm:ss Z\"]\n      target         =\u003e \"@timestamp\"\n      remove_field   =\u003e [\"icinga.debug.timestamp\"]\n      tag_on_failure =\u003e [\"_dateparsefailur\", \"filter.icinga.debug.date.failure\"]\n    }\n  }\n}\n\noutput {\n  stdout {\n    codec =\u003e \"rubydebug\"\n  }\n}\n```\n\n### Icinga 2 Startup Log\nThe startup log of Icinga 2 is generated each time when the daemon is restarted\nor reloaded. It includes information about the amount of objects on startup,\nwhich features are enabled, connection to the database and suchlike. The startup\nlog does not include a timestamp, the file is rewritten completely every time.\n\nThis example is based on a Logstash file input plugin. Other inputs can be used\nas well.\n\n```shell\ninput {\n  file {\n    path           =\u003e \"/var/log/icinga2/startup.log\"\n    type           =\u003e \"icinga.startup\"\n    start_position =\u003e \"beginning\"\n    sincedb_path   =\u003e \"/dev/null\"\n    codec          =\u003e multiline {\n      pattern             =\u003e \"^[a-z]*\\/[a-zA-Z]*:\"\n      negate              =\u003e true\n      what                =\u003e previous\n      auto_flush_interval =\u003e 2\n    }\n  }\n}\n\nfilter {\n  if [type] == \"icinga.startup\" {\n    grok {\n      patterns_dir   =\u003e [\"/etc/logstash/patterns/icinga\"]\n      match          =\u003e [\"message\", \"%{ICINGA_STARTUP}\"]\n      remove_field   =\u003e [\"message\"]\n      add_tag        =\u003e [\"filter.grok.icinga.startup\"]\n      tag_on_failure =\u003e [\"_grokparsefailure\", \"filter.grok.icinga.startup.failure\"]\n    }\n  }\n}\n\noutput {\n  stdout {\n    codec =\u003e \"rubydebug\"\n  }\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ficinga%2Flogstash-grok-pattern","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ficinga%2Flogstash-grok-pattern","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ficinga%2Flogstash-grok-pattern/lists"}