{"id":21011755,"url":"https://github.com/reyjrar/parse-syslog-line","last_synced_at":"2025-05-15T04:32:28.351Z","repository":{"id":56840684,"uuid":"2020044","full_name":"reyjrar/Parse-Syslog-Line","owner":"reyjrar","description":"Flexible library for parsing syslog messages in Perl","archived":false,"fork":false,"pushed_at":"2024-06-20T21:10:48.000Z","size":258,"stargazers_count":10,"open_issues_count":1,"forks_count":9,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-06-21T15:14:19.345Z","etag":null,"topics":["logging","perl"],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reyjrar.png","metadata":{"files":{"readme":"README.mkdn","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2011-07-08T21:04:56.000Z","updated_at":"2024-06-20T21:10:50.000Z","dependencies_parsed_at":"2024-06-21T14:02:54.428Z","dependency_job_id":null,"html_url":"https://github.com/reyjrar/Parse-Syslog-Line","commit_stats":{"total_commits":149,"total_committers":8,"mean_commits":18.625,"dds":"0.12751677852348997","last_synced_commit":"b899a60cf465ac19c3576861923988611ad07303"},"previous_names":[],"tags_count":51,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reyjrar%2FParse-Syslog-Line","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reyjrar%2FParse-Syslog-Line/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reyjrar%2FParse-Syslog-Line/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reyjrar%2FParse-Syslog-Line/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reyjrar","download_url":"https://codeload.github.com/reyjrar/Parse-Syslog-Line/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225326489,"owners_count":17456954,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["logging","perl"],"created_at":"2024-11-19T09:31:02.046Z","updated_at":"2024-11-19T09:31:20.634Z","avatar_url":"https://github.com/reyjrar.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NAME\n\nParse::Syslog::Line - Simple syslog line parser\n\n# VERSION\n\nversion 5.3\n\n# SYNOPSIS\n\nI wanted a very simple log parser for network based syslog input.\nNothing existed that simply took a line and returned a hash ref all\nparsed out.\n\n    use Parse::Syslog::Line qw(parse_syslog_line);\n\n    $Parse::Syslog::Line::DateTimeCreate = 1;\n    $Parse::Syslog::Line::AutoDetectJSON = 1;\n\n    my $href = parse_syslog_line( $msg );\n    #\n    # $href = {\n    #       preamble        =\u003e '13',\n    #       priority        =\u003e 'notice',\n    #       priority_int    =\u003e 5,\n    #       facility        =\u003e 'user',\n    #       facility_int    =\u003e 8,\n    #       date            =\u003e 'YYYY-MM-DD',\n    #       time            =\u003e 'HH::MM:SS',\n    #       epoch           =\u003e 1361095933,\n    #       datetime_str    =\u003e ISO 8601 datetime, $NormalizeToUTC = 1 then UTC, else local\n    #       datetime_obj    =\u003e undef,       # If $DateTimeCreate = 1, else undef\n    #       datetime_raw    =\u003e 'Feb 17 11:12:13'\n    #       date_raw        =\u003e 'Feb 17 11:12:13'\n    #       host_raw        =\u003e 'hostname',  # Hostname as it appeared in the message\n    #       host            =\u003e 'hostname',  # Hostname without domain\n    #       domain          =\u003e 'blah.com',  # if provided\n    #       program_raw     =\u003e 'sshd(blah)[pid]',\n    #       program_name    =\u003e 'sshd',\n    #       program_sub     =\u003e 'pam_unix',\n    #       program_pid     =\u003e 20345,\n    #       content         =\u003e 'the rest of the message'\n    #       message         =\u003e 'program[pid]: the rest of the message',\n    #       message_raw     =\u003e 'The message as it was passed',\n    #       ntp             =\u003e 'ok',        # Only set for Cisco messages\n    #       version         =\u003e 1,\n    #       SDATA           =\u003e { ... },     # RFC Structured data, decoded JSON, or K/V Pairs in the message\n    # };\n    ...\n\n# EXPORT\n\nExported by default:\n       parse\\_syslog\\_line( $one\\_line\\_of\\_syslog\\_message );\n\nOptional Exports:\n  :preamble\n       preamble\\_priority\n       preamble\\_facility\n\n    :constants\n         %LOG_FACILITY\n         %LOG_PRIORITY\n\n    :with_timezones\n         set_syslog_timezone\n         get_syslog_timezone\n         use_utc_syslog\n\n# VARIABLES\n\n## ExtractProgram\n\nIf this variable is set to 1 (the default), parse\\_syslog\\_line() will try it's\nbest to extract a \"program\" field from the input.  This is the most expensive\nset of regex in the module, so if you don't need that pre-parsed, you can speed\nthe module up significantly by setting this variable.\n\nVendors who do proprietary non-sense with their syslog formats are to blame for\nthis setting.\n\nUsage:\n\n    $Parse::Syslog::Line::ExtractProgram = 0;\n\n## DateParsing\n\nIf this variable is set to 0 raw date will not be parsed further into\ncomponents (datetime\\_str date time epoch).  Default is 1 (parsing enabled).\n\nUsage:\n\n    $Parse::Syslog::Line::DateParsing = 0;\n\n## DateTimeCreate\n\nIf this variable is set to 1 (the default), a DateTime object will be returned in the\n$m-\u003e{datetime\\_obj} field.  Otherwise, this will be skipped.\n\nNOTE: DateTime timezone calculation is fairly slow. Unless you really need to\ntake timezones into account, you're better off using other modes (below).\n\nUsage:\n\n    $Parse::Syslog::Line::DateTimeCreate = 0;\n\n## EpochCreate\n\nIf this variable is set to 1, the default, the number of seconds from UNIX\nepoch will be returned in the $m-\u003e{epoch} field.  Setting this to false will\nonly delete the epoch before returning the hash reference.\n\n## NormalizeToUTC\n\nWhen set, the datetime\\_str will be ISO8601 UTC.\n\n## OutputTimeZones\n\nDefault is false, but is enabled if you call set\\_syslog\\_timezone() or\nuse\\_utc\\_syslog().  If enabled, this will append the timezone offset to the\ndatetime\\_str.\n\n## FmtDate\n\nYou can pass your own formatter/parser here. Given a raw datetime string it\nshould output a list containing date, time, epoch, datetime\\_str,\nin your wanted format.\n\n    use Parse::Syslog::Line;\n\n    local $Parse::Syslog::Line::FmtDate = sub {\n        my ($raw_datestr) = @_;\n        my @elements = (\n            #date\n            #time\n            #epoch\n            #datetime_str\n        );\n        return @elements;\n    };\n\n**NOTE**: No further date processing will be done, you're on your own here.\n\n## HiResFmt\n\nDefault is `%0.6f`, or microsecond resolution.  This variable only comes into\nplay when the syslog date string contains a high resolution timestamp.  It\ndefaults to using microsecond resolution.\n\n## AutoDetectJSON\n\nDefault is false.  If true, we'll autodetect the presence of JSON in the syslog\nmessage and use [JSON::MaybeXS](https://metacpan.org/pod/JSON%3A%3AMaybeXS) to decode it.  The detection/decoding is\nsimple.  If a '{' is detected, everything until the end of the message is\nassumed to be JSON.  The decoded JSON will be added to the `SDATA` field.\n\n    $Parse::Syslog::Line::AutoDetectJSON = 1;\n\n## AutoDetectKeyValues\n\nDefault is false.  If true, we'll autodetect the presence of Splunk style\nkey/value pairds in the message stream.  That format is `k1=v1, k2=v2`.\nResulting K/V pairs will be added to the `SDATA` field.\n\n    $Parse::Syslog::Line::AutoDetectKeyValues = 1;\n\n## RFC5424StructuredData\n\nDefault is true.  When enabled, this will extract the RFC standard structured data\nfrom the message content.  That content will be stripped from the message\n`content` field.\n\nSome examples:\n\n    # Input\n    [foo x=1] some words [bar x=2]\n\n    # To (YAML for brevity)\n    ---\n    SDATA:\n      bar:\n        x: 2\n      foo:\n        x: 1\n    content: some words\n\n    # Input\n    [x=1] some words\n\n    # To (YAML for brevity)\n    ---\n    SDATA:\n      x: 1\n    content: some words\n\nTo disable:\n\n    $Parse::Syslog::Line::RFC5424StructuredData = 0;\n\n## RFC5424StructuredDataStrict\n\nRequire the format:\n\n    [namespace@id property=\"value\"][namespace@id property=\"value\"]\n\nDefaults to 0, set to 1 to only parse the RFC5424 formatted structured data.\n\n## PruneRaw\n\nThis variable defaults to 0, set to 1 to delete all keys in the return hash\nending in \"\\_raw\"\n\nUsage:\n\n    $Parse::Syslog::Line::PruneRaw = 1;\n\n## PruneEmpty\n\nThis variable defaults to 0, set to 1 to delete all keys in the return hash\nwhich are undefined.\n\nUsage:\n\n    $Parse::Syslog::Line::PruneEmpty = 1;\n\n## PruneFields\n\nThis should be an array of fields you'd like to be removed from the hash reference.\n\nUsage:\n\n    @Parse::Syslog::Line::PruneFields = qw(date_raw facility_int priority_int);\n\n# FUNCTIONS\n\n## parse\\_syslog\\_line\n\nReturns a hash reference of syslog message parsed data.\n\n**NOTE**: Date/time parsing is hard.  This module has been optimized to balance\ncommon sense and processing speed. Care is taken to ensure that any data input\ninto the system isn't lost, but with the varieties of vendor and admin crafted\ndate formats, we don't always get it right.  Feel free to override date\nprocessing using by setting the $FmtDate variable or completely disable it with\n$DateParsing set to 0.\n\n## set\\_syslog\\_timezone($timezone\\_name)\n\nSets a timezone $timezone\\_name for parsed messages. This timezone will be used\nto calculate offset from UTC if a timezone designation is not present in the\nmessage being parsed.  This timezone will also serve as the source timezone for\nthe datetime\\_str field.\n\n## get\\_syslog\\_timezone\n\nReturns the name of the timezone currently set by set\\_syslog\\_timezone.\n\n## use\\_utc\\_syslog\n\nA convenient function which sets the syslog timezone to UTC and sets the config\nvariables accordingly.  Automatically sets $NormaizeToUTC and datetime\\_str will\nbe set to the UTC equivalent.\n\n## parse\\_syslog\\_lines\n\nReturns a list of hashes of the lines interpretted.\n\nWhen passed one or more line of text, attempts to parse that text as syslog data.  This function\nvaries from `parse_syslog_line` in that it handles multi-line messages.  The caveat to this, is\nafter the last iteration of the loop, you to call the function by itself to get the last message.\n\n    use strict;\n    use warnings;\n    use DDP;\n    use Parse::Syslog::Line qw(parse_syslog_lines);\n\n    while(\u003c\u003e) {\n        foreach my $log ( parse_syslog_lines($_) ) {\n            p($log);\n        }\n    }\n    p($_) for parse_syslog_lines();\n\nThis function holds a parsing buffer which it flushes any time it encounters a\nline in the stream that starts with non-whitespace.  Any lines beginning with\nwhitespace will be assumed to be a continuation of the previous line.\n\nIt is not exported by default.\n\n## preamble\\_priority\n\nTakes the Integer portion of the syslog messsage and returns\na hash reference as such:\n\n    $prioRef = {\n        'preamble'  =\u003e 13\n        'as_text'   =\u003e 'notice',\n        'as_int'    =\u003e 5,\n    };\n\n## preamble\\_facility\n\nTakes the Integer portion of the syslog messsage and returns\na hash reference as such:\n\n    $facRef = {\n        'preamble'  =\u003e 13\n        'as_text'   =\u003e 'user',\n        'as_int'    =\u003e 8,\n    };\n\n# ENVIRONMENT VARIABLES\n\nThere are environment variables that affect how we operate. They are not\noptions as they are not intended to be used by our users. Use at your own risk.\n\n## PARSE\\_SYSLOG\\_LINE\\_DEBUG\n\nOutputs debugging information about the parser, not really intended for end-users.\n\n## PARSE\\_SYSLOG\\_LINE\\_QUIET\n\nDisables warnings in the parse\\_syslog\\_line() function\n\n## TEST\\_ACTIVE / TEST2\\_ACTIVE\n\nDisables warnings in the parse\\_syslog\\_line() function\n\n# DEVELOPMENT\n\nThis module is developed with Dist::Zilla.  To build from the repository, use Dist::Zilla:\n\n    dzil authordeps --missing |cpanm\n    dzil listdeps --missing |cpanm\n    dzil build\n    dzil test\n\n# AUTHOR\n\nBrad Lhotsky \u003cbrad@divisionbyzero.net\u003e\n\n# COPYRIGHT AND LICENSE\n\nThis software is Copyright (c) 2017 by Brad Lhotsky.\n\nThis is free software, licensed under:\n\n    The (three-clause) BSD License\n\n# CONTRIBUTORS\n\n- Bartłomiej Fulanty \u003cstarlight@cpan.org\u003e\n- Csillag Tamas \u003ccstamas@digitus.itk.ppke.hu\u003e\n- Keedi Kim \u003ckeedi.k@gmail.com\u003e\n- Mateu X Hunter \u003cmhunter@maxmind.com\u003e\n- Neil Bowers \u003cneil@bowers.com\u003e\n- Shawn Wilson \u003cswilson@korelogic.com\u003e\n- Tomohiro Hosaka \u003cbokutin@bokut.in\u003e\n\n# SUPPORT\n\n## Websites\n\nThe following websites have more information about this module, and may be of help to you. As always,\nin addition to those websites please use your favorite search engine to discover more resources.\n\n- MetaCPAN\n\n    A modern, open-source CPAN search engine, useful to view POD in HTML format.\n\n    [https://metacpan.org/release/Parse-Syslog-Line](https://metacpan.org/release/Parse-Syslog-Line)\n\n## Bugs / Feature Requests\n\nThis module uses the GitHub Issue Tracker: [https://github.com/reyjrar/Parse-Syslog-Line/issues](https://github.com/reyjrar/Parse-Syslog-Line/issues)\n\n## Source Code\n\nThis module's source code is available by visiting:\n[https://github.com/reyjrar/Parse-Syslog-Line](https://github.com/reyjrar/Parse-Syslog-Line)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freyjrar%2Fparse-syslog-line","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freyjrar%2Fparse-syslog-line","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freyjrar%2Fparse-syslog-line/lists"}