{"id":13397059,"url":"https://github.com/influxdata/go-syslog","last_synced_at":"2025-03-13T23:32:04.637Z","repository":{"id":48538980,"uuid":"127112666","full_name":"influxdata/go-syslog","owner":"influxdata","description":"Blazing fast syslog parser","archived":true,"fork":false,"pushed_at":"2024-05-21T19:16:51.000Z","size":796,"stargazers_count":478,"open_issues_count":9,"forks_count":68,"subscribers_count":19,"default_branch":"develop","last_synced_at":"2024-08-17T01:10:24.927Z","etag":null,"topics":["builder","logging","logs","parser","ragel","rfc3164","rfc5424","rfc5425","rfc5426","rfc6587","syslog"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/influxdata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-28T08:58:02.000Z","updated_at":"2024-08-05T03:43:41.000Z","dependencies_parsed_at":"2024-01-06T01:03:53.414Z","dependency_job_id":"bdab8a7e-0f27-458f-9346-ef0edaf1266d","html_url":"https://github.com/influxdata/go-syslog","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/influxdata%2Fgo-syslog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/influxdata%2Fgo-syslog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/influxdata%2Fgo-syslog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/influxdata%2Fgo-syslog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/influxdata","download_url":"https://codeload.github.com/influxdata/go-syslog/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243500235,"owners_count":20300760,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["builder","logging","logs","parser","ragel","rfc3164","rfc5424","rfc5425","rfc5426","rfc6587","syslog"],"created_at":"2024-07-30T18:01:10.318Z","updated_at":"2025-03-13T23:32:04.287Z","avatar_url":"https://github.com/influxdata.png","language":"Go","readme":"\u003e [!IMPORTANT]\n\u003e ### `go-syslog` is maintained by the original author at [leodido/go-syslog](https://github.com/leodido/go-syslog)\n\u003e This repository is maintained only as a placeholder; it is not actively developed and may be out of date.\n\n\n[![MIT License](http://img.shields.io/badge/license-MIT-blue.svg?style=for-the-badge)](LICENSE)\n\n**A parser for Syslog messages and transports**.\n\n\u003e [Blazing fast](#Performances) Syslog parsers\n\n_By [@leodido](https://github.com/leodido)_.\n\nTo wrap up, this package provides:\n\n- an [RFC5424-compliant parser and builder](/rfc5424)\n- an [RFC3164-compliant parser](/rfc3164) - ie., BSD-syslog messages\n- a parser that works on streams for syslog with [octet counting](https://tools.ietf.org/html/rfc5425#section-4.3) framing technique, see [octetcounting](/octetcounting)\n- a parser that works on streams for syslog with [non-transparent](https://tools.ietf.org/html/rfc6587#section-3.4.2) framing technique, see [nontransparent](/nontransparent)\n\nThis library provides the pieces to parse Syslog messages transported following various RFCs.\n\nFor example:\n\n- TLS with octet count ([RFC5425](https://tools.ietf.org/html/rfc5425))\n- TCP with non-transparent framing or with octet count ([RFC 6587](https://tools.ietf.org/html/rfc6587))\n- UDP carrying one message per packet ([RFC5426](https://tools.ietf.org/html/rfc5426))\n\n## Installation\n\n```\ngo get github.com/influxdata/go-syslog/v3\n```\n\n## Docs\n\n[![Documentation](https://img.shields.io/badge/godoc-reference-blue.svg?style=for-the-badge)](http://godoc.org/github.com/influxdata/go-syslog)\n\nThe [docs](docs/) directory contains `.dot` files representing the finite-state machines (FSMs) implementing the syslog parsers and transports.\n\n## Usage\n\nSuppose you want to parse a given sequence of bytes as a RFC5424 message.\n\n_Notice that the same interface applies for RFC3164. But you can always take a look at the [examples file](./rfc3164/example_test.go)._\n\n```go\ni := []byte(`\u003c165\u003e4 2018-10-11T22:14:15.003Z mymach.it e - 1 [ex@32473 iut=\"3\"] An application event log entry...`)\np := rfc5424.NewParser()\nm, e := p.Parse(i)\n```\n\nThis results in `m` being equal to:\n\n```go\n// (*rfc5424.SyslogMessage)({\n//  Base: (syslog.Base) {\n//   Facility: (*uint8)(20),\n//   Severity: (*uint8)(5),\n//   Priority: (*uint8)(165),\n//   Timestamp: (*time.Time)(2018-10-11 22:14:15.003 +0000 UTC),\n//   Hostname: (*string)((len=9) \"mymach.it\"),\n//   Appname: (*string)((len=1) \"e\"),\n//   ProcID: (*string)(\u003cnil\u003e),\n//   MsgID: (*string)((len=1) \"1\"),\n//   Message: (*string)((len=33) \"An application event log entry...\")\n//  },\n//  Version: (uint16) 4,\n//  StructuredData: (*map[string]map[string]string)((len=1) {\n//   (string) (len=8) \"ex@32473\": (map[string]string) (len=1) {\n//    (string) (len=3) \"iut\": (string) (len=1) \"3\"\n//   }\n//  })\n// })\n```\n\nAnd `e` being equal to `nil` since the `i` byte slice contains a perfectly valid RFC5424 message.\n\n### Best effort mode\n\nRFC5424 parser has the ability to perform partial matches (until it can).\n\nWith this mode enabled, when the parsing process errors out it returns the message collected until that position, and the error that caused the parser to stop.\n\nNotice that in this modality the output is returned _iff_ it represents a minimally valid message - ie., a message containing almost a priority field in `[1,191]` within angular brackets, followed by a version in `]0,999]` (in the case of RFC5424).\n\nLet's look at an example.\n\n```go\ni := []byte(\"\u003c1\u003e1 A - - - - - -\")\np := NewParser(WithBestEffort())\nm, e := p.Parse(i)\n```\n\nThis results in `m` being equal to the following `SyslogMessage` instance.\n\n```go\n// (*rfc5424.SyslogMessage)({\n//  Base: (syslog.Base) {\n//   Facility: (*uint8)(0),\n//   Severity: (*uint8)(1),\n//   Priority: (*uint8)(1),\n//   Timestamp: (*time.Time)(\u003cnil\u003e),\n//   Hostname: (*string)(\u003cnil\u003e),\n//   Appname: (*string)(\u003cnil\u003e),\n//   ProcID: (*string)(\u003cnil\u003e),\n//   MsgID: (*string)(\u003cnil\u003e),\n//   Message: (*string)(\u003cnil\u003e)\n//  },\n//  Version: (uint16) 1,\n//  StructuredData: (*map[string]map[string]string)(\u003cnil\u003e)\n// })\n```\n\nAnd, at the same time, in `e` reporting the error that actually stopped the parser.\n\n```go\n// expecting a RFC3339MICRO timestamp or a nil value [col 5]\n```\n\nBoth `m` and `e` have a value since at the column the parser stopped it already was able to construct a minimally valid RFC5424 `SyslogMessage`.\n\n### Builder\n\nThis library also provides a builder to construct valid syslog messages.\n\nNotice that its API ignores input values that does not match the grammar.\n\nLet's have a look to an example.\n\n```go\nmsg := \u0026rfc5424.SyslogMessage{}\nmsg.SetTimestamp(\"not a RFC3339MICRO timestamp\")\nmsg.Valid() // Not yet a valid message (try msg.Valid())\nmsg.SetPriority(191)\nmsg.SetVersion(1)\nmsg.Valid() // Now it is minimally valid\n```\n\nPrinting `msg` you will verify it contains a `nil` timestamp (since an invalid one has been given).\n\n```go\n// (*rfc5424.SyslogMessage)({\n//  Base: (syslog.Base) {\n//   Facility: (*uint8)(23),\n//   Severity: (*uint8)(7),\n//   Priority: (*uint8)(191),\n//   Timestamp: (*time.Time)(\u003cnil\u003e),\n//   Hostname: (*string)(\u003cnil\u003e),\n//   Appname: (*string)(\u003cnil\u003e),\n//   ProcID: (*string)(\u003cnil\u003e),\n//   MsgID: (*string)(\u003cnil\u003e),\n//   Message: (*string)(\u003cnil\u003e)\n//  },\n//  Version: (uint16) 1,\n//  StructuredData: (*map[string]map[string]string)(\u003cnil\u003e)\n// })\n```\n\nFinally you can serialize the message into a string.\n\n```go\nstr, _ := msg.String()\n// \u003c191\u003e1 - - - - - -\n```\n\n## Message transfer\n\nExcluding encapsulating one message for packet in packet protocols there are two ways to transfer syslog messages over streams.\n\nThe older - ie., the **non-transparent** framing - and the newer one - ie., the **octet counting** framing - which is reliable and has not been seen to cause problems noted with the non-transparent one.\n\nThis library provide stream parsers for both.\n\n### Octet counting\n\nIn short, [RFC5425](https://tools.ietf.org/html/rfc5425#section-4.3) and [RFC6587](https://tools.ietf.org/html/rfc6587), aside from the protocol considerations, describe a **transparent framing** technique for Syslog messages that uses the **octect counting** technique - ie., the message length of the incoming message.\n\nEach Syslog message is sent with a prefix representing the number of bytes it is made of.\n\nThe [octecounting package](./octetcounting) parses messages stream following such rule.\n\nTo quickly understand how to use it please have a look at the [example file](./octetcounting/example_test.go).\n\n### Non transparent\n\nThe [RFC6587](https://tools.ietf.org/html/rfc6587#section-3.4.2) also describes the **non-transparent framing** transport of syslog messages.\n\nIn such case the messages are separated by a trailer, usually a line feed.\n\nThe [nontransparent package](./nontransparent) parses message stream following such [technique](https://tools.ietf.org/html/rfc6587#section-3.4.2).\n\nTo quickly understand how to use it please have a look at the [example file](./nontransparent/example_test.go).\n\nThings we do not support:\n\n- trailers other than `LF` or `NUL`\n- trailers which length is greater than 1 byte\n- trailer change on a frame-by-frame basis\n\n## Performances\n\nTo run the benchmark execute the following command.\n\n```bash\nmake bench\n```\n\nOn my machine\u003csup\u003e[1](#mymachine)\u003c/sup\u003e these are the results obtained paring RFC5424 syslog messages with best effort mode on.\n\n```\n[no]_empty_input__________________________________  4524100        274 ns/op      272 B/op        4 allocs/op\n[no]_multiple_syslog_messages_on_multiple_lines___  3039513        361 ns/op      288 B/op        8 allocs/op\n[no]_impossible_timestamp_________________________  1244562        951 ns/op      512 B/op       11 allocs/op\n[no]_malformed_structured_data____________________  2389249        512 ns/op      512 B/op        9 allocs/op\n[no]_with_duplicated_structured_data_id___________  1000000       1183 ns/op      712 B/op       17 allocs/op\n[ok]_minimal______________________________________  6876235        178 ns/op      227 B/op        5 allocs/op\n[ok]_average_message______________________________   730473       1653 ns/op     1520 B/op       24 allocs/op\n[ok]_complicated_message__________________________   908776       1344 ns/op     1264 B/op       24 allocs/op\n[ok]_very_long_message____________________________   392737       3114 ns/op     2448 B/op       25 allocs/op\n[ok]_all_max_length_and_complete__________________   510740       2431 ns/op     1872 B/op       28 allocs/op\n[ok]_all_max_length_except_structured_data_and_mes   755124       1593 ns/op      867 B/op       13 allocs/op\n[ok]_minimal_with_message_containing_newline______  6142984        199 ns/op      230 B/op        6 allocs/op\n[ok]_w/o_procid,_w/o_structured_data,_with_message  1670286        732 ns/op      348 B/op       10 allocs/op\n[ok]_minimal_with_UTF-8_message___________________  3013480        407 ns/op      339 B/op        6 allocs/op\n[ok]_minimal_with_UTF-8_message_starting_with_BOM_  2926410        423 ns/op      355 B/op        6 allocs/op\n[ok]_with_structured_data_id,_w/o_structured_data_  1558971        814 ns/op      570 B/op       11 allocs/op\n[ok]_with_multiple_structured_data________________  1000000       1243 ns/op     1205 B/op       16 allocs/op\n[ok]_with_escaped_backslash_within_structured_data  1000000       1025 ns/op      896 B/op       17 allocs/op\n[ok]_with_UTF-8_structured_data_param_value,_with_  1000000       1241 ns/op     1034 B/op       19 allocs/op\n```\n\nAs you can see it takes:\n\n* ~250ns to parse the smallest legal message\n\n* less than 2µs to parse an average legal message\n\n* ~3µs to parse a very long legal message\n\nOther RFC5424 implementations, like this [one](https://github.com/roguelazer/rust-syslog-rfc5424) in Rust, spend 8µs to parse an average legal message.\n\n_TBD: comparison against other Go parsers_.\n\n---\n\n* \u003ca name=\"mymachine\"\u003e[1]\u003c/a\u003e: Intel Core i7-8850H CPU @ 2.60GHz\n","funding_links":[],"categories":["Go","开源类库","Open source library"],"sub_categories":["日志","Logs"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finfluxdata%2Fgo-syslog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finfluxdata%2Fgo-syslog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finfluxdata%2Fgo-syslog/lists"}